Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features

Automatic detection and segmentation of kidneys in 3D CT images using random forests

Kidney segmentation in 3D CT images allows extracting useful information for nephrologists. For practical use in clinical routine, such an algorithm should be fast, automatic and robust to contrast-agent enhancement and fields of view. By combining and refining state-of-the-art techniques (random forests and template deformation), we demonstrate the possibility of building an algorithm that meets these requirements. Kidneys are localized with random forests following a coarse-to-fine strategy. Their initial positions detected with global contextual information are refined with a cascade of local regression forests. A classification forest is then used to obtain a probabilistic segmentation of both kidneys. The final segmentation is performed with an implicit template deformation algorithm driven by these kidney probability maps. Our method has been validated on a highly heterogeneous database of 233 CT scans from 89 patients. 80% of the kidneys were accurately detected and segmented (Dice coefficient > 0.90) in a few seconds per volume.

Automatic multi-model-based segmentation of the left atrium in cardiac MRI scans

Model-based segmentation approaches have been proven to produce very accurate segmentation results while simultaneously providing an anatomic labeling for the segmented structures. However, variations of the anatomy, as they are often encountered e.g. on the drainage pattern of the pulmonary veins to the left atrium, cannot be represented by a single model. Automatic model selection extends the model-based segmentation approach to handling significant variational anatomies without user interaction. Using models for the three most common anatomical variations of the left atrium, we propose a method that uses an estimation of the local fit of different models to select the best fitting model automatically. Our approach employs the support vector machine for the automatic model selection. The method was evaluated on 42 very accurate segmentations of MRI scans using three different models. The correct model was chosen in 88.1% of the cases. In a second experiment, reflecting average segmentation results, the model corresponding to the clinical classification was automatically found in 78.0% of the cases.

Data-driven breast decompression and lesion mapping from digital breast tomosynthesis

Digital breast tomosynthesis (DBT) emerges as a new 3D modality for breast cancer screening and diagnosis. Like in conventional 2D mammography the breast is scanned in a compressed state. For orientation during surgical planning, e.g., during presurgical ultrasound-guided anchor-wire marking, as well as for improving communication between radiologists and surgeons it is desirable to estimate an uncompressed model of the acquired breast along with a spatial mapping that allows localizing lesions marked in DBT in the uncompressed model. We therefore propose a method for 3D breast decompression and associated lesion mapping from 3D DBT data. The method is entirely data-driven and employs machine learning methods to predict the shape of the uncompressed breast from a DBT input volume. For this purpose a shape space has been constructed from manually annotated uncompressed breast surfaces and shape parameters are predicted by multiple multi-variate Random Forest regression. By exploiting point correspondences between the compressed and uncompressed breasts, lesions identified in DBT can be mapped to approximately corresponding locations in the uncompressed breast model. To this end, a thin-plate spline mapping is employed. Our method features a novel completely data-driven approach to breast shape prediction that does not necessitate prior knowledge about biomechanical properties and parameters of the breast tissue. Instead, a particular deformation behavior (decompression) is learned from annotated shape pairs, compressed and uncompressed, which are obtained from DBT and magnetic resonance image volumes, respectively. On average, shape prediction takes 26s and achieves a surface distance of 15.80 +/- 4.70 mm. The mean localization error for lesion mapping is 22.48 +/- 8.67 mm.

Automatic nuchal translucency measurement from ultrasonography

This paper proposes a fully automatic approach for computing Nuchal Translucency (NT) measurement in an ultrasound scans of the mid-sagittal plane of a fetal head. This is an improvement upon current NT measurement methods which require manual placement of NT measurement points or user-guidance in semi-automatic segmentation of the NT region. The algorithm starts by finding the pose of the fetal head using discriminative learning-based detectors. The fetal head serves as a robust anchoring structure and the NT region is estimated from the statistical relationship between the fetal head and the NT region. Next, the pose of the NT region is locally refined and its inner and outer edge approximately determined via Dijkstra's shortest path applied on the edge-enhanced image. Finally, these two region edges are used to define foreground and background seeds for accurate graph cut segmentation. The NT measurement is computed from the segmented region. Experiments show that the algorithm efficiently and effectively detects the NT region and provides accurate NT measurement which suggests suitability for clinical use.

Localisation of the brain in fetal MRI using bundled SIFT features

Fetal MRI is a rapidly emerging diagnostic imaging tool. Its main focus is currently on brain imaging, but there is a huge potential for whole body studies. We propose a method for accurate and robust localisation of the fetal brain in MRI when the image data is acquired as a stack of 2D slices misaligned due to fetal motion. We first detect possible brain locations in 2D images with a Bag-of-Words model using SIFT features aggregated within Maximally Stable Extremal Regions (called bundled SIFT), followed by a robust fitting of an axis-aligned 3D box to the selected regions. We rely on prior knowledge of the fetal brain development to define size and shape constraints. In a cross-validation experiment, we obtained a median error distance of 5.7mm from the ground truth and no missed detection on a database of 59 fetuses. This 2D approach thus allows a robust detection even in the presence of substantial fetal motion.

Spine detection in CT and MR using iterated marginal space learning

Examinations of the spinal column with both, Magnetic Resonance (MR) imaging and Computed Tomography (CT), often require a precise three-dimensional positioning, angulation and labeling of the spinal disks and the vertebrae. A fully automatic and robust approach is a prerequisite for an automated scan alignment as well as for the segmentation and analysis of spinal disks and vertebral bodies in Computer Aided Diagnosis (CAD) applications. In this article, we present a novel method that combines Marginal Space Learning (MSL), a recently introduced concept for efficient discriminative object detection, with a generative anatomical network that incorporates relative pose information for the detection of multiple objects. It is used to simultaneously detect and label the spinal disks. While a novel iterative version of MSL is used to quickly generate candidate detections comprising position, orientation, and scale of the disks with high sensitivity, the anatomical network selects the most likely candidates using a learned prior on the individual nine dimensional transformation spaces. Finally, we propose an optional case-adaptive segmentation approach that allows to segment the spinal disks and vertebrae in MR and CT respectively. Since the proposed approaches are learning-based, they can be trained for MR or CT alike. Experimental results based on 42 MR and 30 CT volumes show that our system not only achieves superior accuracy but also is among the fastest systems of its kind in the literature. On the MR data set the spinal disks of a whole spine are detected in 11.5s on average with 98.6% sensitivity and 0.073 false positive detections per volume. On the CT data a comparable sensitivity of 98.0% with 0.267 false positives is achieved. Detected disks are localized with an average position error of 2.4 mm/3.2 mm and angular error of 3.9°/4.5° in MR/CT, which is close to the employed hypothesis resolution of 2.1 mm and 3.3°.

Automatic aorta segmentation and valve landmark detection in C-arm CT for transcatheter aortic valve implantation

Transcatheter aortic valve implantation (TAVI) is a minimally invasive procedure to treat severe aortic valve stenosis. As an emerging imaging technique, C-arm computed tomography (CT) plays a more and more important role in TAVI on both pre-operative surgical planning (e.g., providing 3-D valve measurements) and intra-operative guidance (e.g., determining a proper C-arm angulation). Automatic aorta segmentation and aortic valve landmark detection in a C-arm CT volume facilitate the seamless integration of C-arm CT into the TAVI workflow and improve the patient care. In this paper, we present a part-based aorta segmentation approach, which can handle structural variation of the aorta in case that the aortic arch and descending aorta are missing in the volume. The whole aorta model is split into four parts: aortic root, ascending aorta, aortic arch, and descending aorta. Discriminative learning is applied to train a detector for each part separately to exploit the rich domain knowledge embedded in an expert-annotated dataset. Eight important aortic valve landmarks (three hinges, three commissures, and two coronary ostia) are also detected automatically with an efficient hierarchical approach. Our approach is robust under all kinds of variations observed in a real clinical setting, including changes in the field-of-view, contrast agent injection, scan timing, and aortic valve regurgitation. Taking about 1.1 s to process a volume, it is also computationally efficient. Under the guidance of the automatically extracted patient-specific aorta model, the physicians can properly determine the C-arm angulation and deploy the prosthetic valve. Promising outcomes have been achieved in real clinical applications.

Computational modelling of the right ventricle in repaired tetralogy of Fallot: can it provide insight into patient treatment?

AIMS

Pulmonary regurgitation (PR) causes progressive right ventricle (RV) dilatation and dysfunction in repaired tetralogy of Fallot (rToF). Declining RV function is often insidious and the timing of pulmonary valve replacement remains under debate. Quantifying the pathophysiology of adverse RV remodelling due to worsening PR may help in defining the best timing for pulmonary valve replacement. Our aim was to identify whether complex three-dimensional (3D) deformations of RV shape, as assessed with computer modelling, could constitute an anatomical biomarker that correlated with clinical parameters in rToF patients.

METHODS AND RESULTS

We selected 38 rToF patients (aged 10-30 years) who had complete data sets and had not undergone PVR from a population of 314 consecutive patients recruited in a collaborative study of four hospitals. All patients underwent cardiovascular magnetic resonance (CMR) imaging: PR and RV end-diastolic volumes were measured. An unbiased shape analysis framework was used with principal component analysis and linear regression to correlate shape with indexed PR volume. Regurgitation severity was significantly associated with RV dilatation (P = 0.01) and associated with bulging of the outflow tract (P = 0.07) and a dilatation of the apex (P = 0.08).

CONCLUSION

In this study, we related RV shape at end-diastole to clinical metrics of PR in rToF patients. By considering the entire 3D shape, we identified a link between PR and RV dilatation, outflow tract bulging, and apical dilatation. Our study constitutes a first attempt to correlate 3D RV shape with clinical metrics in rToF, opening new ways to better quantify 3D RV change in rToF.

A framework of whole heart extracellular volume fraction estimation for low-dose cardiac CT images

Cardiac CT (CCT) is widely available and has been validated for the detection of focal myocardial scar using a delayed enhancement technique in this paper. CCT, however, has not been previously evaluated for quantification of diffuse myocardial fibrosis. In our investigation, we sought to evaluate the potential of low-dose CCT for the measurement of myocardial whole heart extracellular volume (ECV) fraction. ECV is altered under conditions of increased myocardial fibrosis. A framework consisting of three main steps was proposed for CCT whole heart ECV estimation. First, a shape-constrained graph cut (GC) method was proposed for myocardium and blood pool segmentation on postcontrast image. Second, the symmetric demons deformable registration method was applied to register precontrast to postcontrast images. So the correspondences between the voxels from precontrast to postcontrast images were established. Finally, the whole heart ECV value was computed. The proposed method was tested on 20 clinical low-dose CCT datasets with precontrast and postcontrast images. The preliminary results demonstrated the feasibility and efficiency of the proposed method.

Fully automated right ventricular volumetry from ECG-gated coronary CT angiography data: evaluation of prototype software

Enlargement and dysfunction of the right ventricle (RV) is a sign and outcome predictor of many cardiopulmonary diseases. Due to the complex geometry of the RV exact volumetry is cumbersome and time-consuming. We evaluated the performance of prototype software for fully automated RV segmentation and volumetry from cardiac CT data. In 50 retrospectively ECG-gated coronary CT angiography scans the endsystolic (RVVmin) and enddiastolic (RVVmax) volume of the right ventricle was calculated fully automatically by prototype software. Manual slice segmentation by two independent radiologists served as the reference standard. Measurement periods were compared for both methods. RV volumes calculated with the software were in strong agreement with the results from manual slice segmentation (Bland-Altman r = 0.95-0.98; p < 0.001; Lin's correlation Rho = 0.87-0.96, p < 0.001) for RVVmax and RVVmin with excellent interobserver agreement between both radiologists (r = 0.97; p < 0.001). The measurement period was significantly shorter with the software (153 ± 9 s) than with manual slice segmentation (658 ± 211 s). The prototype software demonstrated very good performance in comparison to the reference standard. It promises robust RV volume results and minimizes postprocessing time.

Feasibility, accuracy, and reproducibility of real-time full-volume 3D transthoracic echocardiography to measure LV volumes and systolic function: a fully automated endocardial contouring algorithm in sinus rhythm and atrial fibrillation

OBJECTIVES

To assess the feasibility, accuracy, and reproducibility of real-time full-volume 3-dimensional transthoracic echocardiography (3D RT-VTTE) to measure left ventricular (LV) volumes and ejection fraction (EF) using a fully automated endocardial contouring algorithm and to identify and automatically correct the contours to obtain accurate LV volumes in sinus rhythm and atrial fibrillation (AF).

BACKGROUND

3D transthoracic echocardiography is not used routinely to quantify LV volumes and EF. A fully automated workflow using RT-VTTE may improve clinical adoption.

METHODS

RT-VTTE was performed and 3D EF and volumes obtained using an automated trabecular endocardial contouring algorithm; an automated correction was applied to track the compacted myocardium. Cardiac magnetic resonance (CMR) and 2-dimensional biplane Simpson method were the reference standard.

RESULTS

Ninety-one patients (67 in normal sinus rhythm [NSR], 24 in AF) were included. Among all NSR patients, there was excellent correlation between RT-VTTE and CMR for end-diastolic volume (EDV), end-systolic volume (ESV), and EF (r = 0.90, 0.96, and 0.98, respectively; p < 0.001). In patients with EF ≥50% (n = 36), EDV and ESV were underestimated by 10.7 ± 17.5 ml (p = 0.001) and by 4.1 ± 6.1 ml (p < 0.001), respectively. In those with EF <50% (n = 31), EDV and ESV were underestimated by 25.7 ± 32.7 ml (p < 0.001) and by 16.2 ± 24.0 ml (p = 0.001). Automated contour correction to track the compacted myocardium eliminated mean volume differences between RT-VTTE and CMR. In patients with AF, LV volumes and EF were accurate by RT-VTTE (r = 0.94, 0.94, and 0.91 for EDV, ESV, and EF, respectively; p < 0.001). Automated 3D LV volumes and EF were highly reproducible.

CONCLUSIONS

Rapid, accurate, and reproducible EF can be obtained by RT-VTTE in NSR and AF patients by using an automated trabecular edge contouring algorithm. Furthermore, automated contour correction to detect the compacted myocardium yields accurate and reproducible 3D LV volumes.

An integrated framework for finite-element modeling of mitral valve biomechanics from medical images: application to MitralClip intervention planning

Treatment of mitral valve (MV) diseases requires comprehensive clinical evaluation and therapy personalization to optimize outcomes. Finite-element models (FEMs) of MV physiology have been proposed to study the biomechanical impact of MV repair, but their translation into the clinics remains challenging. As a step towards this goal, we present an integrated framework for finite-element modeling of the MV closure based on patient-specific anatomies and boundary conditions. Starting from temporal medical images, we estimate a comprehensive model of the MV apparatus dynamics, including papillary tips, using a machine-learning approach. A detailed model of the open MV at end-diastole is then computed, which is finally closed according to a FEM of MV biomechanics. The motion of the mitral annulus and papillary tips are constrained from the image data for increased accuracy. A sensitivity analysis of our system shows that chordae rest length and boundary conditions have a significant influence upon the simulation results. We quantitatively test the generalization of our framework on 25 consecutive patients. Comparisons between the simulated closed valve and ground truth show encouraging results (average point-to-mesh distance: 1.49 ± 0.62 mm) but also the need for personalization of tissue properties, as illustrated in three patients. Finally, the predictive power of our model is tested on one patient who underwent MitralClip by comparing the simulated intervention with the real outcome in terms of MV closure, yielding promising prediction. By providing an integrated way to perform MV simulation, our framework may constitute a surrogate tool for model validation and therapy planning.

Regression-based cardiac motion prediction from single-phase CTA

State of the art cardiac computed tomography (CT) enables the acquisition of imaging data of the heart over the entire cardiac cycle at concurrent high spatial and temporal resolution. However, in clinical practice, acquisition is increasingly limited to 3-D images. Estimating the shape of the cardiac structures throughout the entire cardiac cycle from a 3-D image is therefore useful in applications such as the alignment of preoperative computed tomography angiography (CTA) to intra-operative X-ray images for improved guidance in coronary interventions. We hypothesize that the motion of the heart is partially explained by its shape and therefore investigate the use of three regression methods for motion estimation from single-phase shape information. Quantitative evaluation on 150 4-D CTA images showed a small, but statistically significant, increase in the accuracy of the predicted shape sequences when using any of the regression methods, compared to shape-independent motion prediction by application of the mean motion. The best results were achieved using principal component regression resulting in point-to-point errors of 2.3±0.5 mm, compared to values of 2.7±0.6 mm for shape-independent motion estimation. Finally, we showed that this significant difference withstands small variations in important parameter settings of the landmarking procedure.

Multi-compartment heart segmentation in CT angiography using a spatially varying gaussian classifier

OBJECTIVE

A fully automated and efficient method for segmenting ten major structures within the heart in Cardiac CT Angiography data for the purposes of display or cardiac functional analysis.

MATERIALS AND METHODS

A spatially varying Gaussian classifier is a flexible model for segmentation, combining the advantages of atlas-based frameworks, with supervised intensity models. It is composed of an independent Gaussian classifier at each voxel and uses non-rigid registration for the initial spatial alignment. We show how this large model can be trained efficiently and present a novel smoothing technique based on normalised convolution to mitigate inherent overfitting issues. The 30 datasets used in this study are selected from a variety of different scanners in order to test the robustness and stability of the algorithm. The datasets were manually segmented by a trained clinician.

RESULTS

The method was evaluated in a leave-one-out fashion, and the results were compared to other state of the art methods in the field, with a mean surface-to-surface distance of between 0.61 and 2.12 mm for different compartments.

CONCLUSION

The accuracy of this method is comparable to other state of the art methods in the field. Its benefits lie in its conceptual simplicity and its general applicability. Only one non-rigid registration is required, giving it a speed advantage over multi-atlas approaches. Further accuracy may be achievable through the incorporation of an explicit shape model.

Automatic model-based contour detection of left ventricle myocardium from cardiac CT images

PURPOSE

For accurate evaluation of myocardial perfusion on computed tomography images, precise identification of the myocardial borders of the left ventricle (LV) is mandatory. In this article, we propose a method to detect the contour of LV myocardium automatically and accurately.

METHODS

Our detection method is based on active shape model. For precise detection, we estimate the pose and shape parameters separately by three steps: LV coordinate system estimation, myocardial shape estimation, and transformation. In LV coordinate system estimation, we detect heart features followed by the entire LV by introducing machine-learning approach. Since the combination of two types feature detection covers the LV variation, such as pose or shape, we can estimate the LV coordinate system robustly. In myocardial shape estimation, we minimize the energy function including pattern error around myocardium with adjustment of pattern model to input image using estimated concentration of contrast dye. Finally, we detect LV myocardial contours in the input images by transforming the estimated myocardial shape using the matrix composed of the vectors calculated by the LV coordinate system estimation.

RESULTS

In our experiments with 211 images from 145 patients, mean myocardial contours point-to-point errors for our method as compared to ground truth were 1.02 mm for LV endocardium and 1.07 mm for LV epicardium. The average computation time was 2.4 s (on a 3.46 GHz processor with 2-multithreading process).

CONCLUSIONS

Our method achieved accurate and fast myocardial contour detection which may be sufficient for myocardial perfusion examination.

Left ventricular myocardium segmentation on delayed phase of multi-detector row computed tomography

RATIONALE AND OBJECTIVES

Advanced ischemic heart disease is usually accompanied by left ventricular (LV) myocardial volume loss and an abnormal enhancing pattern on delayed phase of multi-detector row computed tomography (MDCT). To assist radiologists and physicians in estimating the LV myocardial volume on delayed phase, this paper proposes an adaptive segmentation method for contouring the myocardial region in the delayed-phase MDCT and for computing the volume.

MATERIALS AND METHODS

The proposed method uses an anisotropic diffusion filter as a preprocessing procedure to enhance contrast and reduce specks in MDCT imaging. This work picks the middle of mid-ventricular level image slices as the lead slice. The proposed method develops two contouring modes to sketch the myocardium contour on the lead slice. By establishing the obtained contours as the initial contours, the region-growing method is employed to identify the contour of the myocardial region for each slice. The convex-hull finding algorithm is then used to refine the extracted contour. Finally, the width properties of the myocardial region and the morphological operators are used to obtain the entire LV myocardial volume.

RESULTS

Twenty-seven healthy patients who had no symptoms of ischemic heart disease are examined to evaluate the performance of the proposed method. Compared with manual contours delineated by two experienced experts, the contouring results using computer simulation reveal that the proposed method reliably identifies contours similar to those obtained using manual sketching.

CONCLUSION

The proposed method provides robust contouring for the LV myocardium on delayed-phase MDCT. The potential role of this technique may substantially reduce the time required to sketch manually a precise contour with high stability.

Automatic detection and quantification of tree-in-bud (TIB) opacities from CT scans

This study presents a novel computer-assisted detection (CAD) system for automatically detecting and precisely quantifying abnormal nodular branching opacities in chest computed tomography (CT), termed tree-in-bud (TIB) opacities by radiology literature. The developed CAD system in this study is based on 1) fast localization of candidate imaging patterns using local scale information of the images, and 2) Möbius invariant feature extraction method based on learned local shape and texture properties of TIB patterns. For fast localization of candidate imaging patterns, we use ball-scale filtering and, based on the observation of the pattern of interest, a suitable scale selection is used to retain only small size patterns. Once candidate abnormality patterns are identified, we extract proposed shape features from regions where at least one candidate pattern occupies. The comparative evaluation of the proposed method with commonly used CAD methods is presented with a dataset of 60 chest CTs (laboratory confirmed 39 viral bronchiolitis human parainfluenza CTs and 21 normal chest CTs). The quantitative results are presented as the area under the receiver operator characteristics curves and a computer score (volume affected by TIB) provided as an output of the CAD system. In addition, a visual grading scheme is applied to the patient data by three well-trained radiologists. Interobserver and observer-computer agreements are obtained by the relevant statistical methods over different lung zones. Experimental results demonstrate that the proposed CAD system can achieve high detection rates with an overall accuracy of 90.96%. Moreover, correlations of observer-observer (R(2)=0.8848, and observer-CAD agreements (R(2)=0.824, validate the feasibility of the use of the proposed CAD system in detecting and quantifying TIB patterns.

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.

The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

We present a new supervised learning model designed for the automatic segmentation of the left ventricle (LV) of the heart in ultrasound images. We address the following problems inherent to supervised learning models: 1) the need of a large set of training images; 2) robustness to imaging conditions not present in the training data; and 3) complex search process. The innovations of our approach reside in a formulation that decouples the rigid and nonrigid detections, deep learning methods that model the appearance of the LV, and efficient derivative-based search algorithms. The functionality of our approach is evaluated using a data set of diseased cases containing 400 annotated images (from 12 sequences) and another data set of normal cases comprising 80 annotated images (from two sequences), where both sets present long axis views of the LV. Using several error measures to compute the degree of similarity between the manual and automatic segmentations, we show that our method not only has high sensitivity and specificity but also presents variations with respect to a gold standard (computed from the manual annotations of two experts) within interuser variability on a subset of the diseased cases. We also compare the segmentations produced by our approach and by two state-of-the-art LV segmentation models on the data set of normal cases, and the results show that our approach produces segmentations that are comparable to these two approaches using only 20 training images and increasing the training set to 400 images causes our approach to be generally more accurate. Finally, we show that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations. In the future, we plan to include a dynamical model to improve the performance of the algorithm, to use semisupervised learning methods to reduce even more the dependence on rich and large training sets, and to design a shape model less dependent on the training set.

Automatic detection and segmentation of lymph nodes from CT data

Lymph nodes are assessed routinely in clinical practice and their size is followed throughout radiation or chemotherapy to monitor the effectiveness of cancer treatment. This paper presents a robust learning-based method for automatic detection and segmentation of solid lymph nodes from CT data, with the following contributions. First, it presents a learning based approach to solid lymph node detection that relies on marginal space learning to achieve great speedup with virtually no loss in accuracy. Second, it presents a computationally efficient segmentation method for solid lymph nodes (LN). Third, it introduces two new sets of features that are effective for LN detection, one that self-aligns to high gradients and another set obtained from the segmentation result. The method is evaluated for axillary LN detection on 131 volumes containing 371 LN, yielding a 83.0% detection rate with 1.0 false positive per volume. It is further evaluated for pelvic and abdominal LN detection on 54 volumes containing 569 LN, yielding a 80.0% detection rate with 3.2 false positives per volume. The running time is 5-20 s per volume for axillary areas and 15-40 s for pelvic. An added benefit of the method is the capability to detect and segment conglomerated lymph nodes.

Similarity enhancement for automatic segmentation of cardiac structures in computed tomography volumes

The aim of this research is proposing a 3-D similarity enhancement technique useful for improving the segmentation of cardiac structures in Multi-Slice Computerized Tomography (MSCT) volumes. The similarity enhancement is obtained by subtracting the intensity of the current voxel and the gray levels of their adjacent voxels in two volumes resulting after preprocessing. Such volumes are: a. - a volume obtained after applying a Gaussian distribution and a morphological top-hat filter to the input and b. - a smoothed volume generated by processing the input with an average filter. Then, the similarity volume is used as input to a region growing algorithm. This algorithm is applied to extract the shape of cardiac structures, such as left and right ventricles, in MSCT volumes. Qualitative and quantitative results show the good performance of the proposed approach for discrimination of cardiac cavities.

Precise segmentation of multiple organs in CT volumes using learning-based approach and information theory

In this paper, we present a novel method by incorporating information theory into the learning-based approach for automatic and accurate pelvic organ segmentation (including the prostate, bladder and rectum). We target 3D CT volumes that are generated using different scanning protocols (e.g., contrast and non-contrast, with and without implant in the prostate, various resolution and position), and the volumes come from largely diverse sources (e.g., diseased in different organs). Three key ingredients are combined to solve this challenging segmentation problem. First, marginal space learning (MSL) is applied to efficiently and effectively localize the multiple organs in the largely diverse CT volumes. Second, learning techniques, steerable features, are applied for robust boundary detection. This enables handling of highly heterogeneous texture pattern. Third, a novel information theoretic scheme is incorporated into the boundary inference process. The incorporation of the Jensen-Shannon divergence further drives the mesh to the best fit of the image, thus improves the segmentation performance. The proposed approach is tested on a challenging dataset containing 188 volumes from diverse sources. Our approach not only produces excellent segmentation accuracy, but also runs about eighty times faster than previous state-of-the-art solutions. The proposed method can be applied to CT images to provide visual guidance to physicians during the computer-aided diagnosis, treatment planning and image-guided radiotherapy to treat cancers in pelvic region.

Evaluating segmentation error without ground truth

The automatic delineation of the boundaries of organs and other anatomical structures is a key component of many medical image processing systems. In this paper we present a generic learning approach based on a novel space of segmentation features, which can be trained to predict the overlap error and Dice coefficient of an arbitrary organ segmentation without knowing the ground truth delineation. We show the regressor to be much stronger a predictor of these error metrics than the responses of probabilistic boosting classifiers trained on the segmentation boundary. The presented approach not only allows us to build reliable confidence measures and fidelity checks, but also to rank several segmentation hypotheses against each other during online usage of the segmentation algorithm in clinical practice.

Robust automatic knee MR slice positioning through redundant and hierarchical anatomy detection

Diagnostic magnetic resonance (MR) image quality is highly dependent on the position and orientation of the slice groups, due to the intrinsic high in-slice and low through-slice resolutions of MR imaging. Hence, the higher speed, accuracy, and reproducibility of automatic slice positioning, make it highly desirable over manual slice positioning. However, imaging artifacts, diseases, joint articulation, variations across ages and demographics as well as the extremely high performance requirements prevent state-of-the-art methods, such as volumetric registration, to be an off-the-shelf solution. In this paper, we address all these issues through an automatic slice positioning framework based on redundant and hierarchical learning. Our method has two hallmarks that are specifically designed to achieve high robustness and accuracy. 1) A redundant set of anatomy detectors are learned to provide local appearance cues. These detections are pruned and assembled according to a distributed anatomy model, which captures group-wise spatial configurations among anatomy primitives. This strategy brings about a high level of robustness and works even if a large portion of the target is distorted, missing, or occluded. 2) The detectors are learned and invoked in a hierarchical fashion, with each local detection scheduled and iterated according to its intrinsic invariance property. This iterative alignment process is shown to dramatically improve alignment accuracy. The proposed system is extensively validated on a large dataset including 744 clinical MR scans. Compared to state-of-the-art methods, our method exhibits superior performance in terms of robustness, accuracy, and reproducibility. The methodology is general and can be applied to other anatomies and other imaging modalities.

Multi-part left atrium modeling and segmentation in C-arm CT volumes for atrial fibrillation ablation

As a minimally invasive surgery to treat left atrial (LA) fibrillation, catheter based ablation uses high radio-frequency energy to eliminate potential sources of the abnormal electrical events, especially around the ostia of pulmonary veins (PV). Due to large structural variations of the PV drainage pattern, a personalized LA model is helpful to translate a generic ablation strategy to a specific patient's anatomy. Overlaying the LA model onto 2D fluoroscopic images provides valuable visual guidance during surgery. A holistic shape model is not accurate enough to represent the whole shape population of the LA. In this paper, we propose a part based LA model (including the chamber, appendage, and four major PVs) and each part is a much simpler anatomical structure compared to the holistic one. Our approach works on un-gated C-arm CT, where thin boundaries between the LA blood pool and surrounding tissues are often blurred due to the cardiac motion artifacts (which presents a big challenge compared to the highly contrasted gated CT/MRI). To avoid segmentation leakage, the shape prior is exploited in a model based approach to segment the LA parts. However, independent detection of each part is not optimal and its robustness needs further improvement (especially for the appendage and PVs). We propose to enforce a statistical shape constraint during the estimation of pose parameters (position, orientation, and size) of different parts. Our approach is computationally efficient, taking about 1.5 s to process a volume with 256 x 256 x 250 voxels. Experiments on 469 C-arm CT datasets demonstrate its robustness.

Automatic multi-organ segmentation using learning-based segmentation and level set optimization

We present a novel generic segmentation system for the fully automatic multi-organ segmentation from CT medical images. Thereby we combine the advantages of learning-based approaches on point cloud-based shape representation, such a speed, robustness, point correspondences, with those of PDE-optimization-based level set approaches, such as high accuracy and the straightforward prevention of segment overlaps. In a benchmark on 10-100 annotated datasets for the liver, the lungs, and the kidneys we show that the proposed system yields segmentation accuracies of 1.17-2.89 mm average surface errors. Thereby the level set segmentation (which is initialized by the learning-based segmentations) contributes with an 20%-40% increase in accuracy.

Prediction based collaborative trackers (PCT): a robust and accurate approach toward 3D medical object tracking

Robust and fast 3D tracking of deformable objects, such as heart, is a challenging task because of the relatively low image contrast and speed requirement. Many existing 2D algorithms might not be directly applied on the 3D tracking problem. The 3D tracking performance is limited due to dramatically increased data size, landmarks ambiguity, signal drop-out or complex nonrigid deformation. In this paper, we present a robust, fast, and accurate 3D tracking algorithm: prediction based collaborative trackers (PCT). A novel one-step forward prediction is introduced to generate the motion prior using motion manifold learning. Collaborative trackers are introduced to achieve both temporal consistency and failure recovery. Compared with tracking by detection and 3D optical flow, PCT provides the best results. The new tracking algorithm is completely automatic and computationally efficient. It requires less than 1.5 s to process a 3D volume which contains millions of voxels. In order to demonstrate the generality of PCT, the tracker is fully tested on three large clinical datasets for three 3D heart tracking problems with two different imaging modalities: endocardium tracking of the left ventricle (67 sequences, 1134 3D volumetric echocardiography data), dense tracking in the myocardial regions between the epicardium and endocardium of the left ventricle (503 sequences, roughly 9000 3D volumetric echocardiography data), and whole heart four chambers tracking (20 sequences, 200 cardiac 3D volumetric CT data). Our datasets are much larger than most studies reported in the literature and we achieve very accurate tracking results compared with human experts' annotations and recent literature.

Automatic extraction of 3D dynamic left ventricle model from 2D rotational angiocardiogram

In this paper, we propose an automatic method to directly extract 3D dynamic left ventricle (LV) model from sparse 2D rotational angiocardiogram (each cardiac phase contains only five projections). The extracted dynamic model provides quantitative cardiac function for analysis. The overlay of the model onto 2D real-time fluoroscopic images provides valuable visual guidance during cardiac intervention. Though containing severe cardiac motion artifacts, an ungated CT reconstruction is used in our approach to extract a rough static LV model. The initialized LV model is projected onto each 2D projection image. The silhouette of the projected mesh is deformed to match the boundary of LV blood pool. The deformation vectors of the silhouette are back-projected to 3D space and used as anchor points for thin plate spline (TPS) interpolation of other mesh points. The proposed method is validated on 12 synthesized datasets. The extracted 3D LV meshes match the ground truth quite well with a mean point-to-mesh error of 0.51 +/- 0.11 mm. The preliminary experiments on two real datasets (included a patient and a pig) show promising results too.

A statistical model for quantification and prediction of cardiac remodelling: application to tetralogy of Fallot

Cardiac remodelling plays a crucial role in heart diseases. Analyzing how the heart grows and remodels over time can provide precious insights into pathological mechanisms, eventually resulting in quantitative metrics for disease evaluation and therapy planning. This study aims to quantify the regional impacts of valve regurgitation and heart growth upon the end-diastolic right ventricle (RV) in patients with tetralogy of Fallot, a severe congenital heart defect. The ultimate goal is to determine, among clinical variables, predictors for the RV shape from which a statistical model that predicts RV remodelling is built. Our approach relies on a forward model based on currents and a diffeomorphic surface registration algorithm to estimate an unbiased template. Local effects of RV regurgitation upon the RV shape were assessed with Principal Component Analysis (PCA) and cross-sectional multivariate design. A generative 3-D model of RV growth was then estimated using partial least squares (PLS) and canonical correlation analysis (CCA). Applied on a retrospective population of 49 patients, cross-effects between growth and pathology could be identified. Qualitatively, the statistical findings were found realistic by cardiologists. 10-fold cross-validation demonstrated a promising generalization and stability of the growth model. Compared to PCA regression, PLS was more compact, more precise and provided better predictions.

Patient-specific electromechanical models of the heart for the prediction of pacing acute effects in CRT: a preliminary clinical validation

Cardiac resynchronisation therapy (CRT) is an effective treatment for patients with congestive heart failure and a wide QRS complex. However, up to 30% of patients are non-responders to therapy in terms of exercise capacity or left ventricular reverse remodelling. A number of controversies still remain surrounding patient selection, targeted lead implantation and optimisation of this important treatment. The development of biophysical models to predict the response to CRT represents a potential strategy to address these issues. In this article, we present how the personalisation of an electromechanical model of the myocardium can predict the acute haemodynamic changes associated with CRT. In order to introduce such an approach as a clinical application, we needed to design models that can be individualised from images and electrophysiological mapping of the left ventricle. In this paper the personalisation of the anatomy, the electrophysiology, the kinematics and the mechanics are described. The acute effects of pacing on pressure development were predicted with the in silico model for several pacing conditions on two patients, achieving good agreement with invasive haemodynamic measurements: the mean error on dP/dt(max) is 47.5±35mmHgs(-1), less than 5% error. These promising results demonstrate the potential of physiological models personalised from images and electrophysiology signals to improve patient selection and plan CRT.

Segmentation of the heart and great vessels in CT images using a model-based adaptation framework

Recently, model-based methods for the automatic segmentation of the heart chambers have been proposed. An important application of these methods is the characterization of the heart function. Heart models are, however, increasingly used for interventional guidance making it necessary to also extract the attached great vessels. It is, for instance, important to extract the left atrium and the proximal part of the pulmonary veins to support guidance of ablation procedures for atrial fibrillation treatment. For cardiac resynchronization therapy, a heart model including the coronary sinus is needed. We present a heart model comprising the four heart chambers and the attached great vessels. By assigning individual linear transformations to the heart chambers and to short tubular segments building the great vessels, variable sizes of the heart chambers and bending of the vessels can be described in a consistent way. A configurable algorithmic framework that we call adaptation engine matches the heart model automatically to cardiac CT angiography images in a multi-stage process. First, the heart is detected using a Generalized Hough Transformation. Subsequently, the heart chambers are adapted. This stage uses parametric as well as deformable mesh adaptation techniques. In the final stage, segments of the large vascular structures are successively activated and adapted. To optimize the computational performance, the adaptation engine can vary the mesh resolution and freeze already adapted mesh parts. The data used for validation were independent from the data used for model-building. Ground truth segmentations were generated for 37 CT data sets reconstructed at several cardiac phases from 17 patients. Segmentation errors were assessed for anatomical sub-structures resulting in a mean surface-to-surface error ranging 0.50-0.82mm for the heart chambers and 0.60-1.32mm for the parts of the great vessels visible in the images.

Patient-specific modelling of whole heart anatomy, dynamics and haemodynamics from four-dimensional cardiac CT images

There is a growing need for patient-specific and holistic modelling of the heart to support comprehensive disease assessment and intervention planning as well as prediction of therapeutic outcomes. We propose a patient-specific model of the whole human heart, which integrates morphology, dynamics and haemodynamic parameters at the organ level. The modelled cardiac structures are robustly estimated from four-dimensional cardiac computed tomography (CT), including all four chambers and valves as well as the ascending aorta and pulmonary artery. The patient-specific geometry serves as an input to a three-dimensional Navier-Stokes solver that derives realistic haemodynamics, constrained by the local anatomy, along the entire heart cycle. We evaluated our framework with various heart pathologies and the results correlate with relevant literature reports.

Evaluation of a multi-atlas based method for segmentation of cardiac CTA data: a large-scale, multicenter, and multivendor study

PURPOSE

Computed tomography angiography (CTA) is increasingly used for the diagnosis of coronary artery disease (CAD). However, CTA is not commonly used for the assessment of ventricular and atrial function, although functional information extracted from CTA data is expected to improve the diagnostic value of the examination. In clinical practice, the extraction of ventricular and atrial functional information, such as stroke volume and ejection fraction, requires accurate delineation of cardiac chambers. In this paper, we investigated the accuracy and robustness of cardiac chamber delineation using a multiatlas based segmentation method on multicenter and multivendor CTA data.

METHODS

A fully automatic multiatlas based method for segmenting the whole heart (i.e., the outer surface of the pericardium) and cardiac chambers from CTA data is presented and evaluated. In the segmentation approach, eight atlas images are registered to a new patient's CTA scan. The eight corresponding manually labeled images are then propagated and combined using a per voxel majority voting procedure, to obtain a cardiac segmentation.

RESULTS

The method was evaluated on a multicenter/multivendor database, consisting of (1) a set of 1380 Siemens scans from 795 patients and (2) a set of 60 multivendor scans (Siemens, Philips, and GE) from different patients, acquired in six different institutions worldwide. A leave-one-out 3D quantitative validation was carried out on the eight atlas images; we obtained a mean surface-to-surface error of 0.94 +/- 1.12 mm and an average Dice coefficient of 0.93 was achieved. A 2D quantitative evaluation was performed on the 60 multivendor data sets. Here, we observed a mean surface-to-surface error of 1.26 +/- 1.25 mm and an average Dice coefficient of 0.91 was achieved. In addition to this quantitative evaluation, a large-scale 2D and 3D qualitative evaluation was performed on 1380 and 140 images, respectively. Experts evaluated that 49% of the 1380 images were very accurately segmented (below 1 mm error) and that 29% were accurately segmented (error between 1 and 3 mm), which demonstrates the robustness of the presented method.

CONCLUSIONS

A fully automatic method for whole heart and cardiac chamber segmentation was presented and evaluated using multicenter/multivendor CTA data. The accuracy and robustness of the method were demonstrated by successfully applying the method to 1420 multicenter/ multivendor data sets.

Robust physically-constrained modeling of the mitral valve and subvalvular apparatus

Mitral valve (MV) is often involved in cardiac diseases, with various pathological patterns that require a systemic view of the entire MV apparatus. Due to its complex shape and dynamics, patient-specific modeling of the MV constitutes a particular challenge. We propose a novel approach for personalized modeling of the dynamic MV and its subvalvular apparatus that ensures temporal consistency over the cardiac sequence and provides realistic deformations. The idea is to detect the anatomical MV components under constraints derived from the biomechanical properties of the leaflets. This is achieved by a robust two-step alternate algorithm that combines discriminative learning and leaflet biomechanics. Extensive evaluation on 200 transesophageal echochardiographic sequences showed an average Hausdorff error of 5.1 mm at a speed of 9 sec, which constitutes an improvement of up to 11.5% compared to purely data driven approaches. Clinical evaluation on 42 subjects showed, that the proposed fully-automatic approach could provide discriminant biomarkers to detect and quantify remodeling of annulus and leaflets in functional mitral regurgitation.