Segmentation and interpretation of MR brain images: an improved active shape model

Resolving Colliding Larvae by Fitting ASM to Random Walker-Based Pre-Segmentations

Drosophila melanogaster is an important model organism for research in neuro- and behavioral biology. Automated studies of their locomotion are crucial to link sensory input and neural processing to motor output which has led to numerous vision-based tracking systems. However, most of these approaches share the inability to segment the contours of colliding animals causing identity losses, appearing and disappearing animals, and the absence of posture and motion related measurements during the time of the collision. We present a novel collision resolution algorithm enabling an accurate contour segmentation of multiple touching Drosophila larvae. Our algorithm utilizes an adapted active shape model (ASM) to learn a low dimensional posture space which is fitted to random-walker generated pre-segmentations. We evaluate our collision resolution algorithm using three publicly available datasets and compare it with the current state-of-the-art methods. In addition, we introduce a refined dataset enabling a segmentation evaluation on the level of pixel accuracy. The results demonstrate that our approach outperforms the state-of-the-art approaches in both accuracy and computational time. We will incorporate this algorithm into our widely used tracking program to improve the statistical strength of the behavioral quantification and allow marker-free studies of interacting Drosophila larvae.

Misshapen Pelvis Landmark Detection With Local-Global Feature Learning for Diagnosing Developmental Dysplasia of the Hip

Developmental dysplasia of the hip (DDH) is one of the most common orthopedic disorders in infants and young children. Accurately detecting and identifying the misshapen anatomical landmarks plays a crucial role in the diagnosis of DDH. However, the diversity during the calcification and the deformity due to the dislocation lead it a difficult task to detect the misshapen pelvis landmarks for both human expert and computer. Generally, the anatomical landmarks exhibit stable morphological features in part regions and rigid structural features in long ranges, which can be strong identification for the landmarks. In this paper, we investigate the local morphological features and global structural features for the misshapen landmark detection with a novel Pyramid Non-local UNet (PN-UNet). Firstly, we mine the local morphological features with a series of convolutional neural network (CNN) stacks, and convert the detection of a landmark to the segmentation of the landmark's local neighborhood by UNet. Secondly, a non-local module is employed to capture the global structural features with high-level structural knowledge. With the end-to-end and accurate detection of pelvis landmarks, we realize a fully automatic and highly reliable diagnosis of DDH. In addition, a dataset with 10,000 pelvis X-ray images is constructed in our work. It is the first public dataset for diagnosing DDH and has been already released for open research. To the best of our knowledge, this is the first attempt to apply deep learning method in the diagnosis of DDH. Experimental results show that our approach achieves an excellent precision in landmark detection (average point to point error of 0.9286mm) and illness diagnosis over human experts. Project is available at http://imcc.ustc.edu.cn/project/ddh/.

MRI Segmentation and Classification of Human Brain Using Deep Learning for Diagnosis of Alzheimer's Disease: A Survey

Many neurological diseases and delineating pathological regions have been analyzed, and the anatomical structure of the brain researched with the aid of magnetic resonance imaging (MRI). It is important to identify patients with Alzheimer's disease (AD) early so that preventative measures can be taken. A detailed analysis of the tissue structures from segmented MRI leads to a more accurate classification of specific brain disorders. Several segmentation methods to diagnose AD have been proposed with varying complexity. Segmentation of the brain structure and classification of AD using deep learning approaches has gained attention as it can provide effective results over a large set of data. Hence, deep learning methods are now preferred over state-of-the-art machine learning methods. We aim to provide an outline of current deep learning-based segmentation approaches for the quantitative analysis of brain MRI for the diagnosis of AD. Here, we report how convolutional neural network architectures are used to analyze the anatomical brain structure and diagnose AD, discuss how brain MRI segmentation improves AD classification, describe the state-of-the-art approaches, and summarize their results using publicly available datasets. Finally, we provide insight into current issues and discuss possible future research directions in building a computer-aided diagnostic system for AD.

Segmenting brain tumors from FLAIR MRI using fully convolutional neural networks

BACKGROUND AND OBJECTIVE

Magnetic resonance imaging (MRI) is an indispensable tool in diagnosing brain-tumor patients. Automated tumor segmentation is being widely researched to accelerate the MRI analysis and allow clinicians to precisely plan treatment-accurate delineation of brain tumors is a critical step in assessing their volume, shape, boundaries, and other characteristics. However, it is still a very challenging task due to inherent MR data characteristics and high variability, e.g., in tumor sizes or shapes. We present a new deep learning approach for accurate brain tumor segmentation which can be trained from small and heterogeneous datasets annotated by a human reader (providing high-quality ground-truth segmentation is very costly in practice).

METHODS

In this paper, we present a new deep learning technique for segmenting brain tumors from fluid attenuation inversion recovery MRI. Our technique exploits fully convolutional neural networks, and it is equipped with a battery of augmentation techniques that make the algorithm robust against low data quality, and heterogeneity of small training sets. We train our models using only positive (tumorous) examples, due to the limited amount of available data.

RESULTS

Our algorithm was tested on a set of stage II-IV brain-tumor patients (image data collected using MAGNETOM Prisma 3T, Siemens). Rigorous experiments, backed up with statistical tests, revealed that our approach outperforms the state-of-the-art approach (utilizing hand-crafted features) in terms of segmentation accuracy, offers very fast training and instant segmentation (analysis of an image takes less than a second). Building our deep model is 1.3 times faster compared with extracting features for extremely randomized trees, and this training time can be controlled. Finally, we showed that too aggressive data augmentation may lead to deteriorated performance of the model, especially in the fixed-budget training (with maximum numbers of training epochs).

CONCLUSIONS

Our method yields the better performance when compared with the state of the art method which utilizes hand-crafted features. In addition, our deep network can be effectively applied to difficult (small, imbalanced, and heterogeneous) datasets, offers controllable training time, and infers in real-time.

Level set distribution model of nested structures using logarithmic transformation

In this study, we propose a method for constructing a multishape statistical shape model (SSM) for nested structures such that each is a subset or superset of another. The proposed method has potential application to any pair of shapes with an inclusive relationship. These types of shapes are often found in anatomy, such as the brain surface and ventricles. The main contribution of this paper is to introduce a new shape representation called log-transformed level set function (LT-LSF), which has a vector space structure that preserves the correct inclusive relationship of the nested shape. In addition, our method is applicable to an arbitrary number of nested shapes. We demonstrate the effectiveness of the proposed shape representation by modeling the anatomy of human embryos, including the brain, ventricles, and choroid plexus volumes. The performance of the SSM was evaluated in terms of generalization and specificity ability. Additionally, we measured leakage criteria to assess the ability to preserve inclusive relationships. A quantitative comparison of our SSM with conventional multishape SSMs demonstrates the superiority of the proposed method.

Energy minimization in medical image analysis: Methodologies and applications

Energy minimization is of particular interest in medical image analysis. In the past two decades, a variety of optimization schemes have been developed. In this paper, we present a comprehensive survey of the state-of-the-art optimization approaches. These algorithms are mainly classified into two categories: continuous method and discrete method. The former includes Newton-Raphson method, gradient descent method, conjugate gradient method, proximal gradient method, coordinate descent method, and genetic algorithm-based method, while the latter covers graph cuts method, belief propagation method, tree-reweighted message passing method, linear programming method, maximum margin learning method, simulated annealing method, and iterated conditional modes method. We also discuss the minimal surface method, primal-dual method, and the multi-objective optimization method. In addition, we review several comparative studies that evaluate the performance of different minimization techniques in terms of accuracy, efficiency, or complexity. These optimization techniques are widely used in many medical applications, for example, image segmentation, registration, reconstruction, motion tracking, and compressed sensing. We thus give an overview on those applications as well.

Thalamic segmentation based on improved fuzzy connectedness in structural MRI

Thalamic segmentation serves an important function in localizing targets for deep brain stimulation (DBS). However, thalamic nuclei are still difficult to identify clearly from structural MRI. In this study, an improved algorithm based on the fuzzy connectedness framework was developed. Three-dimensional T1-weighted images in axial orientation were acquired through a 3D SPGR sequence by using a 1.5 T GE magnetic resonance scanner. Twenty-five normal images were analyzed using the proposed method, which involved adaptive fuzzy connectedness combined with confidence connectedness (AFCCC). After non-brain tissue removal and contrast enhancement, the seed point was selected manually, and confidence connectedness was used to perform an ROI update automatically. Both image intensity and local gradient were taken as image features in calculating the fuzzy affinity. Moreover, the weight of the features could be automatically adjusted. Thalamus, ventrointermedius (Vim), and subthalamic nucleus were successfully segmented. The results were evaluated with rules, such as similarity degree (SD), union overlap, and false positive. SD of thalamus segmentation reached values higher than 85%. The segmentation results were also compared with those achieved by the region growing and level set methods, respectively. Higher SD of the proposed method, especially in Vim, was achieved. The time cost using AFCCC was low, although it could achieve high accuracy. The proposed method is superior to the traditional fuzzy connectedness framework and involves reduced manual intervention in time saving.

SEGMENTATION OF BRAIN MRIs BY SUPPORT VECTOR MACHINE: DETECTION AND CHARACTERIZATION OF STROKES

The aim of our work is the segmentation of healthy and pathological brains to obtain brain structures and extract strokes. We used real magnetic resonance (MR) images weighted on diffusion. The brain was isolated, and the images were filtered by an anisotropic filter, and then segmented by support vector machines (SVMs). We first applied the method on synthetic images to test the performance of the algorithm and adjust the parameters. Then, we compared our results with those obtained by a cooperative approach proposed in a previous paper.

Automatic multi-resolution shape modeling of multi-organ structures

Point Distribution Models (PDM) are among the most popular shape description techniques and their usefulness has been demonstrated in a wide variety of medical imaging applications. However, to adequately characterize the underlying modeled population it is essential to have a representative number of training samples, which is not always possible. This problem is especially relevant as the complexity of the modeled structure increases, being the modeling of ensembles of multiple 3D organs one of the most challenging cases. In this paper, we introduce a new GEneralized Multi-resolution PDM (GEM-PDM) in the context of multi-organ analysis able to efficiently characterize the different inter-object relations, as well as the particular locality of each object separately. Importantly, unlike previous approaches, the configuration of the algorithm is automated thanks to a new agglomerative landmark clustering method proposed here, which equally allows us to identify smaller anatomically significant regions within organs. The significant advantage of the GEM-PDM method over two previous approaches (PDM and hierarchical PDM) in terms of shape modeling accuracy and robustness to noise, has been successfully verified for two different databases of sets of multiple organs: six subcortical brain structures, and seven abdominal organs. Finally, we propose the integration of the new shape modeling framework into an active shape-model-based segmentation algorithm. The resulting algorithm, named GEMA, provides a better overall performance than the two classical approaches tested, ASM, and hierarchical ASM, when applied to the segmentation of 3D brain MRI.

MRI segmentation of the human brain: challenges, methods, and applications

Image segmentation is one of the most important tasks in medical image analysis and is often the first and the most critical step in many clinical applications. In brain MRI analysis, image segmentation is commonly used for measuring and visualizing the brain's anatomical structures, for analyzing brain changes, for delineating pathological regions, and for surgical planning and image-guided interventions. In the last few decades, various segmentation techniques of different accuracy and degree of complexity have been developed and reported in the literature. In this paper we review the most popular methods commonly used for brain MRI segmentation. We highlight differences between them and discuss their capabilities, advantages, and limitations. To address the complexity and challenges of the brain MRI segmentation problem, we first introduce the basic concepts of image segmentation. Then, we explain different MRI preprocessing steps including image registration, bias field correction, and removal of nonbrain tissue. Finally, after reviewing different brain MRI segmentation methods, we discuss the validation problem in brain MRI segmentation.

Structured patch model for a unified automatic and interactive segmentation framework

We present a novel interactive segmentation framework incorporating a priori knowledge learned from training data. The knowledge is learned as a structured patch model (StPM) comprising sets of corresponding local patch priors and their pairwise spatial distribution statistics which represent the local shape and appearance along its boundary and the global shape structure, respectively. When successive user annotations are given, the StPM is appropriately adjusted in the target image and used together with the annotations to guide the segmentation. The StPM reduces the dependency on the placement and quantity of user annotations with little increase in complexity since the time-consuming StPM construction is performed offline. Furthermore, a seamless learning system can be established by directly adding the patch priors and the pairwise statistics of segmentation results to the StPM. The proposed method was evaluated on three datasets, respectively, of 2D chest CT, 3D knee MR, and 3D brain MR. The experimental results demonstrate that within an equal amount of time, the proposed interactive segmentation framework outperforms recent state-of-the-art methods in terms of accuracy, while it requires significantly less computing and editing time to obtain results with comparable accuracy.

Temporal-lobe morphology differs between healthy adolescents and those with early-onset of depression

Major depressive disorder (MDD) has previously been linked to structural changes in several brain regions, particularly in the medial temporal lobes (Bellani, Baiano, Brambilla, 2010; Bellani, Baiano, Brambilla, 2011). This has been determined using voxel-based morphometry, segmentation algorithms, and analysis of shape deformations (Bell-McGinty et al., 2002; Bergouignan et al., 2009; Posener et al., 2003; Vasic et al., 2008; Zhao et al., 2008): these are methods in which information related to the shape and the pose (the size, and anatomical position and orientation) of structures is lost. Here, we incorporate information about shape and pose to measure structural deformation in adolescents and young adults with and without depression (as measured using the Beck Depression Inventory and Diagnostic and Statistical Manual of Mental Disorders criteria). As a hypothesis-generating study, a significance level of p < 0.05, uncorrected for multiple comparisons, was used, so that subtle morphological differences in brain structures between adolescent depressed individuals and control participants could be identified. We focus on changes in cortical and subcortical temporal structures, and use a multi-object statistical pose and shape model to analyze imaging data from 16 females (aged 16-21) and 3 males (aged 18) with early-onset MDD, and 25 female and 1 male normal control participants, drawn from the same age range. The hippocampus, parahippocampal gyrus, putamen, and superior, inferior and middle temporal gyri in both hemispheres of the brain were automatically segmented using the LONI Probabilistic Brain Atlas (Shattuck et al., 2008) in MNI space. Points on the surface of each structure in the atlas were extracted and warped to each participant's structural MRI. These surface points were analyzed to extract the pose and shape features. Pose differences were detected between the two groups, particularly in the left and right putamina, right hippocampus, and left and right inferior temporal gyri. Shape differences were detected between the two groups, particularly in the left hippocampus and in the left and right parahippocampal gyri. Furthermore, pose measures were significantly correlated with BDI score across the whole (clinical and control) sample. Since the clinical participants were experiencing their very first episodes of MDD, morphological alteration in the medial temporal lobe appears to be an early sign of MDD, and is unlikely to result from treatment with antidepressants. Pose and shape measures of morphology, which are not usually analyzed in neuromorphometric studies, appear to be sensitive to depressive symptomatology.

Deformable models with sparsity constraints for cardiac motion analysis

Deformable models integrate bottom-up information derived from image appearance cues and top-down priori knowledge of the shape. They have been widely used with success in medical image analysis. One limitation of traditional deformable models is that the information extracted from the image data may contain gross errors, which adversely affect the deformation accuracy. To alleviate this issue, we introduce a new family of deformable models that are inspired from the compressed sensing, a technique for accurate signal reconstruction by harnessing some sparseness priors. In this paper, we employ sparsity constraints to handle the outliers or gross errors, and integrate them seamlessly with deformable models. The proposed new formulation is applied to the analysis of cardiac motion using tagged magnetic resonance imaging (tMRI), where the automated tagging line tracking results are very noisy due to the poor image quality. Our new deformable models track the heart motion robustly, and the resulting strains are consistent with those calculated from manual labels.

A High-resolution Atlas and Statistical Model of the Vocal Tract from Structural MRI

Magnetic resonance imaging (MRI) is an essential tool in the study of muscle anatomy and functional activity in the tongue. Objective assessment of similarities and differences in tongue structure and function has been performed using unnormalized data, but this is biased by the differences in size, shape, and orientation of the structures. To remedy this, we propose a methodology to build a 3D vocal tract atlas based on structural MRI volumes from twenty normal subjects. We first constructed high-resolution volumes from three orthogonal stacks. We then removed extraneous data so that all 3D volumes contained the same anatomy. We used an unbiased diffeomorphic groupwise registration using a cross-correlation similarity metric. Principal component analysis was applied to the deformation fields to create a statistical model from the atlas. Various evaluations and applications were carried out to show the behaviour and utility of the atlas.

Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images

To make Quantitative Radiology (QR) a reality in radiological practice, computerized body-wide Automatic Anatomy Recognition (AAR) becomes essential. With the goal of building a general AAR system that is not tied to any specific organ system, body region, or image modality, this paper presents an AAR methodology for localizing and delineating all major organs in different body regions based on fuzzy modeling ideas and a tight integration of fuzzy models with an Iterative Relative Fuzzy Connectedness (IRFC) delineation algorithm. The methodology consists of five main steps: (a) gathering image data for both building models and testing the AAR algorithms from patient image sets existing in our health system; (b) formulating precise definitions of each body region and organ and delineating them following these definitions; (c) building hierarchical fuzzy anatomy models of organs for each body region; (d) recognizing and locating organs in given images by employing the hierarchical models; and (e) delineating the organs following the hierarchy. In Step (c), we explicitly encode object size and positional relationships into the hierarchy and subsequently exploit this information in object recognition in Step (d) and delineation in Step (e). Modality-independent and dependent aspects are carefully separated in model encoding. At the model building stage, a learning process is carried out for rehearsing an optimal threshold-based object recognition method. The recognition process in Step (d) starts from large, well-defined objects and proceeds down the hierarchy in a global to local manner. A fuzzy model-based version of the IRFC algorithm is created by naturally integrating the fuzzy model constraints into the delineation algorithm. The AAR system is tested on three body regions - thorax (on CT), abdomen (on CT and MRI), and neck (on MRI and CT) - involving a total of over 35 organs and 130 data sets (the total used for model building and testing). The training and testing data sets are divided into equal size in all cases except for the neck. Overall the AAR method achieves a mean accuracy of about 2 voxels in localizing non-sparse blob-like objects and most sparse tubular objects. The delineation accuracy in terms of mean false positive and negative volume fractions is 2% and 8%, respectively, for non-sparse objects, and 5% and 15%, respectively, for sparse objects. The two object groups achieve mean boundary distance relative to ground truth of 0.9 and 1.5 voxels, respectively. Some sparse objects - venous system (in the thorax on CT), inferior vena cava (in the abdomen on CT), and mandible and naso-pharynx (in neck on MRI, but not on CT) - pose challenges at all levels, leading to poor recognition and/or delineation results. The AAR method fares quite favorably when compared with methods from the recent literature for liver, kidneys, and spleen on CT images. We conclude that separation of modality-independent from dependent aspects, organization of objects in a hierarchy, encoding of object relationship information explicitly into the hierarchy, optimal threshold-based recognition learning, and fuzzy model-based IRFC are effective concepts which allowed us to demonstrate the feasibility of a general AAR system that works in different body regions on a variety of organs and on different modalities.

Multi-contrast multi-atlas parcellation of diffusion tensor imaging of the human brain

In this paper, we propose a novel method for parcellating the human brain into 193 anatomical structures based on diffusion tensor images (DTIs). This was accomplished in the setting of multi-contrast diffeomorphic likelihood fusion using multiple DTI atlases. DTI images are modeled as high dimensional fields, with each voxel exhibiting a vector valued feature comprising of mean diffusivity (MD), fractional anisotropy (FA), and fiber angle. For each structure, the probability distribution of each element in the feature vector is modeled as a mixture of Gaussians, the parameters of which are estimated from the labeled atlases. The structure-specific feature vector is then used to parcellate the test image. For each atlas, a likelihood is iteratively computed based on the structure-specific vector feature. The likelihoods from multiple atlases are then fused. The updating and fusing of the likelihoods is achieved based on the expectation-maximization (EM) algorithm for maximum a posteriori (MAP) estimation problems. We first demonstrate the performance of the algorithm by examining the parcellation accuracy of 18 structures from 25 subjects with a varying degree of structural abnormality. Dice values ranging 0.8-0.9 were obtained. In addition, strong correlation was found between the volume size of the automated and the manual parcellation. Then, we present scan-rescan reproducibility based on another dataset of 16 DTI images - an average of 3.73%, 1.91%, and 1.79% for volume, mean FA, and mean MD respectively. Finally, the range of anatomical variability in the normal population was quantified for each structure.

Shape representation for efficient landmark-based segmentation in 3-d

In this paper, we propose a novel approach to landmark-based shape representation that is based on transportation theory, where landmarks are considered as sources and destinations, all possible landmark connections as roads, and established landmark connections as goods transported via these roads. Landmark connections, which are selectively established, are identified through their statistical properties describing the shape of the object of interest, and indicate the least costly roads for transporting goods from sources to destinations. From such a perspective, we introduce three novel shape representations that are combined with an existing landmark detection algorithm based on game theory. To reduce computational complexity, which results from the extension from 2-D to 3-D segmentation, landmark detection is augmented by a concept known in game theory as strategy dominance. The novel shape representations, game-theoretic landmark detection and strategy dominance are combined into a segmentation framework that was evaluated on 3-D computed tomography images of lumbar vertebrae and femoral heads. The best shape representation yielded symmetric surface distance of 0.75 mm and 1.11 mm, and Dice coefficient of 93.6% and 96.2% for lumbar vertebrae and femoral heads, respectively. By applying strategy dominance, the computational costs were further reduced for up to three times.

Lumbar spine segmentation using a statistical multi-vertebrae anatomical shape+pose model

Segmentation of the spinal column from computed tomography (CT) images is a preprocessing step for a range of image-guided interventions. One intervention that would benefit from accurate segmentation is spinal needle injection. Previous spinal segmentation techniques have primarily focused on identification and separate segmentation of each vertebra. Recently, statistical multi-object shape models have been introduced to extract common statistical characteristics between several anatomies. These models can be used for segmentation purposes because they are robust, accurate, and computationally tractable. In this paper, we develop a statistical multi-vertebrae shape+pose model and propose a novel registration-based technique to segment the CT images of spine. The multi-vertebrae statistical model captures the variations in shape and pose simultaneously, which reduces the number of registration parameters. We validate our technique in terms of accuracy and robustness of multi-vertebrae segmentation of CT images acquired from lumbar vertebrae of 32 subjects. The mean error of the proposed technique is below 2 mm, which is sufficient for many spinal needle injection procedures, such as facet joint injections.

Bayesian Parameter Estimation and Segmentation in the Multi-Atlas Random Orbit Model

This paper examines the multiple atlas random diffeomorphic orbit model in Computational Anatomy (CA) for parameter estimation and segmentation of subcortical and ventricular neuroanatomy in magnetic resonance imagery. We assume that there exist multiple magnetic resonance image (MRI) atlases, each atlas containing a collection of locally-defined charts in the brain generated via manual delineation of the structures of interest. We focus on maximum a posteriori estimation of high dimensional segmentations of MR within the class of generative models representing the observed MRI as a conditionally Gaussian random field, conditioned on the atlas charts and the diffeomorphic change of coordinates of each chart that generates it. The charts and their diffeomorphic correspondences are unknown and viewed as latent or hidden variables. We demonstrate that the expectation-maximization (EM) algorithm arises naturally, yielding the likelihood-fusion equation which the a posteriori estimator of the segmentation labels maximizes. The likelihoods being fused are modeled as conditionally Gaussian random fields with mean fields a function of each atlas chart under its diffeomorphic change of coordinates onto the target. The conditional-mean in the EM algorithm specifies the convex weights with which the chart-specific likelihoods are fused. The multiple atlases with the associated convex weights imply that the posterior distribution is a multi-modal representation of the measured MRI. Segmentation results for subcortical and ventricular structures of subjects, within populations of demented subjects, are demonstrated, including the use of multiple atlases across multiple diseased groups.

Atlas-based neuroinformatics via MRI: harnessing information from past clinical cases and quantitative image analysis for patient care

With the ever-increasing amount of anatomical information radiologists have to evaluate for routine diagnoses, computational support that facilitates more efficient education and clinical decision making is highly desired. Despite the rapid progress of image analysis technologies for magnetic resonance imaging of the human brain, these methods have not been widely adopted for clinical diagnoses. To bring computational support into the clinical arena, we need to understand the decision-making process employed by well-trained clinicians and develop tools to simulate that process. In this review, we discuss the potential of atlas-based clinical neuroinformatics, which consists of annotated databases of anatomical measurements grouped according to their morphometric phenotypes and coupled with the clinical informatics upon which their diagnostic groupings are based. As these are indexed via parametric representations, we can use image retrieval tools to search for phenotypes along with their clinical metadata. The review covers the current technology, preliminary data, and future directions of this field.

Robust Diffeomorphic Mapping via Geodesically Controlled Active Shapes

This paper presents recent advances in the use of diffeomorphic active shapes which incorporate the conservation laws of large deformation diffeomorphic metric mapping. The equations of evolution satisfying the conservation law are geodesics under the diffeomorphism metric and therefore termed geodesically controlled diffeomorphic active shapes (GDAS). Our principal application in this paper is on robust diffeomorphic mapping methods based on parameterized surface representations of subcortical template structures. Our parametrization of the GDAS evolution is via the initial momentum representation in the tangent space of the template surface. The dimension of this representation is constrained using principal component analysis generated from training samples. In this work, we seek to use template surfaces to generate segmentations of the hippocampus with three data attachment terms: surface matching, landmark matching, and inside-outside modeling from grayscale T1 MR imaging data. This is formulated as an energy minimization problem, where energy describes shape variability and data attachment accuracy, and we derive a variational solution. A gradient descent strategy is employed in the numerical optimization. For the landmark matching case, we demonstrate the robustness of this algorithm as applied to the workflow of a large neuroanatomical study by comparing to an existing diffeomorphic landmark matching algorithm.

Three-dimensional surface deformation-based shape analysis of hippocampus and caudate nucleus in children with fetal alcohol spectrum disorders

Surface deformation-based analysis was used to assess local shape variations in the hippocampi and caudate nuclei of children with fetal alcohol spectrum disorders. High-resolution structural magnetic resonance imaging images were acquired for 31 children (19 controls and 12 children diagnosed with fetal alcohol syndrome/partial FAS). Hippocampi and caudate nuclei were manually segmented, and surface meshes were reconstructed. An iterative closest point algorithm was used to register the template of one control subject to all other shapes in order to capture the true geometry of the shape with a fixed number of landmark points. A point distribution model was used to quantify the shape variations in terms of a change in co-ordinate positions. Using the localized Hotelling T(2) method, regions of significant shape variations between the control and exposed subjects were identified and mapped onto the mean shapes. Binary masks of hippocampi and caudate nuclei were generated from the segmented volumes of each brain. These were used to compute the volumes and for further statistical analysis. The Mann-Whitney test was performed to predict volume differences between the groups. Although the exposed and control subjects did not differ significantly in their volumes, the shape analysis showed the hippocampus to be more deformed at the head and tail regions in the alcohol-exposed children. Between-group differences in caudate nucleus morphology were dispersed across the tail and head regions. Correlation analysis showed associations between the degree of compression and the level of alcohol exposure. These findings demonstrate that shape analysis using three-dimensional surface measures is sensitive to fetal alcohol exposure and provides additional information than volumetric measures alone.

A game-theoretic framework for landmark-based image segmentation

A novel game-theoretic framework for landmark-based image segmentation is presented. Landmark detection is formulated as a game, in which landmarks are players, landmark candidate points are strategies, and likelihoods that candidate points represent landmarks are payoffs, determined according to the similarity of image intensities and spatial relationships between the candidate points in the target image and their corresponding landmarks in images from the training set. The solution of the formulated game-theoretic problem is the equilibrium of candidate points that represent landmarks in the target image and is obtained by a novel iterative scheme that solves the segmentation problem in polynomial time. The object boundaries are finally extracted by applying dynamic programming to the optimal path searching problem between the obtained adjacent landmarks. The performance of the proposed framework was evaluated for segmentation of lung fields from chest radiographs and heart ventricles from cardiac magnetic resonance cross sections. The comparison to other landmark-based segmentation techniques shows that the results obtained by the proposed game-theoretic framework are highly accurate and precise in terms of mean boundary distance and area overlap. Moreover, the framework overcomes several shortcomings of the existing techniques, such as sensitivity to initialization and convergence to local optima.

Hierarchical statistical shape models of multiobject anatomical structures: application to brain MRI

The accurate segmentation of subcortical brain structures in magnetic resonance (MR) images is of crucial importance in the interdisciplinary field of medical imaging. Although statistical approaches such as active shape models (ASMs) have proven to be particularly useful in the modeling of multiobject shapes, they are inefficient when facing challenging problems. Based on the wavelet transform, the fully generic multiresolution framework presented in this paper allows us to decompose the interobject relationships into different levels of detail. The aim of this hierarchical decomposition is twofold: to efficiently characterize the relationships between objects and their particular localities. Experiments performed on an eight-object structure defined in axial cross sectional MR brain images show that the new hierarchical segmentation significantly improves the accuracy of the segmentation, and while it exhibits a remarkable robustness with respect to the size of the training set.

A new osteophyte segmentation algorithm using partial shape model and its applications to rabbit femur anterior cruciate ligament transection via micro-CT imaging

Osteophyte is an additional bony growth on a normal bone surface limiting or stopping motion at a deteriorating joint. Detection and quantification of osteophytes from CT images is helpful in assessing disease status as well as treatment and surgery planning. However, it is difficult to distinguish between osteophytes and healthy bones using simple thresholding or edge/texture features due to the similarity of their material composition. In this paper, we present a new method primarily based active shape model (ASM) to solve this problem and evaluate its application to anterior cruciate ligament transection (ACLT) rabbit femur model via CT imaging. The common idea behind most ASM based segmentation methods is to first build a parametric shape model from a training dataset and apply the model to find a shape instance in a target image. A common challenge with such approaches is that a diseased bone shape is significantly altered at regions with osteophyte deposition misguiding an ASM method and eventually leading to suboptimum segmentations. This difficulty is overcome using a new partial ASM method that uses bone shape over healthy regions and extrapolates it over the diseased region according to the underlying shape model. Finally, osteophytes are segmented by subtracting partial-ASM derived shape from the overall diseased shape. Also, a new semi-automatic method is presented in this paper for efficiently building a 3D shape model for an anatomic region using manual reference of a few anatomically defined fiducial landmarks that are highly reproducible on individuals. Accuracy of the method has been examined on simulated phantoms while reproducibility and sensitivity have been evaluated on CT images of 2-, 4- and 8-week post-ACLT and sham-treated rabbit femurs. Experimental results have shown that the method is highly accurate ( R2 = 0.99), reproducible (ICC = 0.97), and sensitive in detecting disease progression (p-values: 0.065,0.001 and < 0.001 for 2- vs. 4, 4- vs. 8- and 2- vs. 8-weeks, respectively).

An inter-landmark approach to 4-D shape extraction and interpretation: application to myocardial motion assessment in MRI

This paper presents a novel approach to shape extraction and interpretation in 4-D cardiac magnetic resonance imaging data. Statistical modeling of spatiotemporal interlandmark relationships is performed to enable the decomposition of global shape constraints and subsequently of the image analysis tasks. The introduced descriptors furthermore provide invariance to similarity transformations and thus eliminate pose estimation errors in the presence of image artifacts or geometrical inconsistencies. A set of algorithms are derived to address key technical issues related to constrained boundary tracking, dynamic model relaxation, automatic initialization, and dysfunction localization. The proposed framework is validated with a relatively large dataset of 50 subjects and compared to existing statistical shape modeling methods. The results indicate increased adaptation to spatiotemporal variations and imaging conditions.

Building 3-D statistical shape models by direct optimization

Statistical shape models are powerful tools for image interpretation and shape analysis. A simple, yet effective, way of building such models is to capture the statistics of sampled point coordinates over a training set of example shapes. However, a major drawback of this approach is the need to establish a correspondence across the training set. In 2-D, a correspondence is often defined using a set of manually placed 'landmarks' and linear interpolation to sample the shape in between. Such annotation is, however, time-consuming and subjective, particularly when extended to 3-D. In this paper, we show that it is possible to establish a dense correspondence across the whole training set automatically by treating correspondence as an optimization problem. The objective function we use for the optimization is based on the minimum description length principle, which we argue is a criterion that leads to models with good compactness, specificity, and generalization ability. We manipulate correspondence by reparameterizing each training shape. We describe an explicit representation of reparameterization for surfaces in 3-D that makes it impossible to generate an illegal (i.e., not one-to-one) correspondence. We also describe several large-scale optimization strategies for model building, and perform a detailed analysis of each approach. Finally, we derive quantitative measures of model quality, allowing meaningful comparison between models built using different methods. Results are given for several different training sets of 3-D shapes, which show that the minimum description length models perform significantly better than other approaches.

Intensity Standardization Simplifies Brain MR Image Segmentation

Typically, brain MR images present significant intensity variation across patients and scanners. Consequently, training a classifier on a set of images and using it subsequently for brain segmentation may yield poor results. Adaptive iterative methods usually need to be employed to account for the variations of the particular scan. These methods are complicated, difficult to implement and often involve significant computational costs. In this paper, a simple, non-iterative method is proposed for brain MR image segmentation. Two preprocessing techniques, namely intensity inhomogeneity correction, and more importantly MR image intensity standardization, used prior to segmentation, play a vital role in making the MR image intensities have a tissue-specific numeric meaning, which leads us to a very simple brain tissue segmentation strategy.Vectorial scale-based fuzzy connectedness and certain morphological operations are utilized first to generate the brain intracranial mask. The fuzzy membership value of each voxel within the intracranial mask for each brain tissue is then estimated. Finally, a maximum likelihood criterion with spatial constraints taken into account is utilized in classifying all voxels in the intracranial mask into different brain tissue groups. A set of inhomogeneity corrected and intensity standardized images is utilized as a training data set. We introduce two methods to estimate fuzzy membership values. In the first method, called SMG (for simple membership based on a gaussian model), the fuzzy membership value is estimated by fitting a multivariate Gaussian model to the intensity distribution of each brain tissue whose mean intensity vector and covariance matrix are estimated and fixed from the training data sets. The second method, called SMH (for simple membership based on a histogram), estimates fuzzy membership value directly via the intensity distribution of each brain tissue obtained from the training data sets. We present several studies to evaluate the performance of these two methods based on 10 clinical MR images of normal subjects and 10 clinical MR images of Multiple Sclerosis (MS) patients. A quantitative comparison indicates that both methods have overall better accuracy than the k-nearest neighbors (kNN) method, and have much better efficiency than the Finite Mixture (FM) model based Expectation-Maximization (EM) method. Accuracy is similar for our methods and EM method for the normal subject data sets, but much better for our methods for the patient data sets.

Oriented active shape models

Active shape models (ASM) are widely employed for recognizing anatomic structures and for delineating them in medical images. In this paper, a novel strategy called oriented active shape models (OASM) is presented in an attempt to overcome the following five limitations of ASM: 1) lower delineation accuracy, 2) the requirement of a large number of landmarks, 3) sensitivity to search range, 4) sensitivity to initialization, and 5) inability to fully exploit the specific information present in the given image to be segmented. OASM effectively combines the rich statistical shape information embodied in ASM with the boundary orientedness property and the globally optimal delineation capability of the live wire methodology of boundary segmentation. The latter characteristics allow live wire to effectively separate an object boundary from other nonobject boundaries with similar properties especially when they come very close in the image domain. The approach leads to a two-level dynamic programming method, wherein the first level corresponds to boundary recognition and the second level corresponds to boundary delineation, and to an effective automatic initialization method. The method outputs a globally optimal boundary that agrees with the shape model if the recognition step is successful in bringing the model close to the boundary in the image. Extensive evaluation experiments have been conducted by utilizing 40 image (magnetic resonance and computed tomography) data sets in each of five different application areas for segmenting breast, liver, bones of the foot, and cervical vertebrae of the spine. Comparisons are made between OASM and ASM based on precision, accuracy, and efficiency of segmentation. Accuracy is assessed using both region-based false positive and false negative measures and boundary-based distance measures. The results indicate the following: 1) The accuracy of segmentation via OASM is considerably better than that of ASM; 2) The number of landmarks can be reduced by a factor of 3 in OASM over that in ASM; 3) OASM becomes largely independent of search range and initialization becomes automatic. All three benefits of OASM ensue mainly from the severe constraints brought in by the boundary-orientedness property of live wire and the globally optimal solution found by the 2-level dynamic programming algorithm.

Automated segmentation of mouse brain images using extended MRF

We introduce an automated segmentation method, extended Markov random field (eMRF), to classify 21 neuroanatomical structures of mouse brain based on three dimensional (3D) magnetic resonance images (MRI). The image data are multispectral: T2-weighted, proton density-weighted, diffusion x, y and z weighted. Earlier research (Ali, A.A., Dale, A.M., Badea, A., Johnson, G.A., 2005. Automated segmentation of neuroanatomical structures in multispectral MR microscopy of the mouse brain. NeuroImage 27 (2), 425-435) successfully explored the use of MRF for mouse brain segmentation. In this research, we study the use of information generated from support vector machine (SVM) to represent the probabilistic information. Since SVM in general has a stronger discriminative power than the Gaussian likelihood method and is able to handle nonlinear classification problems, integrating SVM into MRF improved the classification accuracy. The eMRF employs the posterior probability distribution obtained from SVM to generate a classification based on the MR intensity. Secondly, the eMRF introduces a new potential function based on location information. Third, to maximize the classification performance, the eMRF uses the contribution weights optimally determined for each of the three potential functions: observation, location and contextual functions, which are traditionally equally weighted. We use the voxel overlap percentage and volume difference percentage to evaluate the accuracy of eMRF segmentation and compare the algorithm with three other segmentation methods--mixed ratio sampling SVM (MRS-SVM), atlas-based segmentation and MRF. Validation using classification accuracy indices between automatically segmented and manually traced data shows that eMRF outperforms other methods.

Markov dependence tree-based segmentation of deep brain structures

We propose a new framework for multi-object segmentation of deep brain structures, which have significant shape variations and relatively small sizes in medical brain images. In the images, the structure boundaries may be blurry or even missing, and the surrounding background is a clutter and full of irrelevant edges. We suggest a template-based framework, which fuses the information of edge features, region statistics and inter-structure constraints to detect and locate all the targeted brain structures such that manual initialization is unnecessary. The multi-object template is organized in the form of a hierarchical Markov dependence tree. It makes the matching of multiple objects efficient. Our approach needs only one example as training data and alleviates the demand of a large training set. The obtained segmentation results on real data are encouraging and the proposed method enjoys several important advantages over existing methods.

Shape-driven three-dimensional watersnake segmentation of biological membranes in electron tomography

Due to the significant complexity of membrane morphology and the generally poor image quality in electron tomographic volumes, current automatic methods for segmentation of membranes perform poorly. Users must resort to manual tracing of recognized patterns on 2-D slices of the volume, a method that suffers from subjectivity and is very labor intensive, preventing quantitative analyses of tomographic data that require comparative analyses of many volumes. To overcome these limitations, we develop an automatic 3-D segmentation method that fully exploits the prior knowledge about the shape of the membranes as well as the 3-D information provided by the tomograms, and systematically combines this knowledge with the image data to improve segmentation results. The method is based on the watersnake framework. By mathematically reformulating the traditional watershed segmentation as an energy minimization problem, the watersnake inherits the many strengths of the watershed method while overcoming the limitations of the traditional energy-based segmentation methods. In our previous work (H. Nguyen et al., 2003), the original watersnake model was successfully modified by incorporating smoothness into watershed segmentation. In this work, we further extend that model to incorporate into the energy function various constraints representing our prior knowledge about the global shape of the cellular features to be segmented. Segmentation can, therefore, be accomplished via minimization of the energy function subject to the shape prior constraints. Finally, the mathematical framework is further extended from 2-D to 3-D so that segmentation can be carried out in 3-D to take advantage of the additional information provided by the tomograms. We apply this method for the automatic extraction of biological membranes of varying complexities including those of bacterial walls and mitochondrial boundaries.

Minimal shape and intensity cost path segmentation

A new generic model-based segmentation algorithm is presented, which can be trained from examples akin to the active shape model (ASM) approach in order to acquire knowledge about the shape to be segmented and about the gray-level appearance of the object in the image. Whereas ASM alternates between shape and intensity information during search, the proposed approach optimizes for shape and intensity characteristics simultaneously. Local gray-level appearance information at the landmark points extracted from feature images is used to automatically detect a number of plausible candidate locations for each landmark. The shape information is described by multiple landmark-specific statistical models that capture local dependencies between adjacent landmarks on the shape. The shape and intensity models are combined in a single cost function that is optimized noniteratively using dynamic programming, without the need for initialization. The algorithm was validated for segmentation of anatomical structures in chest and hand radiographs. In each experiment, the presented method had a significant higher performance when compared to the ASM schemes. As the method is highly effective, optimally suited for pathological cases and easy to implement, it is highly useful for many medical image segmentation tasks.

Carotid artery segmentation using an outlier immune 3D active shape models framework

This paper presents an outlier immune 3D active shape models framework for robust volumetric segmentation of the carotid artery required for accurate plaque burden assessment. In the proposed technique, outlier handling is based on a shape metric that is invariant to scaling, rotation and translation by using the ratio of inter-landmark distances as a local shape dissimilarity measure. Tolerance intervals for each descriptor are calculated from the training samples and used to infer the validity of landmarks. The identified outliers are corrected prior to the model fitting using the ratios distributions and appearance information. To improve the feature point search, the method exploits the geometrical knowledge from the outlier analysis at the previous iteration to weight the gray level appearance based fitness measure. A combined intensity-phase feature point search is also introduced which significantly limits the presence of outliers and improves the overall search accuracy. Both numerical and in vivo assessments of the method involving volumetric segmentation of the carotid artery have shown that the outlier handling technique is capable of handling a significant presence of outliers independently of the amplitudes.

Outlier detection and handling for robust 3-D active shape models search

This paper presents a new outlier handling method for volumetric segmentation with three-dimensional (3-D) active shape models. The method is based on a shape metric that is invariant to scaling, rotation and translation by using the ratio of interlandmark distances as a local shape dissimilarity measure. Tolerance intervals for the descriptors are calculated from the training samples and used as a statistical tolerance model to infer the validity of the feature points. A replacement point is then suggested for each outlier based on the tolerance model and the position of the valid points. A geometrically weighted fitness measure is introduced for feature point detection, which limits the presence of outliers and improves the convergence of the proposed segmentation framework. The algorithm is immune to the extremity of the outliers and can handle a highly significant presence of erroneous feature points. The practical value of the technique is validated with 3-D magnetic resonance (MR) segmentation tasks of the carotid artery and myocardial borders of the left ventricle.

Learning-based deformable registration of MR brain images

This paper presents a learning-based method for deformable registration of magnetic resonance (MR) brain images. There are two novelties in the proposed registration method. First, a set of best-scale geometric features are selected for each point in the brain, in order to facilitate correspondence detection during the registration procedure. This is achieved by optimizing an energy function that requires each point to have its best-scale geometric features consistent over the corresponding points in the training samples, and at the same time distinctive from those of nearby points in the neighborhood. Second, the active points used to drive the brain registration are hierarchically selected during the registration procedure, based on their saliency and consistency measures. That is, the image points with salient and consistent features (across different individuals) are considered for the initial registration of two images, while other less salient and consistent points join the registration procedure later. By incorporating these two novel strategies into the framework of the HAMMER registration algorithm, the registration accuracy has been improved according to the results on simulated brain data, and also visible improvement is observed particularly in the cortical regions of real brain data.

Adaptive model initialization and deformation for automatic segmentation of T1-weighted brain MRI data

A fully automatic, two-step, T1-weighted brain magnetic resonance imaging (MRI) segmentation method is presented. A preliminary mask of parenchyma is first estimated through adaptive image intensity analysis and mathematical morphological operations. It serves as the initial model and probability reference for a level-set algorithm in the second step, which finalizes the segmentation based on both image intensity and geometric information. The Dice coefficient and Euclidean distance between boundaries of automatic results and the corresponding references are reported for both phantom and clinical MR data. For the 28 patient scans acquired at our institution, the average Dice coefficient was 98.2% and the mean Euclidean surface distance measure was 0.074 mm. The entire segmentation for either a simulated or a clinical image volume finishes within 2 min on a modern PC system. The accuracy and speed of this technique allow us to automatically create patient-specific finite element models within the operating room on a timely basis for application in image-guided updating of preoperative scans.

Coupled Shape Distribution-Based Segmentation of Multiple Objects

In this paper we develop a multi-object prior shape model for use in curve evolution-based image segmentation. Our prior shape model is constructed from a family of shape distributions (cumulative distribution functions) of features related to the shape. Shape distribution-based object representations possess several desired properties, such as robustness, invariance, and good discriminative and generalizing properties. Further, our prior can capture information about the interaction between multiple objects. We incorporate this prior in a curve evolution formulation for shape estimation. We apply this methodology to problems in medical image segmentation.