GRAM: A framework for geodesic registration on anatomical manifolds

Stratification of Lung Cancer Risk with Thoracic Imaging Phenotypes

In lung cancer screening, estimation of future lung cancer risk is usually guided by demographics and smoking status. The role of constitutional profiles of human body, a.k.a. body habitus, is increasingly understood to be important, but has not been integrated into risk models. Chest low dose computed tomography (LDCT) is the standard imaging study in lung cancer screening, with the capability to discriminate differences in body composition and organ arrangement in the thorax. We hypothesize that the primary phenotypes identified using lung screening chest LDCT can form a representation of body habitus and add predictive power for lung cancer risk stratification. In this pilot study, we evaluated the feasibility of body habitus image-based phenotyping on a large lung screening LDCT dataset. A thoracic imaging manifold was estimated based on an intensity-based pairwise (dis)similarity metric for pairs of spatial normalized chest LDCT images. We applied the hierarchical clustering method on this manifold to identify the primary phenotypes. Body habitus features of each identified phenotype were evaluated and associated with future lung cancer risk using time-to-event analysis. We evaluated the method on the baseline LDCT scans of 1,200 male subjects sampled from National Lung Screening Trial. Five primary phenotypes were identified, which were associated with highly distinguishable clinical and body habitus features. Time-to-event analysis against future lung cancer incidences showed two of the five identified phenotypes were associated with elevated future lung cancer risks (HR=1.61, 95% CI = [1.08, 2.38], p=0.019; HR=1.67, 95% CI = [0.98, 2.86], p=0.057). These results indicated that it is feasible to capture the body habitus by image-base phenotyping using lung screening LDCT and the learned body habitus representation can potentially add value for future lung cancer risk stratification.

neuromaps: structural and functional interpretation of brain maps

Imaging technologies are increasingly used to generate high-resolution reference maps of brain structure and function. Comparing experimentally generated maps to these reference maps facilitates cross-disciplinary scientific discovery. Although recent data sharing initiatives increase the accessibility of brain maps, data are often shared in disparate coordinate systems, precluding systematic and accurate comparisons. Here we introduce neuromaps, a toolbox for accessing, transforming and analyzing structural and functional brain annotations. We implement functionalities for generating high-quality transformations between four standard coordinate systems. The toolbox includes curated reference maps and biological ontologies of the human brain, such as molecular, microstructural, electrophysiological, developmental and functional ontologies. Robust quantitative assessment of map-to-map similarity is enabled via a suite of spatial autocorrelation-preserving null models. neuromaps combines open-access data with transparent functionality for standardizing and comparing brain maps, providing a systematic workflow for comprehensive structural and functional annotation enrichment analysis of the human brain.

Hubless keypoint-based 3D deformable groupwise registration

We present a novel algorithm for Fast Registration Of image Groups (FROG), applied to large 3D image groups. Our approach extracts 3D SURF keypoints from images, computes matched pairs of keypoints and registers the group by minimizing pair distances in a hubless way i.e. without computing any central mean image. Using keypoints significantly reduces the problem complexity compared to voxel-based approaches, and enables us to provide an in-core global optimization, similar to the Bundle Adjustment for 3D reconstruction. As we aim to register images of different patients, the matching step yields many outliers. Then we propose a new EM-weighting algorithm which efficiently discards outliers. Global optimization is carried out with a fast gradient descent algorithm. This allows our approach to robustly register large datasets. The result is a set of diffeomorphic half transforms which link the volumes together and can be subsequently exploited for computational anatomy and landmark detection. We show experimental results on whole-body CT scans, with groups of up to 103 volumes. On a benchmark based on anatomical landmarks, our algorithm compares favorably with the star-groupwise voxel-based ANTs and NiftyReg approaches while being much faster. We also discuss the limitations of our approach for lower resolution images such as brain MRI.

Visualizing Alzheimer's disease progression in low dimensional manifolds

While tomographic neuroimaging data is information rich, objective, and with high sensitivity in the study of brain diseases such as Alzheimer's disease (AD), its direct use in clinical practice and in regulated clinical trial (CT) still has many challenges. Taking CT as an example, unless the relevant policy and the perception of the primary outcome measures change, the need to construct univariate indices (out of the 3-D imaging data) to serve as CT's primary outcome measures will remain the focus of active research. More relevant to this current study, an overall global index that summarizes multiple complicated features from neuroimages should be developed in order to provide high diagnostic accuracy and sensitivity in tracking AD progression over time in clinical setting. Such index should also be practically intuitive and logically explainable to patients and their families. In this research, we propose a new visualization tool, derived from the manifold-based nonlinear dimension reduction of brain MRI features, to track AD progression over time. In specific, we investigate the locally linear embedding (LLE) method using a dataset from Alzheimer's Disease Neuroimaging Initiative (ADNI), which includes the longitudinal MRIs from 562 subjects. About 20% of them progressed to the next stage of dementia. Using only the baseline data of cognitively unimpaired (CU) and AD subjects, LLE reduces the feature dimension to two and a subject's AD progression path can be plotted in this low dimensional LLE feature space. In addition, the likelihood of being categorized to AD is indicated by color. This LLE map is a new data visualization tool that can assist in tracking AD progression over time.

Clinical Study of Diffusion-Weighted Imaging in the Diagnosis of Liver Focal Lesion

Apparent diffusion coefficient (ADC), derived from diffusion-weighted magnetic resonance images (DW-MRI), measures the motion of water molecules in vivo and can be used to quantify tumor response so as to determine the best therapy approach. In this paper, our goal was to determine whether the DW-MRI can be used for qualitative and quantitative liver cancer analysis, where an automated method will be proposed for improving the accuracy of liver segmentation in DW-MRI to increase the ability of diagnosis of disease. We firstly analyzed the research status of liver cancer diagnosis, especially on the issues of liver image segmentation technology in MRI. Then, the imaging mechanism and image features of the DW-MRI were analyzed, and the initial DW-MRI slice was segmented by graph-cut algorithm. Finally, our obtained result from the liver DW-MRI image is quantitatively and qualitatively analyzed. Experimental results show that DW-MRI has a great advantage in the diagnosis, the DWI images of benign lesion group was lower than that of malignant lesion, thus DW-MRI is segmented by graph-cut algorithm can provide important additional information regarding differential diagnosis of specific liver cancer to some extend.

Accurate nonlinear mapping between MNI volumetric and FreeSurfer surface coordinate systems

The results of most neuroimaging studies are reported in volumetric (e.g., MNI152) or surface (e.g., fsaverage) coordinate systems. Accurate mappings between volumetric and surface coordinate systems can facilitate many applications, such as projecting fMRI group analyses from MNI152/Colin27 to fsaverage for visualization or projecting resting‐state fMRI parcellations from fsaverage to MNI152/Colin27 for volumetric analysis of new data. However, there has been surprisingly little research on this topic. Here, we evaluated three approaches for mapping data between MNI152/Colin27 and fsaverage coordinate systems by simulating the above applications: projection of group‐average data from MNI152/Colin27 to fsaverage and projection of fsaverage parcellations to MNI152/Colin27. Two of the approaches are currently widely used. A third approach (registration fusion) was previously proposed, but not widely adopted. Two implementations of the registration fusion (RF) approach were considered, with one implementation utilizing the Advanced Normalization Tools (ANTs). We found that RF‐ANTs performed the best for mapping between fsaverage and MNI152/Colin27, even for new subjects registered to MNI152/Colin27 using a different software tool (FSL FNIRT). This suggests that RF‐ANTs would be useful even for researchers not using ANTs. Finally, it is worth emphasizing that the most optimal approach for mapping data to a coordinate system (e.g., fsaverage) is to register individual subjects directly to the coordinate system, rather than via another coordinate system. Only in scenarios where the optimal approach is not possible (e.g., mapping previously published results from MNI152 to fsaverage), should the approaches evaluated in this manuscript be considered. In these scenarios, we recommend RF‐ANTs (https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/registration/Wu2017_RegistrationFusion).

PCANet-Based Structural Representation for Nonrigid Multimodal Medical Image Registration

Nonrigid multimodal image registration remains a challenging task in medical image processing and analysis. The structural representation (SR)-based registration methods have attracted much attention recently. However, the existing SR methods cannot provide satisfactory registration accuracy due to the utilization of hand-designed features for structural representation. To address this problem, the structural representation method based on the improved version of the simple deep learning network named PCANet is proposed for medical image registration. In the proposed method, PCANet is firstly trained on numerous medical images to learn convolution kernels for this network. Then, a pair of input medical images to be registered is processed by the learned PCANet. The features extracted by various layers in the PCANet are fused to produce multilevel features. The structural representation images are constructed for two input images based on nonlinear transformation of these multilevel features. The Euclidean distance between structural representation images is calculated and used as the similarity metrics. The objective function defined by the similarity metrics is optimized by L-BFGS method to obtain parameters of the free-form deformation (FFD) model. Extensive experiments on simulated and real multimodal image datasets show that compared with the state-of-the-art registration methods, such as modality-independent neighborhood descriptor (MIND), normalized mutual information (NMI), Weber local descriptor (WLD), and the sum of squared differences on entropy images (ESSD), the proposed method provides better registration performance in terms of target registration error (TRE) and subjective human vision.

Brain Atlas Fusion from High-Thickness Diagnostic Magnetic Resonance Images by Learning-Based Super-Resolution

It is fundamentally important to fuse the brain atlas from magnetic resonance (MR) images for many imaging-based studies. Most existing works focus on fusing the atlases from high-quality MR images. However, for low-quality diagnostic images (i.e., with high inter-slice thickness), the problem of atlas fusion has not been addressed yet. In this paper, we intend to fuse the brain atlas from the high-thickness diagnostic MR images that are prevalent for clinical routines. The main idea of our works is to extend the conventional groupwise registration by incorporating a novel super-resolution strategy. The contribution of the proposed super-resolution framework is two-fold. First, each high-thickness subject image is reconstructed to be isotropic by the patch-based sparsity learning. Then, the reconstructed isotropic image is enhanced for better quality through the random-forest-based regression model. In this way, the images obtained by the super-resolution strategy can be fused together by applying the groupwise registration method to construct the required atlas. Our experiments have shown that the proposed framework can effectively solve the problem of atlas fusion from the low-quality brain MR images.

Multi-template analysis of human perirhinal cortex in brain MRI: Explicitly accounting for anatomical variability

RATIONAL

The human perirhinal cortex (PRC) plays critical roles in episodic and semantic memory and visual perception. The PRC consists of Brodmann areas 35 and 36 (BA35, BA36). In Alzheimer's disease (AD), BA35 is the first cortical site affected by neurofibrillary tangle pathology, which is closely linked to neural injury in AD. Large anatomical variability, manifested in the form of different cortical folding and branching patterns, makes it difficult to segment the PRC in MRI scans. Pathology studies have found that in ~97% of specimens, the PRC falls into one of three discrete anatomical variants. However, current methods for PRC segmentation and morphometry in MRI are based on single-template approaches, which may not be able to accurately model these discrete variants METHODS: A multi-template analysis pipeline that explicitly accounts for anatomical variability is used to automatically label the PRC and measure its thickness in T2-weighted MRI scans. The pipeline uses multi-atlas segmentation to automatically label medial temporal lobe cortices including entorhinal cortex, PRC and the parahippocampal cortex. Pairwise registration between label maps and clustering based on residual dissimilarity after registration are used to construct separate templates for the anatomical variants of the PRC. An optimal path of deformations linking these templates is used to establish correspondences between all the subjects. Experimental evaluation focuses on the ability of single-template and multi-template analyses to detect differences in the thickness of medial temporal lobe cortices between patients with amnestic mild cognitive impairment (aMCI, n=41) and age-matched controls (n=44).

RESULTS

The proposed technique is able to generate templates that recover the three dominant discrete variants of PRC and establish more meaningful correspondences between subjects than a single-template approach. The largest reduction in thickness associated with aMCI, in absolute terms, was found in left BA35 using both regional and summary thickness measures. Further, statistical maps of regional thickness difference between aMCI and controls revealed different patterns for the three anatomical variants.

Liver DCE-MRI Registration in Manifold Space Based on Robust Principal Component Analysis

A technical challenge in the registration of dynamic contrast-enhanced magnetic resonance (DCE-MR) imaging in the liver is intensity variations caused by contrast agents. Such variations lead to the failure of the traditional intensity-based registration method. To address this problem, a manifold-based registration framework for liver DCE-MR time series is proposed. We assume that liver DCE-MR time series are located on a low-dimensional manifold and determine intrinsic similarities between frames. Based on the obtained manifold, the large deformation of two dissimilar images can be decomposed into a series of small deformations between adjacent images on the manifold through gradual deformation of each frame to the template image along the geodesic path. Furthermore, manifold construction is important in automating the selection of the template image, which is an approximation of the geodesic mean. Robust principal component analysis is performed to separate motion components from intensity changes induced by contrast agents; the components caused by motion are used to guide registration in eliminating the effect of contrast enhancement. Visual inspection and quantitative assessment are further performed on clinical dataset registration. Experiments show that the proposed method effectively reduces movements while preserving the topology of contrast-enhancing structures and provides improved registration performance.

Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

Feature selection is a critical step in deformable image registration. In particular, selecting the most discriminative features that accurately and concisely describe complex morphological patterns in image patches improves correspondence detection, which in turn improves image registration accuracy. Furthermore, since more and more imaging modalities are being invented to better identify morphological changes in medical imaging data, the development of deformable image registration method that scales well to new image modalities or new image applications with little to no human intervention would have a significant impact on the medical image analysis community. To address these concerns, a learning-based image registration framework is proposed that uses deep learning to discover compact and highly discriminative features upon observed imaging data. Specifically, the proposed feature selection method uses a convolutional stacked autoencoder to identify intrinsic deep feature representations in image patches. Since deep learning is an unsupervised learning method, no ground truth label knowledge is required. This makes the proposed feature selection method more flexible to new imaging modalities since feature representations can be directly learned from the observed imaging data in a very short amount of time. Using the LONI and ADNI imaging datasets, image registration performance was compared to two existing state-of-the-art deformable image registration methods that use handcrafted features. To demonstrate the scalability of the proposed image registration framework, image registration experiments were conducted on 7.0-T brain MR images. In all experiments, the results showed that the new image registration framework consistently demonstrated more accurate registration results when compared to state of the art.

Accurate Learning with Few Atlases (ALFA): an algorithm for MRI neonatal brain extraction and comparison with 11 publicly available methods

Accurate whole-brain segmentation, or brain extraction, of magnetic resonance imaging (MRI) is a critical first step in most neuroimage analysis pipelines. The majority of brain extraction algorithms have been developed and evaluated for adult data and their validity for neonatal brain extraction, which presents age-specific challenges for this task, has not been established. We developed a novel method for brain extraction of multi-modal neonatal brain MR images, named ALFA (Accurate Learning with Few Atlases). The method uses a new sparsity-based atlas selection strategy that requires a very limited number of atlases 'uniformly' distributed in the low-dimensional data space, combined with a machine learning based label fusion technique. The performance of the method for brain extraction from multi-modal data of 50 newborns is evaluated and compared with results obtained using eleven publicly available brain extraction methods. ALFA outperformed the eleven compared methods providing robust and accurate brain extraction results across different modalities. As ALFA can learn from partially labelled datasets, it can be used to segment large-scale datasets efficiently. ALFA could also be applied to other imaging modalities and other stages across the life course.

Simultaneous segmentation and iterative registration method for computing ADC with reduced artifacts from DW-MRI

PURPOSE

Apparent diffusion coefficient (ADC), derived from diffusion-weighted magnetic resonance images (DW-MRI), measures the motion of water molecules in vivo and can be used to quantify tumor response to therapy. The accurate measurement of ADC can be adversely affected by organ motion and imaging artifacts. In this paper, the authors' goal was to develop an automated method for reducing artifacts and thereby improve the accuracy of ADC measurements in moving organs such as liver.

METHODS

The authors developed a novel method of computing ADC with fewer artifacts, through simultaneous image segmentation and iterative registration (SSIR) of multiple b-value DW-MRI. The authors' approach reduces artifacts by automatically finding the best possible alignment between the individual b-value images and a reference DW image using a sequence of transformations. It selects such a sequence by an iterative choice of b-value DW images based on the accuracy of their alignment with the reference DW image. The authors' approach quantifies the accuracy of alignment between a pair of images using modified Hausdroff distance computed between the structures of interest. The structures of interest are identified by a user through strokes drawn in one or more slices in the reference DW image, which are then volumetrically segmented using GrowCut. The same structures are segmented in the remaining b-value images by transforming the user-drawn strokes through registration. The ADC values are computed from all the aligned b-value images. The images are aligned by using affine registration followed by deformable B-spline registration with cubic B-spline resampling.

RESULTS

The authors compared the results of ADC computed using their approach with ADC computed (a) without registration and (b) with basic affine registration of all b-value images to a chosen reference. The authors' approach was the most effective in reducing artifacts compared to the other two methods. It resulted in a mean artifact ratio (fraction of voxels in a structure with negative ADC over total number of voxels in the structure) of 2.7% versus 5.4% for affine registration and 32% for no registration for >200 tumors. The authors' approach also resulted in the lowest median standard deviation in the computed mean ADC for all tumors [0.05,0.09,0.07,0.58] compared to those from affine image registration [0.02, 0.14, 0.58, 0.79] and no image registration [0.64, 0.83, 0.83, 1.09] on tests where random displacement [8,10,12,16] pixels were introduced in multiple trials in the b-value images.

CONCLUSIONS

The authors developed a novel approach for reducing artifacts in ADC maps through simultaneous registration and segmentation of multiple b-value DW images. The authors' method explicitly employs a registration quality metric to align images. When compared to basic affine and no image registrations, the authors' approach produces registrations of greater accuracy with lowest artifact ratio and median standard deviation of the computed mean ADC values for a wide range of displacements.

Learning intervention-induced deformations for non-rigid MR-CT registration and electrode localization in epilepsy patients

This paper describes a framework for learning a statistical model of non-rigid deformations induced by interventional procedures. We make use of this learned model to perform constrained non-rigid registration of pre-procedural and post-procedural imaging. We demonstrate results applying this framework to non-rigidly register post-surgical computed tomography (CT) brain images to pre-surgical magnetic resonance images (MRIs) of epilepsy patients who had intra-cranial electroencephalography electrodes surgically implanted. Deformations caused by this surgical procedure, imaging artifacts caused by the electrodes, and the use of multi-modal imaging data make non-rigid registration challenging. Our results show that the use of our proposed framework to constrain the non-rigid registration process results in significantly improved and more robust registration performance compared to using standard rigid and non-rigid registration methods.

Echocardiography without electrocardiogram using nonlinear dimensionality reduction methods

PURPOSE

The aim of this study is to evaluate the efficiency of a new automatic image processing technique, based on nonlinear dimensionality reduction (NLDR) to separate a cardiac cycle and also detect end-diastole (ED) (cardiac cycle start) and end-systole (ES) frames on an echocardiography system without using ECG.

METHODS

Isometric feature mapping (Isomap) and locally linear embeddings (LLE) are the most popular NLDR algorithms. First, Isomap algorithm is applied on recorded echocardiography images. By this approach, the nonlinear embedded information in sequential images is represented in a two-dimensional manifold and each image is characterized by a symbol on the constructed manifold. Cyclicity analysis of the resultant manifold, which is derived from the cyclic nature of the heart motion, is used to perform cardiac cycle length estimation. Then, LLE algorithm is applied on extracted left ventricle (LV) echocardiography images of one cardiac cycle. Finally, the relationship between consecutive symbols of the resultant manifold by the LLE algorithm, which is based on LV volume changes, is used to estimate ED (cycle start) and ES frames. The proposed algorithms are quantitatively compared to those obtained by a highly experienced echocardiographer from ECG as a reference in 20 healthy volunteers and 12 subjects with pathology.

RESULTS

Mean difference in cardiac cycle length, ED, and ES frame estimation between our method and ECG detection by the experienced echocardiographer is approximately 7, 17, and 17 ms (0.4, 1, and 1 frame), respectively.

CONCLUSION

The proposed image-based method, based on NLDR, can be used as a useful tool for estimation of cardiac cycle length, ED and ES frames in echocardiography systems, with good agreement to ECG assessment by an experienced echocardiographer in routine clinical evaluation.

Geodesic Information Flows: Spatially-Variant Graphs and Their Application to Segmentation and Fusion

Clinical annotations, such as voxel-wise binary or probabilistic tissue segmentations, structural parcellations, pathological regions-of-interest and anatomical landmarks are key to many clinical studies. However, due to the time consuming nature of manually generating these annotations, they tend to be scarce and limited to small subsets of data. This work explores a novel framework to propagate voxel-wise annotations between morphologically dissimilar images by diffusing and mapping the available examples through intermediate steps. A spatially-variant graph structure connecting morphologically similar subjects is introduced over a database of images, enabling the gradual diffusion of information to all the subjects, even in the presence of large-scale morphological variability. We illustrate the utility of the proposed framework on two example applications: brain parcellation using categorical labels and tissue segmentation using probabilistic features. The application of the proposed method to categorical label fusion showed highly statistically significant improvements when compared to state-of-the-art methodologies. Significant improvements were also observed when applying the proposed framework to probabilistic tissue segmentation of both synthetic and real data, mainly in the presence of large morphological variability.

Low-Dimensional Non-Rigid Image Registration Using Statistical Deformation Models From Semi-Supervised Training Data

Accurate and robust image registration is a fundamental task in medical image analysis applications, and requires non-rigid transformations with a large number of degrees of freedom. Statistical deformation models (SDMs) attempt to learn the distribution of non-rigid deformations, and can be used both to reduce the transformation dimensionality and to constrain the registration process. However, high-dimensional SDMs are difficult to train given orders of magnitude fewer training samples. In this paper, we utilize both a small set of annotated imaging data and a large set of unlabeled data to effectively learn an SDM of non-rigid transformations in a semi-supervised training (SST) framework. We demonstrate results applying this framework towards inter-subject registration of skull-stripped, magnetic resonance (MR) brain images. Our approach makes use of 39 labeled MR datasets to create a set of supervised registrations, which we augment with a set of over 1200 unsupervised registrations using unlabeled MRIs. Through leave-one-out cross validation, we show that SST of a non-rigid SDM results in a robust registration algorithm with significantly improved accuracy compared to standard, intensity-based registration, and does so with a 99% reduction in transformation dimensionality.

Local manifold learning for multiatlas segmentation: application to hippocampal segmentation in healthy population and Alzheimer's disease

AIMS

Automated hippocampal segmentation is an important issue in many neuroscience studies.

METHODS

We presented and evaluated a novel segmentation method that utilized a manifold learning technique under the multiatlas-based segmentation scenario. A manifold representation of local patches for each voxel was achieved by applying an Isomap algorithm, which can then be used to obtain spatially local weights of atlases for label fusion. The obtained atlas weights potentially depended on all pairwise similarities of the population, which is in contrast to most existing label fusion methods that only rely on similarities between the target image and the atlases. The performance of the proposed method was evaluated for hippocampal segmentation and compared with two representative local weighted label fusion methods, that is, local majority voting and local weighted inverse distance voting, on an in-house dataset of 28 healthy adolescents (age range: 10-17 years) and two ADNI datasets of 100 participants (age range: 60-89 years). We also implemented hippocampal volumetric analysis and evaluated segmentation performance using atlases from a different dataset.

RESULTS

The median Dice similarities obtained by our proposed method were approximately 0.90 for healthy subjects and above 0.88 for two mixed diagnostic groups of ADNI subjects.

CONCLUSION

The experimental results demonstrated that the proposed method could obtain consistent and significant improvements over label fusion strategies that are implemented in the original space.

Label image constrained multiatlas selection

Multiatlas based method is commonly used in medical image segmentation. In multiatlas based image segmentation, atlas selection and combination are considered as two key factors affecting the performance. Recently, manifold learning based atlas selection methods have emerged as very promising methods. However, due to the complexity of prostate structures in raw images, it is difficult to get accurate atlas selection results by only measuring the distance between raw images on the manifolds. Although the distance between the regions to be segmented across images can be readily obtained by the label images, it is infeasible to directly compute the distance between the test image (gray) and the label images (binary). This paper tries to address this problem by proposing a label image constrained atlas selection method, which exploits the label images to constrain the manifold projection of raw images. Analyzing the data point distribution of the selected atlases in the manifold subspace, a novel weight computation method for atlas combination is proposed. Compared with other related existing methods, the experimental results on prostate segmentation from T2w MRI showed that the selected atlases are closer to the target structure and more accurate segmentation were obtained by using our proposed method.

Morphological covariance in anatomical MRI scans can identify discrete neural pathways in the brain and their disturbances in persons with neuropsychiatric disorders

We hypothesize that coordinated functional activity within discrete neural circuits induces morphological organization and plasticity within those circuits. Identifying regions of morphological covariation that are independent of morphological covariation in other regions therefore may therefore allow us to identify discrete neural systems within the brain. Comparing the magnitude of these variations in individuals who have psychiatric disorders with the magnitude of variations in healthy controls may allow us to identify aberrant neural pathways in psychiatric illnesses. We measured surface morphological features by applying nonlinear, high-dimensional warping algorithms to manually defined brain regions. We transferred those measures onto the surface of a unit sphere via conformal mapping and then used spherical wavelets and their scaling coefficients to simplify the data structure representing these surface morphological features of each brain region. We used principal component analysis (PCA) to calculate covariation in these morphological measures, as represented by their scaling coefficients, across several brain regions. We then assessed whether brain subregions that covaried in morphology, as identified by large eigenvalues in the PCA, identified specific neural pathways of the brain. To do so, we spatially registered the subnuclei for each eigenvector into the coordinate space of a Diffusion Tensor Imaging dataset; we used these subnuclei as seed regions to track and compare fiber pathways with known fiber pathways identified in neuroanatomical atlases. We applied these procedures to anatomical MRI data in a cohort of 82 healthy participants (42 children, 18 males, age 10.5 ± 2.43 years; 40 adults, 22 males, age 32.42 ± 10.7 years) and 107 participants with Tourette's Syndrome (TS) (71 children, 59 males, age 11.19 ± 2.2 years; 36 adults, 21 males, age 37.34 ± 10.9 years). We evaluated the construct validity of the identified covariation in morphology using DTI data from a different set of 20 healthy adults (10 males, mean age 29.7 ± 7.7 years). The PCA identified portions of structures that covaried across the brain, the eigenvalues measuring the magnitude of the covariation in morphology along the respective eigenvectors. Our results showed that the eigenvectors, and the DTI fibers tracked from their associated brain regions, corresponded with known neural pathways in the brain. In addition, the eigenvectors that captured morphological covariation across regions, and the principal components along those eigenvectors, identified neural pathways with aberrant morphological features associated with TS. These findings suggest that covariations in brain morphology can identify aberrant neural pathways in specific neuropsychiatric disorders.

BrainPrint: a discriminative characterization of brain morphology

We introduce BrainPrint, a compact and discriminative representation of brain morphology. BrainPrint captures shape information of an ensemble of cortical and subcortical structures by solving the eigenvalue problem of the 2D and 3D Laplace-Beltrami operator on triangular (boundary) and tetrahedral (volumetric) meshes. This discriminative characterization enables new ways to study the similarity between brains; the focus can either be on a specific brain structure of interest or on the overall brain similarity. We highlight four applications for BrainPrint in this article: (i) subject identification, (ii) age and sex prediction, (iii) brain asymmetry analysis, and (iv) potential genetic influences on brain morphology. The properties of BrainPrint require the derivation of new algorithms to account for the heterogeneous mix of brain structures with varying discriminative power. We conduct experiments on three datasets, including over 3000 MRI scans from the ADNI database, 436 MRI scans from the OASIS dataset, and 236 MRI scans from the VETSA twin study. All processing steps for obtaining the compact representation are fully automated, making this processing framework particularly attractive for handling large datasets.

A Feature-Based Approach to Big Data Analysis of Medical Images

This paper proposes an inference method well-suited to large sets of medical images. The method is based upon a framework where distinctive 3D scale-invariant features are indexed efficiently to identify approximate nearest-neighbor (NN) feature matches-in O (log N) computational complexity in the number of images N. It thus scales well to large data sets, in contrast to methods based on pair-wise image registration or feature matching requiring O(N) complexity. Our theoretical contribution is a density estimator based on a generative model that generalizes kernel density estimation and K-nearest neighbor (KNN) methods.. The estimator can be used for on-the-fly queries, without requiring explicit parametric models or an off-line training phase. The method is validated on a large multi-site data set of 95,000,000 features extracted from 19,000 lung CT scans. Subject-level classification identifies all images of the same subjects across the entire data set despite deformation due to breathing state, including unintentional duplicate scans. State-of-the-art performance is achieved in predicting chronic pulmonary obstructive disorder (COPD) severity across the 5-category GOLD clinical rating, with an accuracy of 89% if both exact and one-off predictions are considered correct.

Improved reconstruction of 4D-MR images by motion predictions

The reconstruction of 4D images from 2D navigator and data slices requires sufficient observations per motion state to avoid blurred images and motion artifacts between slices. Especially images from rare motion states, like deep inhalations during free-breathing, suffer from too few observations. To address this problem, we propose to actively generate more suitable images instead of only selecting from the available images. The method is based on learning the relationship between navigator and data-slice motion by linear regression after dimensionality reduction. This can then be used to predict new data slices for a given navigator by warping existing data slices by their predicted displacement field. The method was evaluated for 4D-MRIs of the liver under free-breathing, where sliding boundaries pose an additional challenge for image registration. Leave-one-out tests for five short sequences of ten volunteers showed that the proposed prediction method improved on average the residual mean (95%) motion between the ground truth and predicted data slice from 0.9mm (1.9mm) to 0.8mm (1.6mm) in comparison to the best selection method. The approach was particularly suited for unusual motion states, where the mean error was reduced by 40% (2.2mm vs. 1.3mm).

Predict brain MR image registration via sparse learning of appearance and transformation

We propose a new approach to register the subject image with the template by leveraging a set of intermediate images that are pre-aligned to the template. We argue that, if points in the subject and the intermediate images share similar local appearances, they may have common correspondence in the template. In this way, we learn the sparse representation of a certain subject point to reveal several similar candidate points in the intermediate images. Each selected intermediate candidate can bridge the correspondence from the subject point to the template space, thus predicting the transformation associated with the subject point at the confidence level that relates to the learned sparse coefficient. Following this strategy, we first predict transformations at selected key points, and retain multiple predictions on each key point, instead of allowing only a single correspondence. Then, by utilizing all key points and their predictions with varying confidences, we adaptively reconstruct the dense transformation field that warps the subject to the template. We further embed the prediction-reconstruction protocol above into a multi-resolution hierarchy. In the final, we refine our estimated transformation field via existing registration method in effective manners. We apply our method to registering brain MR images, and conclude that the proposed framework is competent to improve registration performances substantially.

Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights

Evaluating various algorithms for the inter-subject registration of brain magnetic resonance images (MRI) is a necessary topic receiving growing attention. Existing studies evaluated image registration algorithms in specific tasks or using specific databases (e.g., only for skull-stripped images, only for single-site images, etc.). Consequently, the choice of registration algorithms seems task- and usage/parameter-dependent. Nevertheless, recent large-scale, often multi-institutional imaging-related studies create the need and raise the question whether some registration algorithms can 1) generally apply to various tasks/databases posing various challenges; 2) perform consistently well, and while doing so, 3) require minimal or ideally no parameter tuning. In seeking answers to this question, we evaluated 12 general-purpose registration algorithms, for their generality, accuracy and robustness. We fixed their parameters at values suggested by algorithm developers as reported in the literature. We tested them in 7 databases/tasks, which present one or more of 4 commonly-encountered challenges: 1) inter-subject anatomical variability in skull-stripped images; 2) intensity homogeneity, noise and large structural differences in raw images; 3) imaging protocol and field-of-view (FOV) differences in multi-site data; and 4) missing correspondences in pathology-bearing images. Totally 7,562 registrations were performed. Registration accuracies were measured by (multi-)expert-annotated landmarks or regions of interest (ROIs). To ensure reproducibility, we used public software tools, public databases (whenever possible), and we fully disclose the parameter settings. We show evaluation results, and discuss the performances in light of algorithms' similarity metrics, transformation models and optimization strategies. We also discuss future directions for the algorithm development and evaluations.

Diffusion maps for multimodal registration

Multimodal image registration is a difficult task, due to the significant intensity variations between the images. A common approach is to use sophisticated similarity measures, such as mutual information, that are robust to those intensity variations. However, these similarity measures are computationally expensive and, moreover, often fail to capture the geometry and the associated dynamics linked with the images. Another approach is the transformation of the images into a common space where modalities can be directly compared. Within this approach, we propose to register multimodal images by using diffusion maps to describe the geometric and spectral properties of the data. Through diffusion maps, the multimodal data is transformed into a new set of canonical coordinates that reflect its geometry uniformly across modalities, so that meaningful correspondences can be established between them. Images in this new representation can then be registered using a simple Euclidean distance as a similarity measure. Registration accuracy was evaluated on both real and simulated brain images with known ground-truth for both rigid and non-rigid registration. Results showed that the proposed approach achieved higher accuracy than the conventional approach using mutual information.

Regional manifold learning for disease classification

While manifold learning from images itself has become widely used in medical image analysis, the accuracy of existing implementations suffers from viewing each image as a single data point. To address this issue, we parcellate images into regions and then separately learn the manifold for each region. We use the regional manifolds as low-dimensional descriptors of high-dimensional morphological image features, which are then fed into a classifier to identify regions affected by disease. We produce a single ensemble decision for each scan by the weighted combination of these regional classification results. Each weight is determined by the regional accuracy of detecting the disease. When applied to cardiac magnetic resonance imaging of 50 normal controls and 50 patients with reconstructive surgery of Tetralogy of Fallot, our method achieves significantly better classification accuracy than approaches learning a single manifold across the entire image domain.

AUTO-ENCODING OF DISCRIMINATING MORPHOMETRY FROM CARDIAC MRI

We propose a fully-automatic morphometric encoding targeted towards differentiating diseased from healthy cardiac MRI. Existing encodings rely on accurate segmentations of each scan. Segmentation generally includes labour-intensive editing and increases the risk associated with intra- and inter-rater variability. Our morphometric framework only requires the segmentation of a template scan. This template is non-rigidly registered to the other scans. We then confine the resulting deformation maps to the regions outlined by the segmentations. We learn a manifold for each region and identify the most informative coordinates with respect to distinguishing diseased from healthy scans. Compared with volumetric measurements and a deformation-based score, this encoding is much more accurate in capturing morphometric patterns distinguishing healthy subjects from those with Tetralogy of Fallot, diastolic dysfunction, and hypertrophic cardiomyopathy.

Multi-atlas segmentation with particle-based group-wise image registration

We propose a novel multi-atlas segmentation method that employs a group-wise image registration method for the brain segmentation on rodent magnetic resonance (MR) images. The core element of the proposed segmentation is the use of a particle-guided image registration method that extends the concept of particle correspondence into the volumetric image domain. The registration method performs a group-wise image registration that simultaneously registers a set of images toward the space defined by the average of particles. The particle-guided image registration method is robust with low signal-to-noise ratio images as well as differing sizes and shapes observed in the developing rodent brain. Also, the use of an implicit common reference frame can prevent potential bias induced by the use of a single template in the segmentation process. We show that the use of a particle guided-image registration method can be naturally extended to a novel multi-atlas segmentation method and improves the registration method to explicitly use the provided template labels as an additional constraint. In the experiment, we show that our segmentation algorithm provides more accuracy with multi-atlas label fusion and stability against pair-wise image registration. The comparison with previous group-wise registration method is provided as well.

Individualized statistical learning from medical image databases: application to identification of brain lesions

This paper presents a method for capturing statistical variation of normal imaging phenotypes, with emphasis on brain structure. The method aims to estimate the statistical variation of a normative set of images from healthy individuals, and identify abnormalities as deviations from normality. A direct estimation of the statistical variation of the entire volumetric image is challenged by the high-dimensionality of images relative to smaller sample sizes. To overcome this limitation, we iteratively sample a large number of lower dimensional subspaces that capture image characteristics ranging from fine and localized to coarser and more global. Within each subspace, a "target-specific" feature selection strategy is applied to further reduce the dimensionality, by considering only imaging characteristics present in a test subject's images. Marginal probability density functions of selected features are estimated through PCA models, in conjunction with an "estimability" criterion that limits the dimensionality of estimated probability densities according to available sample size and underlying anatomy variation. A test sample is iteratively projected to the subspaces of these marginals as determined by PCA models, and its trajectory delineates potential abnormalities. The method is applied to segmentation of various brain lesion types, and to simulated data on which superiority of the iterative method over straight PCA is demonstrated.

Multi-atlas segmentation of subcortical brain structures via the AutoSeg software pipeline

Automated segmenting and labeling of individual brain anatomical regions, in MRI are challenging, due to the issue of individual structural variability. Although atlas-based segmentation has shown its potential for both tissue and structure segmentation, due to the inherent natural variability as well as disease-related changes in MR appearance, a single atlas image is often inappropriate to represent the full population of datasets processed in a given neuroimaging study. As an alternative for the case of single atlas segmentation, the use of multiple atlases alongside label fusion techniques has been introduced using a set of individual “atlases” that encompasses the expected variability in the studied population. In our study, we proposed a multi-atlas segmentation scheme with a novel graph-based atlas selection technique. We first paired and co-registered all atlases and the subject MR scans. A directed graph with edge weights based on intensity and shape similarity between all MR scans is then computed. The set of neighboring templates is selected via clustering of the graph. Finally, weighted majority voting is employed to create the final segmentation over the selected atlases. This multi-atlas segmentation scheme is used to extend a single-atlas-based segmentation toolkit entitled AutoSeg, which is an open-source, extensible C++ based software pipeline employing BatchMake for its pipeline scripting, developed at the Neuro Image Research and Analysis Laboratories of the University of North Carolina at Chapel Hill. AutoSeg performs N4 intensity inhomogeneity correction, rigid registration to a common template space, automated brain tissue classification based skull-stripping, and the multi-atlas segmentation. The multi-atlas-based AutoSeg has been evaluated on subcortical structure segmentation with a testing dataset of 20 adult brain MRI scans and 15 atlas MRI scans. The AutoSeg achieved mean Dice coefficients of 81.73% for the subcortical structures.

BrainPrint: identifying subjects by their brain

Introducing BrainPrint, a compact and discriminative representation of anatomical structures in the brain. BrainPrint captures shape information of an ensemble of cortical and subcortical structures by solving the 2D and 3D Laplace-Beltrami operator on triangular (boundary) and tetrahedral (volumetric) meshes. We derive a robust classifier for this representation that identifies the subject in a new scan, based on a database of brain scans. In an example dataset containing over 3000 MRI scans, we show that BrainPrint captures unique information about the subject's anatomy and permits to correctly classify a scan with an accuracy of over 99.8%. All processing steps for obtaining the compact representation are fully automated making this processing framework particularly attractive for handling large datasets.

Geodesics-based image registration: applications to biological and medical images depicting concentric ring patterns

In many biological or medical applications, images that contain sequences of shapes are common. The existence of high inter-individual variability makes their interpretation complex. In this paper, we address the computer-assisted interpretation of such images and we investigate how we can remove or reduce these image variabilities. The proposed approach relies on the development of an efficient image registration technique. We first show the inadequacy of state-of-the-art intensity-based and feature-based registration techniques for the considered image datasets. Then, we propose a robust variational method which benefits from the geometrical information present in this type of images. In the proposed non-rigid geodesics-based registration, the successive shapes are represented by a level-set representation, which we rely on to carry out the registration. The successive level sets are regarded as elements in a shape space and the corresponding matching is that of the optimal geodesic path. The proposed registration scheme is tested on synthetic and real images. The comparison against results of state-of-the-art methods proves the relevance of the proposed method for this type of images.

Atlas learning in fetal brain development

Magnetic resonance imaging has become an important noninvasive technique to gain insight into fetal brain development. Its capabilities go beyond ultrasound when diagnosing high-risk pregnancies. To summarize observations across a population in magnetic resonance imaging studies, reference systems such as atlases that establish correspondences across a cohort are key. In this article, we review the evolution of atlas-building methods in light of their relevance, limitations, and benefits for the modeling of human brain development. Starting with single anatomical templates to which brain scans where mapped to such as Talairach and Montreal Neurological Institute space, we explore the uses of atlases as a means to establish correspondences across a cohort and as a model that captures the population characteristics of the cases the atlas is built from. We discuss methods that capture features of increasingly heterogeneous populations and approaches that are able to generalize with only minimal annotation. The main focus of this review are methods that explicitly model the variability in the population with regard to time, such as in the modeling of disease progression and brain development. We highlight the applicability and limitations of state-of-the art approaches, how insights from the study of disease progression are helpful in developmental studies, and point to the directions of future research that is still necessary.

Using manifold learning for atlas selection in multi-atlas segmentation

Multi-atlas segmentation has been widely used to segment various anatomical structures. The success of this technique partly relies on the selection of atlases that are best mapped to a new target image after registration. Recently, manifold learning has been proposed as a method for atlas selection. Each manifold learning technique seeks to optimize a unique objective function. Therefore, different techniques produce different embeddings even when applied to the same data set. Previous studies used a single technique in their method and gave no reason for the choice of the manifold learning technique employed nor the theoretical grounds for the choice of the manifold parameters. In this study, we compare side-by-side the results given by 3 manifold learning techniques (Isomap, Laplacian Eigenmaps and Locally Linear Embedding) on the same data set. We assess the ability of those 3 different techniques to select the best atlases to combine in the framework of multi-atlas segmentation. First, a leave-one-out experiment is used to optimize our method on a set of 110 manually segmented atlases of hippocampi and find the manifold learning technique and associated manifold parameters that give the best segmentation accuracy. Then, the optimal parameters are used to automatically segment 30 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI). For our dataset, the selection of atlases with Locally Linear Embedding gives the best results. Our findings show that selection of atlases with manifold learning leads to segmentation accuracy close to or significantly higher than the state-of-the-art method and that accuracy can be increased by fine tuning the manifold learning process.

Registration of challenging pre-clinical brain images

The size and complexity of brain imaging studies in pre-clinical populations are increasing, and automated image analysis pipelines are urgently required. Pre-clinical populations can be subjected to controlled interventions (e.g., targeted lesions), which significantly change the appearance of the brain obtained by imaging. Existing systems for registration (the systematic alignment of scans into a consistent anatomical coordinate system), which assume image similarity to a reference scan, may fail when applied to these images. However, affine registration is a particularly vital pre-processing step for subsequent image analysis which is assumed to be an effective procedure in recent literature describing sophisticated techniques such as manifold learning. Therefore, in this paper, we present an affine registration solution that uses a graphical model of a population to decompose difficult pairwise registrations into a composition of steps using other members of the population. We developed this methodology in the context of a pre-clinical model of stroke in which large, variable hyper-intense lesions significantly impact registration performance. We tested this technique systematically in a simulated human population of brain tumour images before applying it to pre-clinical models of Parkinson's disease and stroke.

Multiple Atlas construction from a heterogeneous brain MR image collection

In this paper, we propose a novel framework for computing single or multiple atlases (templates) from a large population of images. Unlike many existing methods, our proposed approach is distinguished by its emphasis on the sharpness of the computed atlases and the requirement of rotational invariance. In particular, we argue that sharp atlas images that retain crucial and important anatomical features with high fidelity are more useful for many medical imaging applications when compared with the blurry and fuzzy atlas images computed by most existing methods. The geometric notion that underlies our approach is the idea of manifold learning in a quotient space, the quotient space of the image space by the rotations. We present an extension of the existing manifold learning approach to quotient spaces by using invariant metrics, and utilizing the manifold structure for partitioning the images into more homogeneous sub-collections, each of which can be represented by a single atlas image. Specifically, we propose a three-step algorithm. First, we partition the input images into subgroups using unsupervised or semi-supervised learning methods on manifolds. Then we formulate a convex optimization problem in each subgroup to locate the atlases and determine the crucial neighbors that are used in the realization step to form the template images. We have evaluated our algorithm using whole brain MR volumes from OASIS database. Experimental results demonstrate that the atlases computed using the proposed algorithm not only discover the brain structural changes in different age groups but also preserve important structural details and generally enjoy better image quality.

Locally Weighted Multi-atlas Construction

In image-based medical research, atlases are widely used in many tasks, for example, spatial normalization and segmentation. If atlases are regarded as representative patterns for a population of images, then multiple atlases are required for a heterogeneous population. In conventional atlas construction methods, the "unit" of representative patterns is images. Every input image is associated with its most similar atlas. As the number of subjects increases, the heterogeneity increases accordingly, and a big number of atlases may be needed. In this paper, we explore using region-wise, instead of image-wise, patterns to represent a population. Different parts of an input image is fuzzily associated with different atlases according to voxel-level association weights. In this way, regional structure patterns from different atlases can be combined together. Based on this model, we design a variational framework for multi-atlas construction. In the application to two T1-weighted MRI data sets, the method shows promising performance, in comparison with a conventional unbiased atlas construction method.

Multi-atlas segmentation with robust label transfer and label fusion

Multi-atlas segmentation has been widely applied in medical image analysis. This technique relies on image registration to transfer segmentation labels from pre-labeled atlases to a novel target image and applies label fusion to reduce errors produced by registration-based label transfer. To improve the performance of registration-based label transfer against registration errors, our first contribution is to propose a label transfer scheme that generates multiple warped versions of each atlas to one target image through registration paths obtained by composing inter-atlas registrations and atlas-target registrations. The problem of decreasing quality of warped atlases caused by accumulative errors in composing multiple registrations is properly addressed by an atlas selection method that is guided by atlas segmentations. To improve the performance of label fusion against registration errors, our second contribution is to integrate the probabilistic correspondence model employed by the non-local mean approach with the joint label fusion technique, both of which have shown excellent performance for label fusion. Experiments on mitral-valve segmentation in 3D transesophageal echocardiography (TEE) show the effectiveness of the proposed techniques.

Manifold learning of brain MRIs by deep learning

Manifold learning of medical images plays a potentially important role for modeling anatomical variability within a population with pplications that include segmentation, registration, and prediction of clinical parameters. This paper describes a novel method for learning the manifold of 3D brain images that, unlike most existing manifold learning methods, does not require the manifold space to be locally linear, and does not require a predefined similarity measure or a prebuilt proximity graph. Our manifold learning method is based on deep learning, a machine learning approach that uses layered networks (called deep belief networks, or DBNs) and has received much attention recently in the computer vision field due to their success in object recognition tasks. DBNs have traditionally been too computationally expensive for application to 3D images due to the large number of trainable parameters. Our primary contributions are (1) a much more computationally efficient training method for DBNs that makes training on 3D medical images with a resolution of up to 128 x 128 x 128 practical, and (2) the demonstration that DBNs can learn a low-dimensional manifold of brain volumes that detects modes of variations that correlate to demographic and disease parameters.

COMBINING REGIONAL METRICS FOR DISEASE-RELATED BRAIN POPULATION ANALYSIS

In this paper, we present a new metric combining regional measurements to improve image based population studies that use manifold learning techniques. These studies currently rely on a single score over the whole brain image domain. Thus, they require large amount of training data to uncover spatially complex variation in the whole brain impacted by diseases. We reduce the impact of this issue by first computing pairwise measurements in local regions separately and then combining regional measurements into a single pairwise metric. We apply the new metric to learn the manifold of ADNI data and evaluate the resulting morphological representation by fitting multiple linear regression models to the mini-mental state examination (MMSE) score. The regression models show that the morphological representations from the proposed metric achieves higher estimation accuracy of MMSE score compared to those from the conventional global scores.