Statistical Learning Methods for Neuroimaging Data Analysis with Applications

The aim of this review is to provide a comprehensive survey of statistical challenges in neuroimaging data analysis, from neuroimaging techniques to large-scale neuroimaging studies and statistical learning methods. We briefly review eight popular neuroimaging techniques and their potential applications in neuroscience research and clinical translation. We delineate four themes of neuroimaging data and review major image processing analysis methods for processing neuroimaging data at the individual level. We briefly review four large-scale neuroimaging-related studies and a consortium on imaging genomics and discuss four themes of neuroimaging data analysis at the population level. We review nine major population-based statistical analysis methods and their associated statistical challenges and present recent progress in statistical methodology to address these challenges.

Projective Diffeomorphic Mapping of Molecular Digital Pathology with Tissue MRI

Reconstructing dense 3D anatomical coordinates from 2D projective measurements has become a central problem in digital pathology for both animal models and human studies. Here we describe Projective Large Deformation Diffeomorphic Metric Mapping (LDDMM), a technique which projects diffeomorphic mappings of dense human magnetic resonance imaging (MRI) atlases at tissue scales onto sparse measurements at micrometre scales associated with histological and more general optical imaging modalities. We solve the problem of dense mapping surjectively onto histological sections by incorporating technologies for crossing modalities that use nonlinear scattering transforms to represent multiple radiomic-like textures at micron scales, together with a Gaussian mixture-model framework for modelling tears and distortions associated to each section. We highlight the significance of our method through incorporation of neuropathological measures and MRI, of relevance to the development of biomarkers for Alzheimer's disease and one instance of the integration of imaging data across the scales of clinical imaging and digital pathology.

Fast GPU 3D diffeomorphic image registration

3D image registration is one of the most fundamental and computationally expensive operations in medical image analysis. Here, we present a mixed-precision, Gauss-Newton-Krylov solver for diffeomorphic registration of two images. Our work extends the publicly available CLAIRE library to GPU architectures. Despite the importance of image registration, only a few implementations of large deformation diffeomorphic registration packages support GPUs. Our contributions are new algorithms to significantly reduce the run time of the two main computational kernels in CLAIRE: calculation of derivatives and scattered-data interpolation. We deploy (i) highly-optimized, mixed-precision GPU-kernels for the evaluation of scattered-data interpolation, (ii) replace Fast-Fourier-Transform (FFT)-based first-order derivatives with optimized 8th-order finite differences, and (iii) compare with state-of-the-art CPU and GPU implementations. As a highlight, we demonstrate that we can register 2563 clinical images in less than 6 seconds on a single NVIDIA Tesla V100. This amounts to over 20× speed-up over the current version of CLAIRE and over 30× speed-up over existing GPU implementations.

Image Morphing in Deep Feature Spaces: Theory and Applications

This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller and Younes (Int J Comput Vis 41(1):61-84, 2001) and Trouvé and Younes (Found Comput Math 5(2):173-198, 2005). For this model, a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space; the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.

Multimodal cross-registration and quantification of metric distortions in marmoset whole brain histology using diffeomorphic mappings

Whole brain neuroanatomy using tera-voxel light-microscopic data sets is of much current interest. A fundamental problem in this field is the mapping of individual brain data sets to a reference space. Previous work has not rigorously quantified in-vivo to ex-vivo distortions in brain geometry from tissue processing. Further, existing approaches focus on registering unimodal volumetric data; however, given the increasing interest in the marmoset model for neuroscience research and the importance of addressing individual brain architecture variations, new algorithms are necessary to cross-register multimodal data sets including MRIs and multiple histological series. Here we present a computational approach for same-subject multimodal MRI-guided reconstruction of a series of consecutive histological sections, jointly with diffeomorphic mapping to a reference atlas. We quantify the scale change during different stages of brain histological processing using the Jacobian determinant of the diffeomorphic transformations involved. By mapping the final image stacks to the ex-vivo post-fixation MRI, we show that (a) tape-transfer assisted histological sections can be reassembled accurately into 3D volumes with a local scale change of 2.0 ± 0.4% per axis dimension; in contrast, (b) tissue perfusion/fixation as assessed by mapping the in-vivo MRIs to the ex-vivo post fixation MRIs shows a larger median absolute scale change of 6.9 ± 2.1% per axis dimension. This is the first systematic quantification of local metric distortions associated with whole-brain histological processing, and we expect that the results will generalize to other species. These local scale changes will be important for computing local properties to create reference brain maps.

Analysis of shape data: From landmarks to elastic curves

Proliferation of high-resolution imaging data in recent years has led to sub-stantial improvements in the two popular approaches for analyzing shapes of data objects based on landmarks and/or continuous curves. We provide an expository account of elastic shape analysis of parametric planar curves representing shapes of two-dimensional (2D) objects by discussing its differences, and its commonalities, to the landmark-based approach. Particular attention is accorded to the role of reparameterization of a curve, which in addition to rotation, scaling and translation, represents an important shape-preserving transformation of a curve. The transition to the curve-based approach moves the mathematical setting of shape analysis from finite-dimensional non-Euclidean spaces to infinite-dimensional ones. We discuss some of the challenges associated with the infinite-dimensionality of the shape space, and illustrate the use of geometry-based methods in the computation of intrinsic statistical summaries and in the definition of statistical models on a 2D imaging dataset consisting of mouse vertebrae. We conclude with an overview of the current state-of-the-art in the field.

Simplifying Transforms for General Elastic Metrics on the Space of Plane Curves

In the shape analysis approach to computer vision problems, one treats shapes as points in an infinite-dimensional Riemannian manifold, thereby facilitating algorithms for statistical calculations such as geodesic distance between shapes and averaging of a collection of shapes. The performance of these algorithms depends heavily on the choice of the Riemannian metric. In the setting of plane curve shapes, attention has largely been focused on a two-parameter family of first order Sobolev metrics, referred to as elastic metrics. They are particularly useful due to the existence of simplifying coordinate transformations for particular parameter values, such as the well-known square-root velocity transform. In this paper, we extend the transformations appearing in the existing literature to a family of isometries, which take any elastic metric to the flat L ² metric. We also extend the transforms to treat piecewise linear curves and demonstrate the existence of optimal matchings over the diffeomorphism group in this setting. We conclude the paper with multiple examples of shape geodesics for open and closed curves. We also show the benefits of our approach in a simple classification experiment.

CLAIRE: A DISTRIBUTED-MEMORY SOLVER FOR CONSTRAINED LARGE DEFORMATION DIFFEOMORPHIC IMAGE REGISTRATION

With this work we release CLAIRE, a distributed-memory implementation of an effective solver for constrained large deformation diifeomorphic image registration problems in three dimensions. We consider an optimal control formulation. We invert for a stationary velocity field that parameterizes the deformation map. Our solver is based on a globalized, preconditioned, inexact reduced space Gauss‒Newton‒Krylov scheme. We exploit state-of-the-art techniques in scientific computing to develop an eifective solver that scales to thousands of distributed memory nodes on high-end clusters. We present the formulation, discuss algorithmic features, describe the software package, and introduce an improved preconditioner for the reduced space Hessian to speed up the convergence of our solver. We test registration performance on synthetic and real data. We Demonstrate registration accuracy on several neuroimaging datasets. We compare the performance of our scheme against diiferent flavors of the Demons algorithm for diifeomorphic image registration. We study convergence of our preconditioner and our overall algorithm. We report scalability results on state-of-the-art supercomputing platforms. We Demonstrate that we can solve registration problems for clinically relevant data sizes in two to four minutes on a standard compute node with 20 cores, attaining excellent data fidelity. With the present work we achieve a speedup of (on average) 5× with a peak performance of up to 17× compared to our former work.

A Review of Statistical Methods in Imaging Genetics

With the rapid growth of modern technology, many biomedical studies are being conducted to collect massive datasets with volumes of multi-modality imaging, genetic, neurocognitive, and clinical information from increasingly large cohorts. Simultaneously extracting and integrating rich and diverse heterogeneous information in neuroimaging and/or genomics from these big datasets could transform our understanding of how genetic variants impact brain structure and function, cognitive function, and brain-related disease risk across the lifespan. Such understanding is critical for diagnosis, prevention, and treatment of numerous complex brain-related disorders (e.g., schizophrenia and Alzheimer's disease). However, the development of analytical methods for the joint analysis of both high-dimensional imaging phenotypes and high-dimensional genetic data, a big data squared (BD2) problem, presents major computational and theoretical challenges for existing analytical methods. Besides the high-dimensional nature of BD2, various neuroimaging measures often exhibit strong spatial smoothness and dependence and genetic markers may have a natural dependence structure arising from linkage disequilibrium. We review some recent developments of various statistical techniques for imaging genetics, including massive univariate and voxel-wise approaches, reduced rank regression, mixture models, and group sparse multi-task regression. By doing so, we hope that this review may encourage others in the statistical community to enter into this new and exciting field of research.

On variational solutions for whole brain serial-section histology using a Sobolev prior in the computational anatomy random orbit model

This paper presents a variational framework for dense diffeomorphic atlas-mapping onto high-throughput histology stacks at the 20 μm meso-scale. The observed sections are modelled as Gaussian random fields conditioned on a sequence of unknown section by section rigid motions and unknown diffeomorphic transformation of a three-dimensional atlas. To regularize over the high-dimensionality of our parameter space (which is a product space of the rigid motion dimensions and the diffeomorphism dimensions), the histology stacks are modelled as arising from a first order Sobolev space smoothness prior. We show that the joint maximum a-posteriori, penalized-likelihood estimator of our high dimensional parameter space emerges as a joint optimization interleaving rigid motion estimation for histology restacking and large deformation diffeomorphic metric mapping to atlas coordinates. We show that joint optimization in this parameter space solves the classical curvature non-identifiability of the histology stacking problem. The algorithms are demonstrated on a collection of whole-brain histological image stacks from the Mouse Brain Architecture Project.

New developments in neural tracing techniques have motivated the widespread use of histology as a modality for exploring the circuitry of the brain. Automated mapping of pre-labeled atlases onto modern large datasets of histological imagery is a critical step for elucidating the brain’s neural circuitry and shape. This task is challenging as histological sections are imaged independently and the reconstruction of the unsectioned volume is nontrivial. Typically, neuroanatomists use reference volumes of the same subject (e.g. MRI) to guide reconstruction. However, obtaining reference imagery is often non-standard, as in high-throughput animal models like mouse histology. Others have proposed using anatomical atlases as guides, but have not accounted for the intrinsic nonlinear shape difference from atlas to subject. Our method addresses these limitations by jointly optimizing reconstruction informed by an atlas simultaneously with the nonlinear change of coordinates that encapsulates anatomical variation. This accounts for intrinsic shape differences and enables rigorous, direct comparisons of atlas and subject coordinates. Using simulations, we demonstrate that our method recovers the reconstruction parameters more accurately than atlas-free models and innately produces accurate segmentations from simultaneous atlas mapping. We also demonstrate our method on the Mouse Brain Architecture dataset, successfully mapping and reconstructing over 1000 brains.

The use of a custom-made virtual template for corrective surgeries of asymmetric patients: proof of principle and a multi-center end-user survey

AIM

To evaluate the utility of an individualized template for corrective surgeries for patients suffering from mandibular asymmetry.

MATERIALS AND METHOD

Twenty patients with history of favorable clinical outcome of the correction of their mandibular asymmetry were chosen. CBCTs were taken before and 6 weeks postoperative using NewTom 3G. Each volume is mirrored and registered on the cranial base. Surface models for the mandible and its registered mirror were used to compute a template using deformable fluid registration. Surgery was simulated based of the resulting template. A multi-center survey using "Qualtrics" was conducted to gain clinical feedback of 20 surgeons/orthodontists comparing treatment outcomes.

RESULTS

Twenty-three clinicians participated. More clinicians rated simulated outcome to be "Good," whereas the actual surgical outcomes were rated as "fair" and "poor." This was true for regional appraisal for the chin, Rami, and body of the mandible as well as the overall assessment of the outcome of surgeries. The gains of computer-assisted simulation tend to be greater for difficult cases especially for the body of the mandible, then the chin, and then the Ramus correction.

CONCLUSIONS

This approach has the potential to optimize and increase the predictability of the outcome of craniofacial corrective surgeries for asymmetric patients.

Computational anatomy and diffeomorphometry: A dynamical systems model of neuroanatomy in the soft condensed matter continuum

The nonlinear systems models of computational anatomy that have emerged over the past several decades are a synthesis of three significant areas of computational science and biological modeling. First is the algebraic model of biological shape as a Riemannian orbit, a set of objects under diffeomorphic action. Second is the embedding of anatomical shapes into the soft condensed matter physics continuum via the extension of the Euler equations to geodesic, smooth flows with inverses, encoding divergence for the compressibility of atrophy and expansion of growth. Third, is making human shape and form a metrizable space via geodesic connections of coordinate systems. These three themes place our formalism into the modern data science world of personalized medicine supporting inference of high-dimensional anatomical phenotypes for studying neurodegeneration and neurodevelopment. The dynamical systems model of growth and atrophy that emerges is one which is organized in terms of forces, accelerations, velocities, and displacements, with the associated Hamiltonian momentum and the diffeomorphic flow acting as the state, and the smooth vector field the control. The forces that enter the model derive from external measurements through which the dynamical system must flow, and the internal potential energies of structures making up the soft condensed matter. We examine numerous examples on growth and atrophy. This article is categorized under: Analytical and Computational Methods > Computational Methods Laboratory Methods and Technologies > Imaging Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models.

Parametric Surface Diffeomorphometry for Low Dimensional Embeddings of Dense Segmentations and Imagery

In the field of Computational Anatomy, biological form (including our focus, neuroanatomy) is studied quantitatively through the action of the diffeomorphism group on example anatomies - a technique called diffeomorphometry. Here we design an algorithm within this framework to pass from dense objects common in neuromaging studies (binary segmentations, structural images) to a sparse representation defined on the surface boundaries of anatomical structures, and embedded into the low dimensional coordinates of a parametric model. Our main new contribution is to introduce an expanded group action to simultaneously deform surfaces through direct mapping of points, as well as images through functional composition with the inverse. This allows us to index the diffeomorphisms with respect to two-dimensional surface geometries like subcortical gray matter structures, but explicitly map onto cost functions determined by noisy 3-dimensional measurements. We consider models generated from empirical covariance of training data, as well as bandlimited (Laplace-Beltrami eigenfunction) models when no such data is available. We show applications to noisy or anomalous segmentations, and other typical problems in neuroimaging studies. We reproduce statistical results detecting changes in Alzheimer's disease, despite dimensionality reduction. Lastly we apply our algorithm to the common problem of segmenting subcortical structures from T1 MR images.

The conceptual framework of ontogenetic trajectories: parallel transport allows the recognition and visualization of pure deformation patterns

Ontogeny is usually studied by analyzing a deformation series spanning over juvenile to adult shapes. In geometric morphometrics, this approach implies applying generalized Procrustes analysis coupled with principal component analysis on multiple individuals or multiple species datasets. The trouble with such a procedure is that it mixes intra- and inter-group variation. While MANCOVA models are relevant statistical/mathematical tools to draw inferences about the similarities of trajectories, if one wants to observe and interpret the morphological deformation alone by filtering inter-group variability, a particular tool, namely parallel transport, is necessary. In the context of ontogenetic trajectories, one should firstly perform separate multivariate regressions between shape and size, using regression predictions to estimate within-group deformations relative to the smallest individuals. These deformations are then applied to a common reference (the mean of per-group smallest individuals). The estimation of deformations can be performed on the Riemannian manifold by using sophisticated connection metrics. Nevertheless, parallel transport can be effectively achieved by estimating deformations in the Euclidean space via ordinary Procrustes analysis. This approach proved very useful in comparing ontogenetic trajectories of species presenting large morphological differences at early developmental stages.

A LAGRANGIAN GAUSS-NEWTON-KRYLOV SOLVER FOR MASS- AND INTENSITY-PRESERVING DIFFEOMORPHIC IMAGE REGISTRATION

We present an efficient solver for diffeomorphic image registration problems in the framework of Large Deformations Diffeomorphic Metric Mappings (LDDMM). We use an optimal control formulation, in which the velocity field of a hyperbolic PDE needs to be found such that the distance between the final state of the system (the transformed/transported template image) and the observation (the reference image) is minimized. Our solver supports both stationary and non-stationary (i.e., transient or time-dependent) velocity fields. As transformation models, we consider both the transport equation (assuming intensities are preserved during the deformation) and the continuity equation (assuming mass-preservation). We consider the reduced form of the optimal control problem and solve the resulting unconstrained optimization problem using a discretize-then-optimize approach. A key contribution is the elimination of the PDE constraint using a Lagrangian hyperbolic PDE solver. Lagrangian methods rely on the concept of characteristic curves. We approximate these curves using a fourth-order Runge-Kutta method. We also present an efficient algorithm for computing the derivatives of the final state of the system with respect to the velocity field. This allows us to use fast Gauss-Newton based methods. We present quickly converging iterative linear solvers using spectral preconditioners that render the overall optimization efficient and scalable. Our method is embedded into the image registration framework FAIR and, thus, supports the most commonly used similarity measures and regularization functionals. We demonstrate the potential of our new approach using several synthetic and real world test problems with up to 14.7 million degrees of freedom.

MIND Demons: Symmetric Diffeomorphic Deformable Registration of MR and CT for Image-Guided Spine Surgery

Intraoperative localization of target anatomy and critical structures defined in preoperative MR/CT images can be achieved through the use of multimodality deformable registration. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality-independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, finds a deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the integrated velocity fields, a modality-insensitive similarity function suitable to multimodality images, and smoothness on the diffeomorphisms themselves. Direct optimization without relying on the exponential map and stationary velocity field approximation used in conventional diffeomorphic Demons is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, normalized MI (NMI) Demons, and MIND with a diffusion-based registration method (MIND-elastic). The method yielded sub-voxel invertibility (0.008 mm) and nonzero-positive Jacobian determinants. It also showed improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.7 mm compared to 11.3, 3.1, 5.6, and 2.4 mm for MI FFD, LMI FFD, NMI Demons, and MIND-elastic methods, respectively. Validation in clinical studies demonstrated realistic deformations with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

Conformal invariants for multiply connected surfaces: Application to landmark curve-based brain morphometry analysis

Landmark curves were widely adopted in neuroimaging research for surface correspondence computation and quantified morphometry analysis. However, most of the landmark based morphometry studies only focused on landmark curve shape difference. Here we propose to compute a set of conformal invariant-based shape indices, which are associated with the landmark curve induced boundary lengths in the hyperbolic parameter domain. Such shape indices may be used to identify which surfaces are conformally equivalent and further quantitatively measure surface deformation. With the surface Ricci flow method, we can conformally map a multiply connected surface to the Poincaré disk. Our algorithm provides a stable method to compute the shape index values in the 2D (Poincaré Disk) parameter domain. The proposed shape indices are succinct, intrinsic and informative. Experimental results with synthetic data and 3D MRI data demonstrate that our method is invariant under isometric transformations and able to detect brain surface abnormalities. We also applied the new shape indices to analyze brain morphometry abnormalities associated with Alzheimer' s disease (AD). We studied the baseline MRI scans of a set of healthy control and AD patients from the Alzheimer' s Disease Neuroimaging Initiative (ADNI: 30 healthy control subjects vs. 30 AD patients). Although the lengths of the landmarks in Euclidean space, cortical surface area, and volume features did not differ between the two groups, our conformal invariant based shape indices revealed significant differences by Hotelling' s T² test. The novel conformal invariant shape indices may offer a new sensitive biomarker and enrich our brain imaging analysis toolset for studying diagnosis and prognosis of AD.

Dynamic Facial Expression Recognition With Atlas Construction and Sparse Representation

In this paper, a new dynamic facial expression recognition method is proposed. Dynamic facial expression recognition is formulated as a longitudinal groupwise registration problem. The main contributions of this method lie in the following aspects: 1) subject-specific facial feature movements of different expressions are described by a diffeomorphic growth model; 2) salient longitudinal facial expression atlas is built for each expression by a sparse groupwise image registration method, which can describe the overall facial feature changes among the whole population and can suppress the bias due to large intersubject facial variations; and 3) both the image appearance information in spatial domain and topological evolution information in temporal domain are used to guide recognition by a sparse representation method. The proposed framework has been extensively evaluated on five databases for different applications: the extended Cohn-Kanade, MMI, FERA, and AFEW databases for dynamic facial expression recognition, and UNBC-McMaster database for spontaneous pain expression monitoring. This framework is also compared with several state-of-the-art dynamic facial expression recognition methods. The experimental results demonstrate that the recognition rates of the new method are consistently higher than other methods under comparison.

MIND Demons for MR-to-CT Deformable Image Registration In Image-Guided Spine Surgery

PURPOSE

Localization of target anatomy and critical structures defined in preoperative MR images can be achieved by means of multi-modality deformable registration to intraoperative CT. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration.

METHOD

The method, called MIND Demons, solves for the deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the velocity fields and the diffeomorphisms, a modality-insensitive similarity function suitable to multi-modality images, and constraints on geodesics in Lagrangian coordinates. Direct optimization (without relying on an exponential map of stationary velocity fields used in conventional diffeomorphic Demons) is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, in phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to conventional mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, and normalized MI (NMI) Demons.

RESULT

The method yielded sub-voxel invertibility (0.006 mm) and nonsingular spatial Jacobians with capability to preserve local orientation and topology. It demonstrated improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.5 mm compared to 10.9, 2.3, and 4.6 mm for MI FFD, LMI FFD, and NMI Demons methods, respectively. Validation in clinical studies demonstrated realistic deformation with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

CONCLUSIONS

A modality-independent deformable registration method has been developed to estimate a viscoelastic diffeomorphic map between preoperative MR and intraoperative CT. The method yields registration accuracy suitable to application in image-guided spine surgery across a broad range of anatomical sites and modes of deformation.

Temporal filtering of longitudinal brain magnetic resonance images for consistent segmentation

Longitudinal analysis of magnetic resonance images of the human brain provides knowledge of brain changes during both normal aging as well as the progression of many diseases. Previous longitudinal segmentation methods have either ignored temporal information or have incorporated temporal consistency constraints within the algorithm. In this work, we assume that some anatomical brain changes can be explained by temporal transitions in image intensities. Once the images are aligned in the same space, the intensities of each scan at the same voxel constitute a temporal (or 4D) intensity trend at that voxel. Temporal intensity variations due to noise or other artifacts are corrected by a 4D intensity-based filter that smooths the intensity values where appropriate, while preserving real anatomical changes such as atrophy. Here smoothing refers to removal of sudden changes or discontinuities in intensities. Images processed with the 4D filter can be used as a pre-processing step to any segmentation method. We show that such a longitudinal pre-processing step produces robust and consistent longitudinal segmentation results, even when applying 3D segmentation algorithms. We compare with state-of-the-art 4D segmentation algorithms. Specifically, we experimented on three longitudinal datasets containing 4-12 time-points, and showed that the 4D temporal filter is more robust and has more power in distinguishing between healthy subjects and those with dementia, mild cognitive impairment, as well as different phenotypes of multiple sclerosis.

Hamiltonian Systems and Optimal Control in Computational Anatomy: 100 Years Since D'Arcy Thompson

The Computational Anatomy project is the morphome-scale study of shape and form, which we model as an orbit under diffeomorphic group action. Metric comparison calculates the geodesic length of the diffeomorphic flow connecting one form to another. Geodesic connection provides a positioning system for coordinatizing the forms and positioning their associated functional information. This article reviews progress since the Euler-Lagrange characterization of the geodesics a decade ago. Geodesic positioning is posed as a series of problems in Hamiltonian control, which emphasize the key reduction from the Eulerian momentum with dimension of the flow of the group, to the parametric coordinates appropriate to the dimension of the submanifolds being positioned. The Hamiltonian viewpoint provides important extensions of the core setting to new, object-informed positioning systems. Several submanifold mapping problems are discussed as they apply to metamorphosis, multiple shape spaces, and longitudinal time series studies of growth and atrophy via shape splines.

A Riemannian Framework for Intrinsic Comparison of Closed Genus-Zero Shapes

We present a framework for intrinsic comparison of surface metric structures and curvatures. This work parallels the work of Kurtek et al. on parameterization-invariant comparison of genus zero shapes. Here, instead of comparing the embedding of spherically parameterized surfaces in space, we focus on the first fundamental form. To ensure that the distance on spherical metric tensor fields is invariant to parameterization, we apply the conjugation-invariant metric arising from the L2 norm on symmetric positive definite matrices. As a reparameterization changes the metric tensor by a congruent Jacobian transform, this metric perfectly suits our purpose. The result is an intrinsic comparison of shape metric structure that does not depend on the specifics of a spherical mapping. Further, when restricted to tensors of fixed volume form, the manifold of metric tensor fields and its quotient of the group of unitary diffeomorphisms becomes a proper metric manifold that is geodesically complete. Exploiting this fact, and augmenting the metric with analogous metrics on curvatures, we derive a complete Riemannian framework for shape comparison and reconstruction. A by-product of our framework is a near-isometric and curvature-preserving mapping between surfaces. The correspondence is optimized using the fast spherical fluid algorithm. We validate our framework using several subcortical boundary surface models from the ADNI dataset.

A Bayesian approach to the creation of a study-customized neonatal brain atlas

Atlas-based image analysis (ABA), in which an anatomical "parcellation map" is used for parcel-by-parcel image quantification, is widely used to analyze anatomical and functional changes related to brain development, aging, and various diseases. The parcellation maps are often created based on common MRI templates, which allow users to transform the template to target images, or vice versa, to perform parcel-by-parcel statistics, and report the scientific findings based on common anatomical parcels. The use of a study-specific template, which represents the anatomical features of the study population better than common templates, is preferable for accurate anatomical labeling; however, the creation of a parcellation map for a study-specific template is extremely labor intensive, and the definitions of anatomical boundaries are not necessarily compatible with those of the common template. In this study, we employed a volume-based template estimation (VTE) method to create a neonatal brain template customized to a study population, while keeping the anatomical parcellation identical to that of a common MRI atlas. The VTE was used to morph the standardized parcellation map of the JHU-neonate-SS atlas to capture the anatomical features of a study population. The resultant "study-customized" T1-weighted and diffusion tensor imaging (DTI) template, with three-dimensional anatomical parcellation that defined 122 brain regions, was compared with the JHU-neonate-SS atlas, in terms of the registration accuracy. A pronounced increase in the accuracy of cortical parcellation and superior tensor alignment were observed when the customized template was used. With the customized atlas-based analysis, the fractional anisotropy (FA) detected closely approximated the manual measurements. This tool provides a solution for achieving normalization-based measurements with increased accuracy, while reporting scientific findings in a consistent framework.

Diffeomorphometry and geodesic positioning systems for human anatomy

The Computational Anatomy project has largely been a study of large deformations within a Riemannian framework as an efficient point of view for generating metrics between anatomical configurations. This approach turns D'Arcy Thompson's comparative morphology of human biological shape and form into a metrizable space. Since the metric is constructed based on the geodesic length of the flows of diffeomorphisms connecting the forms, we call it diffeomorphometry. Just as importantly, since the flows describe algebraic group action on anatomical submanifolds and associated functional measurements, they become the basis for positioning information, which we term geodesic positioning. As well the geodesic connections provide Riemannian coordinates for locating forms in the anatomical orbit, which we call geodesic coordinates. These three components taken together - the metric, geodesic positioning of information, and geodesic coordinates - we term the geodesic positioning system. We illustrate via several examples in human and biological coordinate systems and machine learning of the statistical representation of shape and form.

Quantifying anatomical shape variations in neurological disorders

We develop a multivariate analysis of brain anatomy to identify the relevant shape deformation patterns and quantify the shape changes that explain corresponding variations in clinical neuropsychological measures. We use kernel Partial Least Squares (PLS) and formulate a regression model in the tangent space of the manifold of diffeomorphisms characterized by deformation momenta. The scalar deformation momenta completely encode the diffeomorphic changes in anatomical shape. In this model, the clinical measures are the response variables, while the anatomical variability is treated as the independent variable. To better understand the "shape-clinical response" relationship, we also control for demographic confounders, such as age, gender, and years of education in our regression model. We evaluate the proposed methodology on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database using baseline structural MR imaging data and neuropsychological evaluation test scores. We demonstrate the ability of our model to quantify the anatomical deformations in units of clinical response. Our results also demonstrate that the proposed method is generic and generates reliable shape deformations both in terms of the extracted patterns and the amount of shape changes. We found that while the hippocampus and amygdala emerge as mainly responsible for changes in test scores for global measures of dementia and memory function, they are not a determinant factor for executive function. Another critical finding was the appearance of thalamus and putamen as most important regions that relate to executive function. These resulting anatomical regions were consistent with very high confidence irrespective of the size of the population used in the study. This data-driven global analysis of brain anatomy was able to reach similar conclusions as other studies in Alzheimer's disease based on predefined ROIs, together with the identification of other new patterns of deformation. The proposed methodology thus holds promise for discovering new patterns of shape changes in the human brain that could add to our understanding of disease progression in neurological disorders.

Video texture synthesis with multi-frame LBP-TOP and diffeomorphic growth model

Video texture synthesis is the process of providing a continuous and infinitely varying stream of frames, which plays an important role in computer vision and graphics. However, it still remains a challenging problem to generate high-quality synthesis results. Considering the two key factors that affect the synthesis performance, frame representation and blending artifacts, we improve the synthesis performance from two aspects: 1) Effective frame representation is designed to capture both the image appearance information in spatial domain and the longitudinal information in temporal domain. 2) Artifacts that degrade the synthesis quality are significantly suppressed on the basis of a diffeomorphic growth model. The proposed video texture synthesis approach has two major stages: video stitching stage and transition smoothing stage. In the first stage, a video texture synthesis model is proposed to generate an infinite video flow. To find similar frames for stitching video clips, we present a new spatial-temporal descriptor to provide an effective representation for different types of dynamic textures. In the second stage, a smoothing method is proposed to improve synthesis quality, especially in the aspect of temporal continuity. It aims to establish a diffeomorphic growth model to emulate local dynamics around stitched frames. The proposed approach is thoroughly tested on public databases and videos from the Internet, and is evaluated in both qualitative and quantitative ways.

Longitudinal image registration with temporally-dependent image similarity measure

Longitudinal imaging studies are frequently used to investigate temporal changes in brain morphology and often require spatial correspondence between images achieved through image registration. Beside morphological changes, image intensity may also change over time, for example when studying brain maturation. However, such intensity changes are not accounted for in image similarity measures for standard image registration methods. Hence, 1) local similarity measures, 2) methods estimating intensity transformations between images, and 3) metamorphosis approaches have been developed to either achieve robustness with respect to intensity changes or to simultaneously capture spatial and intensity changes. For these methods, longitudinal intensity changes are not explicitly modeled and images are treated as independent static samples. Here, we propose a model-based image similarity measure for longitudinal image registration that estimates a temporal model of intensity change using all available images simultaneously.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Bayesian Parameter Estimation and Segmentation in the Multi-Atlas Random Orbit Model

This paper examines the multiple atlas random diffeomorphic orbit model in Computational Anatomy (CA) for parameter estimation and segmentation of subcortical and ventricular neuroanatomy in magnetic resonance imagery. We assume that there exist multiple magnetic resonance image (MRI) atlases, each atlas containing a collection of locally-defined charts in the brain generated via manual delineation of the structures of interest. We focus on maximum a posteriori estimation of high dimensional segmentations of MR within the class of generative models representing the observed MRI as a conditionally Gaussian random field, conditioned on the atlas charts and the diffeomorphic change of coordinates of each chart that generates it. The charts and their diffeomorphic correspondences are unknown and viewed as latent or hidden variables. We demonstrate that the expectation-maximization (EM) algorithm arises naturally, yielding the likelihood-fusion equation which the a posteriori estimator of the segmentation labels maximizes. The likelihoods being fused are modeled as conditionally Gaussian random fields with mean fields a function of each atlas chart under its diffeomorphic change of coordinates onto the target. The conditional-mean in the EM algorithm specifies the convex weights with which the chart-specific likelihoods are fused. The multiple atlases with the associated convex weights imply that the posterior distribution is a multi-modal representation of the measured MRI. Segmentation results for subcortical and ventricular structures of subjects, within populations of demented subjects, are demonstrated, including the use of multiple atlases across multiple diseased groups.

Multiple Atlas construction from a heterogeneous brain MR image collection

In this paper, we propose a novel framework for computing single or multiple atlases (templates) from a large population of images. Unlike many existing methods, our proposed approach is distinguished by its emphasis on the sharpness of the computed atlases and the requirement of rotational invariance. In particular, we argue that sharp atlas images that retain crucial and important anatomical features with high fidelity are more useful for many medical imaging applications when compared with the blurry and fuzzy atlas images computed by most existing methods. The geometric notion that underlies our approach is the idea of manifold learning in a quotient space, the quotient space of the image space by the rotations. We present an extension of the existing manifold learning approach to quotient spaces by using invariant metrics, and utilizing the manifold structure for partitioning the images into more homogeneous sub-collections, each of which can be represented by a single atlas image. Specifically, we propose a three-step algorithm. First, we partition the input images into subgroups using unsupervised or semi-supervised learning methods on manifolds. Then we formulate a convex optimization problem in each subgroup to locate the atlases and determine the crucial neighbors that are used in the realization step to form the template images. We have evaluated our algorithm using whole brain MR volumes from OASIS database. Experimental results demonstrate that the atlases computed using the proposed algorithm not only discover the brain structural changes in different age groups but also preserve important structural details and generally enjoy better image quality.

Longitudinal image registration with non-uniform appearance change

Longitudinal imaging studies are frequently used to investigate temporal changes in brain morphology. Image intensity may also change over time, for example when studying brain maturation. However, such intensity changes are not accounted for in image similarity measures for standard image registration methods. Hence, (i) local similarity measures, (ii) methods estimating intensity transformations between images, and (iii) metamorphosis approaches have been developed to either achieve robustness with respect to intensity changes or to simultaneously capture spatial and intensity changes. For these methods, longitudinal intensity changes are not explicitly modeled and images are treated as independent static samples. Here, we propose a model-based image similarity measure for longitudinal image registration in the presence of spatially non-uniform intensity change.

Metamorphic geodesic regression

We propose a metamorphic geodesic regression approach approximating spatial transformations for image time-series while simultaneously accounting for intensity changes. Such changes occur for example in magnetic resonance imaging (MRI) studies of the developing brain due to myelination. To simplify computations we propose an approximate metamorphic geodesic regression formulation that only requires pairwise computations of image metamorphoses. The approximated solution is an appropriately weighted average of initial momenta. To obtain initial momenta reliably, we develop a shooting method for image metamorphosis.

Nonrigid brain MR image registration using uniform spherical region descriptor

There are two main issues that make nonrigid image registration a challenging task. First, voxel intensity similarity may not be necessarily equivalent to anatomical similarity in the image correspondence searching process. Second, during the imaging process, some interferences such as unexpected rotations of input volumes and monotonic gray-level bias fields can adversely affect the registration quality. In this paper, a new feature-based nonrigid image registration method is proposed. The proposed method is based on a new type of image feature, namely, uniform spherical region descriptor (USRD), as signatures for each voxel. The USRD is rotation and monotonic gray-level transformation invariant and can be efficiently calculated. The registration process is therefore formulated as a feature matching problem. The USRD feature is integrated with the Markov random field labeling framework in which energy function is defined for registration. The energy function is then optimized by the α-expansion algorithm. The proposed method has been compared with five state-of-the-art registration approaches on both the simulated and real 3-D databases obtained from the BrainWeb and Internet Brain Segmentation Repository, respectively. Experimental results demonstrate that the proposed method can achieve high registration accuracy and reliable robustness behavior.

Simultaneous multi-scale registration using large deformation diffeomorphic metric mapping

In the framework of large deformation diffeomorphic metric mapping (LDDMM), we present a practical methodology to integrate prior knowledge about the registered shapes in the regularizing metric. Our goal is to perform rich anatomical shape comparisons from volumetric images with the mathematical properties offered by the LDDMM framework. We first present the notion of characteristic scale at which image features are deformed. We then propose a methodology to compare anatomical shape variations in a multi-scale fashion, i.e., at several characteristic scales simultaneously. In this context, we propose a strategy to quantitatively measure the feature differences observed at each characteristic scale separately. After describing our methodology, we illustrate the performance of the method on phantom data. We then compare the ability of our method to segregate a group of subjects having Alzheimer's disease and a group of controls with a classical coarse to fine approach, on standard 3D MR longitudinal brain images. We finally apply the approach to quantify the anatomical development of the human brain from 3D MR longitudinal images of pre-term babies. Results show that our method registers accurately volumetric images containing feature differences at several scales simultaneously with smooth deformations.

FUZZY SIMILARITIES TO COMPARE DEFORMED IMAGES

We use the concept of fuzzy similarity to compare the objects of a free image algebra (a set of objects which a group is acting on). In particular, we study those fuzzy similarities that are preserved by the action of the group. Later we consider a deformation mechanism of the image algebra and trackle the problem of comparing deformed images. For that purpose, we characterize those deformation mechanisms that are equivalent to the induced action from a subgroup of the group of deformations. In that case, by using techniques from group representation theory, we extend any fuzzy similarity defined on the image algebra to a fuzzy similarity defined on the whole space of deformed images. Moreover, we prove that the invariance of the similarity with respect to the group action is preserved by this extension.

A novel framework for longitudinal atlas construction with groupwise registration of subject image sequences

Longitudinal atlas construction plays an important role in medical image analysis. Given a set of longitudinal images from different subjects, the task of longitudinal atlas construction is to build an atlas sequence which can represent the trend of anatomical changes of the population. The major challenge for longitudinal atlas construction is how to effectively incorporate both the subject-specific information and population information to build the unbiased atlases. In this paper, a novel groupwise longitudinal atlas construction framework is proposed to address this challenge, and the main contributions of the proposed framework lie in the following aspects: (1) The subject-specific longitudinal information is captured by building the growth model for each subject. (2) The longitudinal atlas sequence is constructed by performing groupwise registration among all the subject image sequences, and only one transformation is needed to transform each subject's image sequence to the atlas space. The constructed longitudinal atlases are unbiased and no explicit template is assumed. (3) The proposed method is general, where the number of longitudinal images of each subject and the time points at which they are taken can be different. The proposed method is extensively evaluated on two longitudinal databases, namely the BLSA and ADNI databases, to construct the longitudinal atlas sequence. It is also compared with a state-of-the-art longitudinal atlas construction algorithm based on kernel regression on the temporal domain. Experimental results demonstrate that the proposed method consistently achieves higher registration accuracies and more consistent spatial-temporal correspondences than the compared method on both databases.

Efficient Probabilistic and Geometric Anatomical Mapping Using Particle Mesh Approximation on GPUs

Deformable image registration in the presence of considerable contrast differences and large size and shape changes presents significant research challenges. First, it requires a robust registration framework that does not depend on intensity measurements and can handle large nonlinear shape variations. Second, it involves the expensive computation of nonlinear deformations with high degrees of freedom. Often it takes a significant amount of computation time and thus becomes infeasible for practical purposes. In this paper, we present a solution based on two key ideas: a new registration method that generates a mapping between anatomies represented as a multicompartment model of class posterior images and geometries and an implementation of the algorithm using particle mesh approximation on Graphical Processing Units (GPUs) to fulfill the computational requirements. We show results on the registrations of neonatal to 2-year old infant MRIs. Quantitative validation demonstrates that our proposed method generates registrations that better maintain the consistency of anatomical structures over time and provides transformations that better preserve structures undergoing large deformations than transformations obtained by standard intensity-only registration. We also achieve the speedup of three orders of magnitudes compared to a CPU reference implementation, making it possible to use the technique in time-critical applications.

Automated analysis of craniofacial morphology using magnetic resonance images

Quantitative analysis of craniofacial morphology is of interest to scholars working in a wide variety of disciplines, such as anthropology, developmental biology, and medicine. T1-weighted (anatomical) magnetic resonance images (MRI) provide excellent contrast between soft tissues. Given its three-dimensional nature, MRI represents an ideal imaging modality for the analysis of craniofacial structure in living individuals. Here we describe how T1-weighted MR images, acquired to examine brain anatomy, can also be used to analyze facial features. Using a sample of typically developing adolescents from the Saguenay Youth Study (N = 597; 292 male, 305 female, ages: 12 to 18 years), we quantified inter-individual variations in craniofacial structure in two ways. First, we adapted existing nonlinear registration-based morphological techniques to generate iteratively a group-wise population average of craniofacial features. The nonlinear transformations were used to map the craniofacial structure of each individual to the population average. Using voxel-wise measures of expansion and contraction, we then examined the effects of sex and age on inter-individual variations in facial features. Second, we employed a landmark-based approach to quantify variations in face surfaces. This approach involves: (a) placing 56 landmarks (forehead, nose, lips, jaw-line, cheekbones, and eyes) on a surface representation of the MRI-based group average; (b) warping the landmarks to the individual faces using the inverse nonlinear transformation estimated for each person; and (3) using a principal components analysis (PCA) of the warped landmarks to identify facial features (i.e. clusters of landmarks) that vary in our sample in a correlated fashion. As with the voxel-wise analysis of the deformation fields, we examined the effects of sex and age on the PCA-derived spatial relationships between facial features. Both methods demonstrated significant sexual dimorphism in craniofacial structure in areas such as the chin, mandible, lips, and nose.

Content-based image retrieval in radiology: current status and future directions

Diagnostic radiology requires accurate interpretation of complex signals in medical images. Content-based image retrieval (CBIR) techniques could be valuable to radiologists in assessing medical images by identifying similar images in large archives that could assist with decision support. Many advances have occurred in CBIR, and a variety of systems have appeared in nonmedical domains; however, permeation of these methods into radiology has been limited. Our goal in this review is to survey CBIR methods and systems from the perspective of application to radiology and to identify approaches developed in nonmedical applications that could be translated to radiology. Radiology images pose specific challenges compared with images in the consumer domain; they contain varied, rich, and often subtle features that need to be recognized in assessing image similarity. Radiology images also provide rich opportunities for CBIR: rich metadata about image semantics are provided by radiologists, and this information is not yet being used to its fullest advantage in CBIR systems. By integrating pixel-based and metadata-based image feature analysis, substantial advances of CBIR in medicine could ensue, with CBIR systems becoming an important tool in radiology practice.