1
|
Amor BB, Arguillere S, Shao L. ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023; 45:3707-3720. [PMID: 35560103 DOI: 10.1109/tpami.2022.3174908] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In deformable registration, the Riemannian framework - Large Deformation Diffeomorphic Metric Mapping, or LDDMM for short - has inspired numerous techniques for comparing, deforming, averaging and analyzing shapes or images. Grounded in flows of vector fields, akin to the equations of motion used in fluid dynamics, LDDMM algorithms solve the flow equation in the space of plausible deformations, i.e., diffeomorphisms. In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on an Euler's discretization scheme. The central idea is to represent time-dependent velocity fields as fully connected ReLU neural networks (building blocks) and derive optimal weights by minimizing a regularized loss function. Computing minimizing paths between deformations, thus between shapes, turns to find optimal network parameters by back-propagating over the intermediate building blocks. Geometrically, at each time step, our algorithm searches for an optimal partition of the space into multiple polytopes, and then computes optimal velocity vectors as affine transformations on each of these polytopes. As a result, different parts of the shape, even if they are close (such as two fingers of a hand), can be made to belong to different polytopes, and therefore be moved in different directions without costing too much energy. Importantly, we show how diffeomorphic transformations, or more precisely bilipshitz transformations, are predicted by our registration algorithm. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations. We thus provide essential foundations for more advanced shape variability analysis under a novel joint geometric-neural networks Riemannian-like framework, i.e., ResNet-LDDMM.
Collapse
|
2
|
Corrigendum to "Robust Diffeomorphic Mapping via Geodesically Controlled Active Shapes". Int J Biomed Imaging 2021; 2021:9780202. [PMID: 34135947 PMCID: PMC8179761 DOI: 10.1155/2021/9780202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Accepted: 05/21/2021] [Indexed: 11/29/2022] Open
|
3
|
Lee BC, Lin MK, Fu Y, Hata J, Miller MI, Mitra PP. Multimodal cross-registration and quantification of metric distortions in marmoset whole brain histology using diffeomorphic mappings. J Comp Neurol 2020; 529:281-295. [PMID: 32406083 DOI: 10.1002/cne.24946] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2019] [Revised: 03/23/2020] [Accepted: 04/30/2020] [Indexed: 11/08/2022]
Abstract
Whole brain neuroanatomy using tera-voxel light-microscopic data sets is of much current interest. A fundamental problem in this field is the mapping of individual brain data sets to a reference space. Previous work has not rigorously quantified in-vivo to ex-vivo distortions in brain geometry from tissue processing. Further, existing approaches focus on registering unimodal volumetric data; however, given the increasing interest in the marmoset model for neuroscience research and the importance of addressing individual brain architecture variations, new algorithms are necessary to cross-register multimodal data sets including MRIs and multiple histological series. Here we present a computational approach for same-subject multimodal MRI-guided reconstruction of a series of consecutive histological sections, jointly with diffeomorphic mapping to a reference atlas. We quantify the scale change during different stages of brain histological processing using the Jacobian determinant of the diffeomorphic transformations involved. By mapping the final image stacks to the ex-vivo post-fixation MRI, we show that (a) tape-transfer assisted histological sections can be reassembled accurately into 3D volumes with a local scale change of 2.0 ± 0.4% per axis dimension; in contrast, (b) tissue perfusion/fixation as assessed by mapping the in-vivo MRIs to the ex-vivo post fixation MRIs shows a larger median absolute scale change of 6.9 ± 2.1% per axis dimension. This is the first systematic quantification of local metric distortions associated with whole-brain histological processing, and we expect that the results will generalize to other species. These local scale changes will be important for computing local properties to create reference brain maps.
Collapse
Affiliation(s)
- Brian C Lee
- Center for Imaging Science, Johns Hopkins University, Baltimore, Maryland, USA.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| | - Meng K Lin
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, USA
| | - Yan Fu
- Shanghai Jiaotong University, Shanghai, China
| | | | - Michael I Miller
- Center for Imaging Science, Johns Hopkins University, Baltimore, Maryland, USA.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| | - Partha P Mitra
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, USA
| |
Collapse
|
4
|
TWARD DANIELJ, MITRA PARTHAP, MILLER MICHAELI. ESTIMATING DIFFEOMORPHIC MAPPINGS BETWEEN TEMPLATES AND NOISY DATA: VARIANCE BOUNDS ON THE ESTIMATED CANONICAL VOLUME FORM. QUARTERLY OF APPLIED MATHEMATICS 2018; 77:467-488. [PMID: 31866695 PMCID: PMC6924927 DOI: 10.1090/qam/1527] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Anatomy is undergoing a renaissance driven by the availability of large digital data sets generated by light microscopy. A central computational task is to map individual data volumes to standardized templates. This is accomplished by regularized estimation of a diffeomorphic transformation between the coordinate systems of the individual data and the template, building the transformation incrementally by integrating a smooth flow field. The canonical volume form of this transformation is used to quantify local growth, atrophy, or cell density. While multiple implementations exist for this estimation, less attention has been paid to the variance of the estimated diffeomorphism for noisy data. Notably, there is an infinite dimensional unobservable space defined by those diffeomorphisms which leave the template invariant. These form the stabilizer subgroup of the diffeomorphic group acting on the template. The corresponding flat directions in the energy landscape are expected to lead to increased estimation variance. Here we show that a least-action principle used to generate geodesics in the space of diffeomor-phisms connecting the subject brain to the template removes the stabilizer. This provides reduced-variance estimates of the volume form. Using simulations we demonstrate that the asymmetric large deformation diffeomorphic mapping methods (LDDMM), which explicitly incorporate the asymmetry between idealized template images and noisy empirical images, provide lower variance estimators than their symmetrized counterparts (cf. ANTs). We derive Cramer-Rao bounds for the variances in the limit of small deformations. Analytical results are shown for the Jacobian in terms of perturbations of the vector fields and divergence of the vector field.
Collapse
Affiliation(s)
- DANIEL J. TWARD
- Center for Imaging Science, Johns Hopkins University, Baltimore, Maryland, 21218
| | - PARTHA P. MITRA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York 11724
| | - MICHAEL I. MILLER
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland 21218
| |
Collapse
|
5
|
Miller MI, Arguillère S, Tward DJ, Younes L. Computational anatomy and diffeomorphometry: A dynamical systems model of neuroanatomy in the soft condensed matter continuum. WILEY INTERDISCIPLINARY REVIEWS-SYSTEMS BIOLOGY AND MEDICINE 2018; 10:e1425. [PMID: 29862670 DOI: 10.1002/wsbm.1425] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/29/2017] [Revised: 03/01/2018] [Accepted: 03/09/2018] [Indexed: 11/09/2022]
Abstract
The nonlinear systems models of computational anatomy that have emerged over the past several decades are a synthesis of three significant areas of computational science and biological modeling. First is the algebraic model of biological shape as a Riemannian orbit, a set of objects under diffeomorphic action. Second is the embedding of anatomical shapes into the soft condensed matter physics continuum via the extension of the Euler equations to geodesic, smooth flows with inverses, encoding divergence for the compressibility of atrophy and expansion of growth. Third, is making human shape and form a metrizable space via geodesic connections of coordinate systems. These three themes place our formalism into the modern data science world of personalized medicine supporting inference of high-dimensional anatomical phenotypes for studying neurodegeneration and neurodevelopment. The dynamical systems model of growth and atrophy that emerges is one which is organized in terms of forces, accelerations, velocities, and displacements, with the associated Hamiltonian momentum and the diffeomorphic flow acting as the state, and the smooth vector field the control. The forces that enter the model derive from external measurements through which the dynamical system must flow, and the internal potential energies of structures making up the soft condensed matter. We examine numerous examples on growth and atrophy. This article is categorized under: Analytical and Computational Methods > Computational Methods Laboratory Methods and Technologies > Imaging Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models.
Collapse
Affiliation(s)
- Michael I Miller
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland
| | - Sylvain Arguillère
- Centre National de la Recherche Scientifique, CNRS and Institut Camille Jordan, Université Lyon, Lyon, France
| | - Daniel J Tward
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland
| | - Laurent Younes
- Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, Maryland
| |
Collapse
|
6
|
Tward DJ, Miller MI. On the Complexity of Human Neuroanatomy at the Millimeter Morphome Scale: Developing Codes and Characterizing Entropy Indexed to Spatial Scale. Front Neurosci 2017; 11:577. [PMID: 29093658 PMCID: PMC5651257 DOI: 10.3389/fnins.2017.00577] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2017] [Accepted: 10/02/2017] [Indexed: 11/13/2022] Open
Abstract
In this work we devise a strategy for discrete coding of anatomical form as described by a Bayesian prior model, quantifying the entropy of this representation as a function of code rate (number of bits), and its relationship geometric accuracy at clinically relevant scales. We study the shape of subcortical gray matter structures in the human brain through diffeomorphic transformations that relate them to a template, using data from the Alzheimer's Disease Neuroimaging Initiative to train a multivariate Gaussian prior model. We find that the at 1 mm accuracy all subcortical structures can be described with less than 35 bits, and at 1.5 mm error all structures can be described with less than 12 bits. This work represents a first step towards quantifying the amount of information ordering a neuroimaging study can provide about disease status.
Collapse
|
7
|
Raftopoulos KA, Kollias SD, Sourlas DD, Ferecatu M. On the Beneficial Effect of Noise in Vertex Localization. Int J Comput Vis 2017. [DOI: 10.1007/s11263-017-1034-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
8
|
Tward D, Miller M, Trouve A, Younes L. Parametric Surface Diffeomorphometry for Low Dimensional Embeddings of Dense Segmentations and Imagery. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2017; 39:1195-1208. [PMID: 27295651 PMCID: PMC5663205 DOI: 10.1109/tpami.2016.2578317] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
In the field of Computational Anatomy, biological form (including our focus, neuroanatomy) is studied quantitatively through the action of the diffeomorphism group on example anatomies - a technique called diffeomorphometry. Here we design an algorithm within this framework to pass from dense objects common in neuromaging studies (binary segmentations, structural images) to a sparse representation defined on the surface boundaries of anatomical structures, and embedded into the low dimensional coordinates of a parametric model. Our main new contribution is to introduce an expanded group action to simultaneously deform surfaces through direct mapping of points, as well as images through functional composition with the inverse. This allows us to index the diffeomorphisms with respect to two-dimensional surface geometries like subcortical gray matter structures, but explicitly map onto cost functions determined by noisy 3-dimensional measurements. We consider models generated from empirical covariance of training data, as well as bandlimited (Laplace-Beltrami eigenfunction) models when no such data is available. We show applications to noisy or anomalous segmentations, and other typical problems in neuroimaging studies. We reproduce statistical results detecting changes in Alzheimer's disease, despite dimensionality reduction. Lastly we apply our algorithm to the common problem of segmenting subcortical structures from T1 MR images.
Collapse
|
9
|
Tang X, Qin Y, Zhu W, Miller MI. Surface-based vertexwise analysis of morphometry and microstructural integrity for white matter tracts in diffusion tensor imaging: With application to the corpus callosum in Alzheimer's disease. Hum Brain Mapp 2017; 38:1875-1893. [PMID: 28083895 DOI: 10.1002/hbm.23491] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2016] [Revised: 11/14/2016] [Accepted: 11/30/2016] [Indexed: 11/08/2022] Open
Abstract
In this article, we present a unified statistical pipeline for analyzing the white matter (WM) tracts morphometry and microstructural integrity, both globally and locally within the same WM tract, from diffusion tensor imaging. Morphometry is quantified globally by the volumetric measurement and locally by the vertexwise surface areas. Meanwhile, microstructural integrity is quantified globally by the mean fractional anisotropy (FA) and trace values within the specific WM tract and locally by the FA and trace values defined at each vertex of its bounding surface. The proposed pipeline consists of four steps: (1) fully automated segmentation of WM tracts in a multi-contrast multi-atlas framework; (2) generation of the smooth surface representations for the WM tracts of interest; (3) common template surface generation on which the localized morphometric and microstructural statistics are defined and a variety of statistical analyses can be conducted; (4) multiple comparison correction to determine the significance of the statistical analysis results. Detailed herein, this pipeline has been applied to the corpus callosum in Alzheimer's disease (AD) with significantly decreased FA values and increased trace values, both globally and locally, being detected in patients with AD when compared to normal aging populations. A subdivision of the corpus callosum in both hemispheres revealed that the AD pathology primarily affects the body and splenium of the corpus callosum. Validation analyses and two multiple comparison correction strategies are provided. Hum Brain Mapp 38:1875-1893, 2017. © 2017 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Xiaoying Tang
- Sun Yat-sen University-Carnegie Mellon University (SYSU-CMU) Joint Institute of Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China.,Sun Yat-sen University-Carnegie Mellon University (SYSU-CMU) Shunde International Joint Research Institute, Shunde, Guangdong, China.,School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Yuanyuan Qin
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Wenzhen Zhu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Michael I Miller
- Center for Imaging Science, Johns Hopkins University, Baltimore, Maryland.,Institute for Computational Medicine, Johns Hopkins University, Baltimore, Maryland.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland
| |
Collapse
|
10
|
Unbiased Diffeomorphic Mapping of Longitudinal Data with Simultaneous Subject Specific Template Estimation. GRAPHS IN BIOMEDICAL IMAGE ANALYSIS, COMPUTATIONAL ANATOMY AND IMAGING GENETICS 2017. [DOI: 10.1007/978-3-319-67675-3_12] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
|
11
|
Marino M, Liu Q, Brem S, Wenderoth N, Mantini D. Automated detection and labeling of high-density EEG electrodes from structural MR images. J Neural Eng 2016; 13:056003. [DOI: 10.1088/1741-2560/13/5/056003] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
12
|
Miller MI, Trouvé A, Younes L. Hamiltonian Systems and Optimal Control in Computational Anatomy: 100 Years Since D'Arcy Thompson. Annu Rev Biomed Eng 2015; 17:447-509. [PMID: 26643025 DOI: 10.1146/annurev-bioeng-071114-040601] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The Computational Anatomy project is the morphome-scale study of shape and form, which we model as an orbit under diffeomorphic group action. Metric comparison calculates the geodesic length of the diffeomorphic flow connecting one form to another. Geodesic connection provides a positioning system for coordinatizing the forms and positioning their associated functional information. This article reviews progress since the Euler-Lagrange characterization of the geodesics a decade ago. Geodesic positioning is posed as a series of problems in Hamiltonian control, which emphasize the key reduction from the Eulerian momentum with dimension of the flow of the group, to the parametric coordinates appropriate to the dimension of the submanifolds being positioned. The Hamiltonian viewpoint provides important extensions of the core setting to new, object-informed positioning systems. Several submanifold mapping problems are discussed as they apply to metamorphosis, multiple shape spaces, and longitudinal time series studies of growth and atrophy via shape splines.
Collapse
Affiliation(s)
- Michael I Miller
- Center of Imaging Science.,Department of Biomedical Engineering.,Kavli Neuroscience Discovery Institute, and
| | - Alain Trouvé
- CMLA, ENS Cachan, CNRS, Université Paris-Saclay, 94235 Cachan, France;
| | - Laurent Younes
- Center of Imaging Science.,Department of Applied Mathematics, The John Hopkins University, Baltimore, Maryland 21218; ,
| |
Collapse
|
13
|
Tang X, Holland D, Dale AM, Younes L, Miller MI. Baseline shape diffeomorphometry patterns of subcortical and ventricular structures in predicting conversion of mild cognitive impairment to Alzheimer's disease. J Alzheimers Dis 2015; 44:599-611. [PMID: 25318546 DOI: 10.3233/jad-141605] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
In this paper, we propose a novel predictor for the conversion from mild cognitive impairment (MCI) to Alzheimer's disease (AD). This predictor is based on the shape diffeomorphometry patterns of subcortical and ventricular structures (left and right amygdala, hippocampus, thalamus, caudate, putamen, globus pallidus, and lateral ventricle) of 607 baseline scans from the Alzheimer's Disease Neuroimaging Initiative database, including a total of 210 healthy control subjects, 222 MCI subjects, and 175 AD subjects. The optimal predictor is obtained via a feature selection procedure applied to all of the 14 sets of shape features via linear discriminant analysis, resulting in a combination of the shape diffeomorphometry patterns of the left hippocampus, the left lateral ventricle, the right thalamus, the right caudate, and the bilateral putamen. Via 10-fold cross-validation, we substantiate our method by successfully differentiating 77.04% (104/135) of the MCI subjects who converted to AD within 36 months and 71.26% (62/87) of the non-converters. To be specific, for the MCI-converters, we are capable of correctly predicting 82.35% (14/17) of subjects converting in 6 months, 77.5% (31/40) of subjects converting in 12 months, 74.07% (20/27) of subjects converting in 18 months, 78.13% (25/32) of subjects converting in 24 months, and 73.68% (14/19) of subject converting in 36 months. Statistically significant correlation maps were observed between the shape diffeomorphometry features of each of the 14 structures, especially the bilateral amygdala, hippocampus, lateral ventricle, and two neuropsychological test scores--the Alzheimer's Disease Assessment Scale-Cognitive Behavior Section and the Mini-Mental State Examination.
Collapse
Affiliation(s)
- Xiaoying Tang
- Center for Imaging Science, Johns Hopkins University, Baltimore, MD, USA
| | - Dominic Holland
- Department of Neurosciences, University of California, San Diego, La Jolla, CA, USA
| | - Anders M Dale
- Department of Neurosciences, University of California, San Diego, La Jolla, CA, USA Department of Radiology, University of California, San Diego, La Jolla, CA, USA
| | - Laurent Younes
- Center for Imaging Science, Johns Hopkins University, Baltimore, MD, USA Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, USA Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
| | - Michael I Miller
- Center for Imaging Science, Johns Hopkins University, Baltimore, MD, USA Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, USA Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
14
|
Miller MI, Younes L, Trouvé A. Diffeomorphometry and geodesic positioning systems for human anatomy. TECHNOLOGY 2014; 2:36. [PMID: 24904924 PMCID: PMC4041578 DOI: 10.1142/s2339547814500010] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
The Computational Anatomy project has largely been a study of large deformations within a Riemannian framework as an efficient point of view for generating metrics between anatomical configurations. This approach turns D'Arcy Thompson's comparative morphology of human biological shape and form into a metrizable space. Since the metric is constructed based on the geodesic length of the flows of diffeomorphisms connecting the forms, we call it diffeomorphometry. Just as importantly, since the flows describe algebraic group action on anatomical submanifolds and associated functional measurements, they become the basis for positioning information, which we term geodesic positioning. As well the geodesic connections provide Riemannian coordinates for locating forms in the anatomical orbit, which we call geodesic coordinates. These three components taken together - the metric, geodesic positioning of information, and geodesic coordinates - we term the geodesic positioning system. We illustrate via several examples in human and biological coordinate systems and machine learning of the statistical representation of shape and form.
Collapse
|