Learning task-optimal registration cost functions for localizing cytoarchitecture and function in the cerebral cortex

JOSA: Joint surface-based registration and atlas construction of brain geometry and function

Surface-based cortical registration is an important topic in medical image analysis and facilitates many downstream applications. Current approaches for cortical registration are mainly driven by geometric features, such as sulcal depth and curvature, and often assume that registration of folding patterns leads to alignment of brain function. However, functional variability of anatomically corresponding areas across subjects has been widely reported, particularly in higher-order cognitive areas. In this work, we present JOSA, a novel cortical registration framework that jointly models the mismatch between geometry and function while simultaneously learning an unbiased population-specific atlas. Using a semi-supervised training strategy, JOSA achieves superior registration performance in both geometry and function to the state-of-the-art methods but without requiring functional data at inference. This learning framework can be extended to any auxiliary data to guide spherical registration that is available during training but is difficult or impossible to obtain during inference, such as parcellations, architectonic identity, transcriptomic information, and molecular profiles. By recognizing the mismatch between geometry and function, JOSA provides new insights into the future development of registration methods using joint analysis of brain structure and function.

Joint cortical registration of geometry and function using semi-supervised learning

Brain surface-based image registration, an important component of brain image analysis, establishes spatial correspondence between cortical surfaces. Existing iterative and learning-based approaches focus on accurate registration of folding patterns of the cerebral cortex, and assume that geometry predicts function and thus functional areas will also be well aligned. However, structure/functional variability of anatomically corresponding areas across subjects has been widely reported. In this work, we introduce a learning-based cortical registration framework, JOSA, which jointly aligns folding patterns and functional maps while simultaneously learning an optimal atlas. We demonstrate that JOSA can substantially improve registration performance in both anatomical and functional domains over existing methods. By employing a semi-supervised training strategy, the proposed framework obviates the need for functional data during inference, enabling its use in broad neuroscientific domains where functional data may not be observed. The source code of JOSA will be released to the public at https://voxelmorph.net.

R2Net: Efficient and flexible diffeomorphic image registration using Lipschitz continuous residual networks

Classical diffeomorphic image registration methods, while being accurate, face the challenges of high computational costs. Deep learning based approaches provide a fast alternative to address these issues; however, most existing deep solutions either lose the good property of diffeomorphism or have limited flexibility to capture large deformations, under the assumption that deformations are driven by stationary velocity fields (SVFs). Also, the adopted squaring and scaling technique for integrating SVFs is time- and memory-consuming, hindering deep methods from handling large image volumes. In this paper, we present an unsupervised diffeomorphic image registration framework, which uses deep residual networks (ResNets) as numerical approximations of the underlying continuous diffeomorphic setting governed by ordinary differential equations, which is parameterized by either SVFs or time-varying (non-stationary) velocity fields. This flexible parameterization in our Residual Registration Network (R2Net) not only provides the model's ability to capture large deformation but also reduces the time and memory cost when integrating velocity fields for deformation generation. Also, we introduce a Lipschitz continuity constraint into the ResNet block to help achieve diffeomorphic deformations. To enhance the ability of our model for handling images with large volume sizes, we employ a hierarchical extension with a multi-phase learning strategy to solve the image registration task in a coarse-to-fine fashion. We demonstrate our models on four 3D image registration tasks with a wide range of anatomies, including brain MRIs, cine cardiac MRIs, and lung CT scans. Compared to classical methods SyN and diffeomorphic VoxelMorph, our models achieve comparable or better registration accuracy with much smoother deformations. Our source code is available online at https://github.com/ankitajoshi15/R2Net.

A neurostructural biomarker of dissociative amnesia: a hippocampal study in dissociative identity disorder

BACKGROUND

Little is known about the neural correlates of dissociative amnesia, a transdiagnostic symptom mostly present in the dissociative disorders and core characteristic of dissociative identity disorder (DID). Given the vital role of the hippocampus in memory, a prime candidate for investigation is whether total and/or subfield hippocampal volume can serve as biological markers of dissociative amnesia.

METHODS

A total of 75 women, 32 with DID and 43 matched healthy controls (HC), underwent structural magnetic resonance imaging (MRI). Using Freesurfer (version 6.0), volumes were extracted for bilateral global hippocampus, cornu ammonis (CA) 1-4, the granule cell molecular layer of the dentate gyrus (GC-ML-DG), fimbria, hippocampal-amygdaloid transition area (HATA), parasubiculum, presubiculum and subiculum. Analyses of covariance showed volumetric differences between DID and HC. Partial correlations exhibited relationships between the three factors of the dissociative experience scale scores (dissociative amnesia, absorption, depersonalisation/derealisation) and traumatisation measures with hippocampal global and subfield volumes.

RESULTS

Hippocampal volumes were found to be smaller in DID as compared with HC in bilateral global hippocampus and bilateral CA1, right CA4, right GC-ML-DG, and left presubiculum. Dissociative amnesia was the only dissociative symptom that correlated uniquely and significantly with reduced bilateral hippocampal CA1 subfield volumes. Regarding traumatisation, only emotional neglect correlated negatively with bilateral global hippocampus, bilateral CA1, CA4 and GC-ML-DG, and right CA3.

CONCLUSION

We propose decreased CA1 volume as a biomarker for dissociative amnesia. We also propose that traumatisation, specifically emotional neglect, is interlinked with dissociative amnesia in having a detrimental effect on hippocampal volume.

An adaptive registration algorithm for zebrafish larval brain images

BACKGROUND AND OBJECTIVE

Zebrafish (Danio rerio) in their larval stages have grown increasingly popular as excellent vertebrate models for neurobiological research. Researchers can apply various tools in order to decode the neural structure patterns which can aid the understanding of vertebrate brain development. In order to do so, it is essential to map the gene expression patterns to an anatomical reference precisely. However, high accuracy in sample registration is sometimes difficult to achieve due to laboratory- or protocol-dependent variabilities.

METHODS

In this paper, we propose an accurate adaptive registration algorithm for volumetric zebrafish larval image datasets using a synergistic combination of attractive Free-Form-Deformation (FFD) and diffusive Demons algorithms. A coarse registration is achieved first for 3D volumetric data using a 3D affine transformation. A localized registration algorithm in form of a B-splines based FFD is applied next on the coarsely registered volume. Finally, the Demons algorithm is applied on this FFD registered volume for achieving fine registration by making the solution noise resilient.

RESULTS

Results Experimental procedures are carried out on a number of 72 hpf (hours post fertilization) 3D confocal zebrafish larval datasets. Comparisons with state-of-the-art methods including some ablation studies clearly demonstrate the effectiveness of the proposed method.

CONCLUSIONS

Our adaptive registration algorithm significantly aids Zebrafish imaging analysis over current methods for gene expression anatomical mapping, such as Vibe-Z. We believe the proposed solution would be able to overcome the requirement of high quality images which currently limits the applicability of Zebrafish in neuroimaging research.

Individual-Specific Areal-Level Parcellations Improve Functional Connectivity Prediction of Behavior

Resting-state functional magnetic resonance imaging (rs-fMRI) allows estimation of individual-specific cortical parcellations. We have previously developed a multi-session hierarchical Bayesian model (MS-HBM) for estimating high-quality individual-specific network-level parcellations. Here, we extend the model to estimate individual-specific areal-level parcellations. While network-level parcellations comprise spatially distributed networks spanning the cortex, the consensus is that areal-level parcels should be spatially localized, that is, should not span multiple lobes. There is disagreement about whether areal-level parcels should be strictly contiguous or comprise multiple noncontiguous components; therefore, we considered three areal-level MS-HBM variants spanning these range of possibilities. Individual-specific MS-HBM parcellations estimated using 10 min of data generalized better than other approaches using 150 min of data to out-of-sample rs-fMRI and task-fMRI from the same individuals. Resting-state functional connectivity derived from MS-HBM parcellations also achieved the best behavioral prediction performance. Among the three MS-HBM variants, the strictly contiguous MS-HBM exhibited the best resting-state homogeneity and most uniform within-parcel task activation. In terms of behavioral prediction, the gradient-infused MS-HBM was numerically the best, but differences among MS-HBM variants were not statistically significant. Overall, these results suggest that areal-level MS-HBMs can capture behaviorally meaningful individual-specific parcellation features beyond group-level parcellations. Multi-resolution trained models and parcellations are publicly available (https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/brain_parcellation/Kong2022_ArealMSHBM).

Evolutionary Machine Learning for Multi-Objective Class Solutions in Medical Deformable Image Registration

Current state-of-the-art medical deformable image registration (DIR) methods optimize a weighted sum of key objectives of interest. Having a pre-determined weight combination that leads to high-quality results for any instance of a specific DIR problem (i.e., a class solution) would facilitate clinical application of DIR. However, such a combination can vary widely for each instance and is currently often manually determined. A multi-objective optimization approach for DIR removes the need for manual tuning, providing a set of high-quality trade-off solutions. Here, we investigate machine learning for a multi-objective class solution, i.e., not a single weight combination, but a set thereof, that, when used on any instance of a specific DIR problem, approximates such a set of trade-off solutions. To this end, we employed a multi-objective evolutionary algorithm to learn sets of weight combinations for three breast DIR problems of increasing difficulty: 10 prone-prone cases, 4 prone-supine cases with limited deformations and 6 prone-supine cases with larger deformations and image artefacts. Clinically-acceptable results were obtained for the first two problems. Therefore, for DIR problems with limited deformations, a multi-objective class solution can be machine learned and used to compute straightforwardly multiple high-quality DIR outcomes, potentially leading to more efficient use of DIR in clinical practice.

VoxelMorph: A Learning Framework for Deformable Medical Image Registration

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph.

Accurate nonlinear mapping between MNI volumetric and FreeSurfer surface coordinate systems

The results of most neuroimaging studies are reported in volumetric (e.g., MNI152) or surface (e.g., fsaverage) coordinate systems. Accurate mappings between volumetric and surface coordinate systems can facilitate many applications, such as projecting fMRI group analyses from MNI152/Colin27 to fsaverage for visualization or projecting resting‐state fMRI parcellations from fsaverage to MNI152/Colin27 for volumetric analysis of new data. However, there has been surprisingly little research on this topic. Here, we evaluated three approaches for mapping data between MNI152/Colin27 and fsaverage coordinate systems by simulating the above applications: projection of group‐average data from MNI152/Colin27 to fsaverage and projection of fsaverage parcellations to MNI152/Colin27. Two of the approaches are currently widely used. A third approach (registration fusion) was previously proposed, but not widely adopted. Two implementations of the registration fusion (RF) approach were considered, with one implementation utilizing the Advanced Normalization Tools (ANTs). We found that RF‐ANTs performed the best for mapping between fsaverage and MNI152/Colin27, even for new subjects registered to MNI152/Colin27 using a different software tool (FSL FNIRT). This suggests that RF‐ANTs would be useful even for researchers not using ANTs. Finally, it is worth emphasizing that the most optimal approach for mapping data to a coordinate system (e.g., fsaverage) is to register individual subjects directly to the coordinate system, rather than via another coordinate system. Only in scenarios where the optimal approach is not possible (e.g., mapping previously published results from MNI152 to fsaverage), should the approaches evaluated in this manuscript be considered. In these scenarios, we recommend RF‐ANTs (https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/registration/Wu2017_RegistrationFusion).

Analysis strategies for high-resolution UHF-fMRI data

Functional MRI (fMRI) benefits from both increased sensitivity and specificity with increasing magnetic field strength, making it a key application for Ultra-High Field (UHF) MRI scanners. Most UHF-fMRI studies utilize the dramatic increases in sensitivity and specificity to acquire high-resolution data reaching sub-millimeter scales, which enable new classes of experiments to probe the functional organization of the human brain. This review article surveys advanced data analysis strategies developed for high-resolution fMRI at UHF. These include strategies designed to mitigate distortion and artifacts associated with higher fields in ways that attempt to preserve spatial resolution of the fMRI data, as well as recently introduced analysis techniques that are enabled by these extremely high-resolution data. Particular focus is placed on anatomically-informed analyses, including cortical surface-based analysis, which are powerful techniques that can guide each step of the analysis from preprocessing to statistical analysis to interpretation and visualization. New intracortical analysis techniques for laminar and columnar fMRI are also reviewed and discussed. Prospects for single-subject individualized analyses are also presented and discussed. Altogether, there are both specific challenges and opportunities presented by UHF-fMRI, and the use of proper analysis strategies can help these valuable data reach their full potential.

The feasibility of manual parameter tuning for deformable breast MR image registration from a multi-objective optimization perspective

Deformable image registration is typically formulated as an optimization problem involving a linearly weighted combination of terms that correspond to objectives of interest (e.g. similarity, deformation magnitude). The weights, along with multiple other parameters, need to be manually tuned for each application, a task currently addressed mainly via trial-and-error approaches. Such approaches can only be successful if there is a sensible interplay between parameters, objectives, and desired registration outcome. This, however, is not well established. To study this interplay, we use multi-objective optimization, where multiple solutions exist that represent the optimal trade-offs between the objectives, forming a so-called Pareto front. Here, we focus on weight tuning. To study the space a user has to navigate during manual weight tuning, we randomly sample multiple linear combinations. To understand how these combinations relate to desirability of registration outcome, we associate with each outcome a mean target registration error (TRE) based on expert-defined anatomical landmarks. Further, we employ a multi-objective evolutionary algorithm that optimizes the weight combinations, yielding a Pareto front of solutions, which can be directly navigated by the user. To study how the complexity of manual weight tuning changes depending on the registration problem, we consider an easy problem, prone-to-prone breast MR image registration, and a hard problem, prone-to-supine breast MR image registration. Lastly, we investigate how guidance information as an additional objective influences the prone-to-supine registration outcome. Results show that the interplay between weights, objectives, and registration outcome makes manual weight tuning feasible for the prone-to-prone problem, but very challenging for the harder prone-to-supine problem. Here, patient-specific, multi-objective weight optimization is needed, obtaining a mean TRE of 13.6 mm without guidance information reduced to 7.3 mm with guidance information, but also providing a Pareto front that exhibits an intuitively sensible interplay between weights, objectives, and registration outcome, allowing outcome selection.

Contextual Atlas Regression Forests: Multiple-Atlas-Based Automated Dose Prediction in Radiation Therapy

Radiation therapy is an integral part of cancer treatment, but to date it remains highly manual. Plans are created through optimization of dose volume objectives that specify intent to minimize, maximize, or achieve a prescribed dose level to clinical targets and organs. Optimization is NP-hard, requiring highly iterative and manual initialization procedures. We present a proof-of-concept for a method to automatically infer the radiation dose directly from the patient's treatment planning image based on a database of previous patients with corresponding clinical treatment plans. Our method uses regression forests augmented with density estimation over the most informative features to learn an automatic atlas-selection metric that is tailored to dose prediction. We validate our approach on 276 patients from 3 clinical treatment plan sites (whole breast, breast cavity, and prostate), with an overall dose prediction accuracies of 78.68%, 64.76%, 86.83% under the Gamma metric.

Multi-atlas segmentation of biomedical images: A survey

Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, et al. (2004), Klein, et al. (2005), and Heckemann, et al. (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture anatomical variation, thus offering superior segmentation accuracy. This benefit, however, typically comes at a high computational cost. Recent advancements in computer hardware and image processing software have been instrumental in addressing this challenge and facilitated the wide adoption of MAS. Today, MAS has come a long way and the approach includes a wide array of sophisticated algorithms that employ ideas from machine learning, probabilistic modeling, optimization, and computer vision, among other fields. This paper presents a survey of published MAS algorithms and studies that have applied these methods to various biomedical problems. In writing this survey, we have three distinct aims. Our primary goal is to document how MAS was originally conceived, later evolved, and now relates to alternative methods. Second, this paper is intended to be a detailed reference of past research activity in MAS, which now spans over a decade (2003-2014) and entails novel methodological developments and application-specific solutions. Finally, our goal is to also present a perspective on the future of MAS, which, we believe, will be one of the dominant approaches in biomedical image segmentation.

Reducing Individual Variation for fMRI Studies in Children by Minimizing Template Related Errors

Spatial normalization is an essential process for group comparisons in functional MRI studies. In practice, there is a risk of normalization errors particularly in studies involving children, seniors or diseased populations and in regions with high individual variation. One way to minimize normalization errors is to create a study-specific template based on a large sample size. However, studies with a large sample size are not always feasible, particularly for children studies. The performance of templates with a small sample size has not been evaluated in fMRI studies in children. In the current study, this issue was encountered in a working memory task with 29 children in two groups. We compared the performance of different templates: a study-specific template created by the experimental population, a Chinese children template and the widely used adult MNI template. We observed distinct differences in the right orbitofrontal region among the three templates in between-group comparisons. The study-specific template and the Chinese children template were more sensitive for the detection of between-group differences in the orbitofrontal cortex than the MNI template. Proper templates could effectively reduce individual variation. Further analysis revealed a correlation between the BOLD contrast size and the norm index of the affine transformation matrix, i.e., the SFN, which characterizes the difference between a template and a native image and differs significantly across subjects. Thereby, we proposed and tested another method to reduce individual variation that included the SFN as a covariate in group-wise statistics. This correction exhibits outstanding performance in enhancing detection power in group-level tests. A training effect of abacus-based mental calculation was also demonstrated, with significantly elevated activation in the right orbitofrontal region that correlated with behavioral response time across subjects in the trained group.

Decoupling function and anatomy in atlases of functional connectivity patterns: language mapping in tumor patients

In this paper we construct an atlas that summarizes functional connectivity characteristics of a cognitive process from a population of individuals. The atlas encodes functional connectivity structure in a low-dimensional embedding space that is derived from a diffusion process on a graph that represents correlations of fMRI time courses. The functional atlas is decoupled from the anatomical space, and thus can represent functional networks with variable spatial distribution in a population. In practice the atlas is represented by a common prior distribution for the embedded fMRI signals of all subjects. We derive an algorithm for fitting this generative model to the observed data in a population. Our results in a language fMRI study demonstrate that the method identifies coherent and functionally equivalent regions across subjects. The method also successfully maps functional networks from a healthy population used as a training set to individuals whose language networks are affected by tumors.

Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights

Evaluating various algorithms for the inter-subject registration of brain magnetic resonance images (MRI) is a necessary topic receiving growing attention. Existing studies evaluated image registration algorithms in specific tasks or using specific databases (e.g., only for skull-stripped images, only for single-site images, etc.). Consequently, the choice of registration algorithms seems task- and usage/parameter-dependent. Nevertheless, recent large-scale, often multi-institutional imaging-related studies create the need and raise the question whether some registration algorithms can 1) generally apply to various tasks/databases posing various challenges; 2) perform consistently well, and while doing so, 3) require minimal or ideally no parameter tuning. In seeking answers to this question, we evaluated 12 general-purpose registration algorithms, for their generality, accuracy and robustness. We fixed their parameters at values suggested by algorithm developers as reported in the literature. We tested them in 7 databases/tasks, which present one or more of 4 commonly-encountered challenges: 1) inter-subject anatomical variability in skull-stripped images; 2) intensity homogeneity, noise and large structural differences in raw images; 3) imaging protocol and field-of-view (FOV) differences in multi-site data; and 4) missing correspondences in pathology-bearing images. Totally 7,562 registrations were performed. Registration accuracies were measured by (multi-)expert-annotated landmarks or regions of interest (ROIs). To ensure reproducibility, we used public software tools, public databases (whenever possible), and we fully disclose the parameter settings. We show evaluation results, and discuss the performances in light of algorithms' similarity metrics, transformation models and optimization strategies. We also discuss future directions for the algorithm development and evaluations.

Correspondences between retinotopic areas and myelin maps in human visual cortex

We generated probabilistic area maps and maximum probability maps (MPMs) for a set of 18 retinotopic areas previously mapped in individual subjects (Georgieva et al., 2009 and Kolster et al., 2010) using four different inter-subject registration methods. The best results were obtained using a recently developed multimodal surface matching method. The best set of MPMs had relatively smooth borders between visual areas and group average area sizes that matched the typical size in individual subjects. Comparisons between retinotopic areas and maps of estimated cortical myelin content revealed the following correspondences: (i) areas V1, V2, and V3 are heavily myelinated; (ii) the MT cluster is heavily myelinated, with a peak near the MT/pMSTv border; (iii) a dorsal myelin density peak corresponds to area V3D; (iv) the phPIT cluster is lightly myelinated; and (v) myelin density differs across the four areas of the V3A complex. Comparison of the retinotopic MPM with cytoarchitectonic areas, including those previously mapped to the fs_LR cortical surface atlas, revealed a correspondence between areas V1–3 and hOc1–3, respectively, but little correspondence beyond V3. These results indicate that architectonic and retinotopic areal boundaries are in agreement in some regions, and that retinotopy provides a finer-grained parcellation in other regions. The atlas datasets from this analysis are freely available as a resource for other studies that will benefit from retinotopic and myelin density map landmarks in human visual cortex.

•

Maximum probability maps for 18 retinotopic areas were generated.

•

Multimodal surface matching was used to compare with myelin and cytoarchitectonic maps.

•

Early areas V1–3 areas are heavily myelinated, as are V3D and most of areas MT/pMSTv.

•

The phPIT cluster is lightly myelinated compared to other retinotopic areas.

•

Early areas V1–3 correspond to areas hOc1–3, with little correspondence beyond V3.

The evolution of a disparity decision in human visual cortex

We used fMRI-informed EEG source-imaging in humans to characterize the dynamics of cortical responses during a disparity-discrimination task. After the onset of a disparity-defined target, decision-related activity was found within an extended cortical network that included several occipital regions of interest (ROIs): V4, V3A, hMT+ and the Lateral Occipital Complex (LOC). By using a response-locked analysis, we were able to determine the timing relationships in this network of ROIs relative to the subject's behavioral response. Choice-related activity appeared first in the V4 ROI almost 200 ms before the button press and then subsequently in the V3A ROI. Modeling of the responses in the V4 ROI suggests that this area provides an early contribution to disparity discrimination. Choice-related responses were also found after the button-press in ROIs V4, V3A, LOC and hMT+. Outside the visual cortex, choice-related activity was found in the frontal and temporal poles before the button-press. By combining the spatial resolution of fMRI-informed EEG source imaging with the ability to sort out neural activity occurring before, during and after the behavioral manifestation of the decision, our study is the first to assign distinct functional roles to the extra-striate ROIs involved in perceptual decisions based on disparity, the primary cue for depth.

Dynamics and cortical distribution of neural responses to 2D and 3D motion in human

The perception of motion-in-depth is important for avoiding collisions and for the control of vergence eye-movements and other motor actions. Previous psychophysical studies have suggested that sensitivity to motion-in-depth has a lower temporal processing limit than the perception of lateral motion. The present study used functional MRI-informed EEG source-imaging to study the spatiotemporal properties of the responses to lateral motion and motion-in-depth in human visual cortex. Lateral motion and motion-in-depth displays comprised stimuli whose only difference was interocular phase: monocular oscillatory motion was either in-phase in the two eyes (lateral motion) or in antiphase (motion-in-depth). Spectral analysis was used to break the steady-state visually evoked potentials responses down into even and odd harmonic components within five functionally defined regions of interest: V1, V4, lateral occipital complex, V3A, and hMT+. We also characterized the responses within two anatomically defined regions: the inferior and superior parietal cortex. Even harmonic components dominated the evoked responses and were a factor of approximately two larger for lateral motion than motion-in-depth. These responses were slower for motion-in-depth and were largely independent of absolute disparity. In each of our regions of interest, responses at odd-harmonics were relatively small, but were larger for motion-in-depth than lateral motion, especially in parietal cortex, and depended on absolute disparity. Taken together, our results suggest a plausible neural basis for reduced psychophysical sensitivity to rapid motion-in-depth.

FOCUSR: feature oriented correspondence using spectral regularization--a method for precise surface matching

Existing methods for surface matching are limited by the tradeoff between precision and computational efficiency. Here, we present an improved algorithm for dense vertex-to-vertex correspondence that uses direct matching of features defined on a surface and improves it by using spectral correspondence as a regularization. This algorithm has the speed of both feature matching and spectral matching while exhibiting greatly improved precision (distance errors of 1.4 percent). The method, FOCUSR, incorporates implicitly such additional features to calculate the correspondence and relies on the smoothness of the lowest-frequency harmonics of a graph Laplacian to spatially regularize the features. In its simplest form, FOCUSR is an improved spectral correspondence method that nonrigidly deforms spectral embeddings. We provide here a full realization of spectral correspondence where virtually any feature can be used as an additional information using weights on graph edges, but also on graph nodes and as extra embedded coordinates. As an example, the full power of FOCUSR is demonstrated in a real-case scenario with the challenging task of brain surface matching across several individuals. Our results show that combining features and regularizing them in a spectral embedding greatly improves the matching precision (to a submillimeter level) while performing at much greater speed than existing methods.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Model-driven harmonic parameterization of the cortical surface: HIP-HOP

In the context of inter subject brain surface matching, we present a parameterization of the cortical surface constrained by a model of cortical organization. The parameterization is defined via an harmonic mapping of each hemisphere surface to a rectangular planar domain that integrates a representation of the model. As opposed to previous landmark-based registration methods we do not match folds between individuals but instead optimize the fit between cortical sulci and specific iso-coordinate axis in the model. This strategy overcomes some limitation to sulcus-based registration techniques such as topological variability in sulcal landmarks across subjects. Experiments on 62 subjects with manually traced sulci are presented and compared with the result of the Freesurfer software. The evaluation involves a measure of dispersion of sulci with both angular and area distortions. We show that the model-based strategy can lead to a natural, efficient and very fast (less than 5 min per hemisphere) method for defining inter subjects correspondences. We discuss how this approach also reduces the problems inherent to anatomically defined landmarks and open the way to the investigation of cortical organization through the notion of orientation and alignment of structures across the cortex.

Individual variability in functional connectivity architecture of the human brain

The fact that people think or behave differently from one another is rooted in individual differences in brain anatomy and connectivity. Here, we used repeated-measurement resting-state functional MRI to explore intersubject variability in connectivity. Individual differences in functional connectivity were heterogeneous across the cortex, with significantly higher variability in heteromodal association cortex and lower variability in unimodal cortices. Intersubject variability in connectivity was significantly correlated with the degree of evolutionary cortical expansion, suggesting a potential evolutionary root of functional variability. The connectivity variability was also related to variability in sulcal depth but not cortical thickness, positively correlated with the degree of long-range connectivity but negatively correlated with local connectivity. A meta-analysis further revealed that regions predicting individual differences in cognitive domains are predominantly located in regions of high connectivity variability. Our findings have potential implications for understanding brain evolution and development, guiding intervention, and interpreting statistical maps in neuroimaging.

Bayesian characterization of uncertainty in intra-subject non-rigid registration

In settings where high-level inferences are made based on registered image data, the registration uncertainty can contain important information. In this article, we propose a Bayesian non-rigid registration framework where conventional dissimilarity and regularization energies can be included in the likelihood and the prior distribution on deformations respectively through the use of Boltzmann's distribution. The posterior distribution is characterized using Markov Chain Monte Carlo (MCMC) methods with the effect of the Boltzmann temperature hyper-parameters marginalized under broad uninformative hyper-prior distributions. The MCMC chain permits estimation of the most likely deformation as well as the associated uncertainty. On synthetic examples, we demonstrate the ability of the method to identify the maximum a posteriori estimate and the associated posterior uncertainty, and demonstrate that the posterior distribution can be non-Gaussian. Additionally, results from registering clinical data acquired during neurosurgery for resection of brain tumor are provided; we compare the method to single transformation results from a deterministic optimizer and introduce methods that summarize the high-dimensional uncertainty. At the site of resection, the registration uncertainty increases and the marginal distribution on deformations is shown to be multi-modal.

Selection of optimal hyper-parameters for estimation of uncertainty in MRI-TRUS registration of the prostate

Transrectal ultrasound (TRUS) facilitates intra-treatment delineation of the prostate gland (PG) to guide insertion of brachytherapy seeds, but the prostate substructure and apex are not always visible which may make the seed placement sub-optimal. Based on an elastic model of the prostate created from MRI, where the prostate substructure and apex are clearly visible, we use a Bayesian approach to estimate the posterior distribution on deformations that aligns the pre-treatment MRI with intra-treatment TRUS. Without apex information in TRUS, the posterior prediction of the location of the prostate boundary, and the prostate apex boundary in particular, is mainly determined by the pseudo stiffness hyper-parameter of the prior distribution. We estimate the optimal value of the stiffness through likelihood maximization that is sensitive to the accuracy as well as the precision of the posterior prediction at the apex boundary. From a data-set of 10 pre- and intra-treatment prostate images with ground truth delineation of the total PG, 4 cases were used to establish an optimal stiffness hyper-parameter when 15% of the prostate delineation was removed to simulate lack of apex information in TRUS, while the remaining 6 cases were used to cross-validate the registration accuracy and uncertainty over the PG and in the apex.

Subject-specific functional localizers increase sensitivity and functional resolution of multi-subject analyses

One important goal of cognitive neuroscience is to discover and explain properties common to all human brains. The traditional solution for comparing functional activations across brains in fMRI is to align each individual brain to a template brain in a Cartesian coordinate system (e.g., the Montreal Neurological Institute template). However, inter-individual anatomical variability leads to decreases in sensitivity (ability to detect a significant activation when it is present) and functional resolution (ability to discriminate spatially adjacent but functionally different neural responses) in group analyses. Subject-specific functional localizers have been previously argued to increase the sensitivity and functional resolution of fMRI analyses in the presence of inter-subject variability in the locations of functional activations (e.g., Brett et al., 2002; Fedorenko and Kanwisher, 2009, 2011; Fedorenko et al., 2010; Kanwisher et al., 1997; Saxe et al., 2006). In the current paper we quantify this dependence of sensitivity and functional resolution on functional variability across subjects in order to illustrate the highly detrimental effects of this variability on traditional group analyses. We show that analyses that use subject-specific functional localizers usually outperform traditional group-based methods in both sensitivity and functional resolution, even when the same total amount of data is used for each analysis. We further discuss how the subject-specific functional localization approach, which has traditionally only been considered in the context of ROI-based analyses, can be extended to whole-brain voxel-based analyses. We conclude that subject-specific functional localizers are particularly well suited for investigating questions of functional specialization in the brain. An SPM toolbox that can perform all of the analyses described in this paper is publicly available, and the analyses can be applied retroactively to any dataset, provided that multiple runs were acquired per subject, even if no explicit "localizer" task was included.

The time course of shape discrimination in the human brain

The lateral occipital cortex (LOC) activates selectively to images of intact objects versus scrambled controls, is selective for the figure-ground relationship of a scene, and exhibits at least some degree of invariance for size and position. Because of these attributes, it is considered to be a crucial part of the object recognition pathway. Here we show that human LOC is critically involved in perceptual decisions about object shape. High-density EEG was recorded while subjects performed a threshold-level shape discrimination task on texture-defined figures segmented by either phase or orientation cues. The appearance or disappearance of a figure region from a uniform background generated robust visual evoked potentials throughout retinotopic cortex as determined by inverse modeling of the scalp voltage distribution. Contrasting responses from trials containing shape changes that were correctly detected (hits) with trials in which no change occurred (correct rejects) revealed stimulus-locked, target-selective activity in the occipital visual areas LOC and V4 preceding the subject's response. Activity that was locked to the subjects' reaction time was present in the LOC. Response-locked activity in the LOC was determined to be related to shape discrimination for several reasons: shape-selective responses were silenced when subjects viewed identical stimuli but their attention was directed away from the shapes to a demanding letter discrimination task; shape-selectivity was present across four different stimulus configurations used to define the figure; LOC responses correlated with participants' reaction times. These results indicate that decision-related activity is present in the LOC when subjects are engaged in threshold-level shape discriminations.

Coevolution of brain structures in amnestic mild cognitive impairment

Network accounts of the progression of Alzheimer's disease (AD), based on cross-sectional brain imaging observations, postulate that the biological course of the disease is characterized by coordinated spatial patterns of brain change to distributed cognitive networks. This study tests this conjecture by quantifying inter-regional covariance in cortical gray matter atrophy rates in 317 Alzheimer's Disease Neuroimaging Initiative participants who were clinically diagnosed with amnestic mild cognitive impairment at baseline and underwent serial MRI at 6-month intervals over the course of 2years. A factor analysis model identified five factors (i.e. groupings of regions) that exhibited highly correlated rates of atrophy. Four groupings approximately corresponded to coordinated change within the posterior default mode network, prefrontal cortex, medial temporal lobe, and regions largely spared by the early pathological course of AD (i.e., sensorimotor and occipital cortex), while the fifth grouping represented diffuse, global atrophy. The data-driven observation of "frontal aging" superimposed upon medial temporal atrophy typical of early AD and default mode network changes supports the view that in individuals at high risk of eventual clinical AD, multiple patterns of distributed neuronal death corresponding to multiple biological substrates may be active.

Individual functional ROI optimization via maximization of group-wise consistency of structural and functional profiles

Studying connectivities among functional brain regions and the functional dynamics on brain networks has drawn increasing interest. A fundamental issue that affects functional connectivity and dynamics studies is how to determine the best possible functional brain regions or ROIs (regions of interest) for a group of individuals, since the connectivity measurements are heavily dependent on ROI locations. Essentially, identification of accurate, reliable and consistent corresponding ROIs is challenging due to the unclear boundaries between brain regions, variability across individuals, and nonlinearity of the ROIs. In response to these challenges, this paper presents a novel methodology to computationally optimize ROIs locations derived from task-based fMRI data for individuals so that the optimized ROIs are more consistent, reproducible and predictable across brains. Our computational strategy is to formulate the individual ROI location optimization as a group variance minimization problem, in which group-wise consistencies in functional/structural connectivity patterns and anatomic profiles are defined as optimization constraints. Our experimental results from multimodal fMRI and DTI data show that the optimized ROIs have significantly improved consistency in structural and functional profiles across individuals. These improved functional ROIs with better consistency could contribute to further study of functional interaction and dynamics in the human brain.

LOCALLY-ADAPTIVE SIMILARITY METRIC FOR DEFORMABLE MEDICAL IMAGE REGISTRATION

More and more researchers are beginning to use multiple dissimilarity metrics or image features for medical image registration. In most of these approaches, however, weights for ranking the relative importance between the selected metrics are empirically tuned and fixed for the entire image domain. Different parts of a medical image, however, may contain significantly different appearance properties such that a metric may only be applicable in certain image regions but less so in other regions. In this paper, we propose to adapt this weighting to generate a locally-adaptive set of dissimilarity metrics such that the overall metric set encourages proper spatial alignment. Using contextual information or via a learning procedure, our approach generates a vector weight map that determines, at each spatial location, the relative importance of each constituent of the overall metric. Our approach was evaluated on 2 datasets of 15 computed tomography (CT) lung images and 40 brain magnetic resonance images (MRI). Experiments show that our approach of using a locally-adaptive set of dissimilarity metrics gives superior results when compared against its non-region specific variant.

A general fast registration framework by learning deformation-appearance correlation

In this paper, we propose a general framework for performance improvement of the current state-of-the-art registration algorithms in terms of both accuracy and computation time. The key concept involves rapid prediction of a deformation field for registration initialization, which is achieved by a statistical correlation model learned between image appearances and deformation fields. This allows us to immediately bring a template image as close as possible to a subject image that we need to register. The task of the registration algorithm is hence reduced to estimating small deformation between the subject image and the initially warped template image, i.e., the intermediate template (IT). Specifically, to obtain a good subject-specific initial deformation, support vector regression is utilized to determine the correlation between image appearances and their respective deformation fields. When registering a new subject onto the template, an initial deformation field is first predicted based on the subject's image appearance for generating an IT. With the IT, only the residual deformation needs to be estimated, presenting much less challenge to the existing registration algorithms. Our learning-based framework affords two important advantages: 1) by requiring only the estimation of the residual deformation between the IT and the subject image, the computation time can be greatly reduced; 2) by leveraging good deformation initialization, local minima giving suboptimal solution could be avoided. Our framework has been extensively evaluated using medical images from different sources, and the results indicate that, on top of accuracy improvement, significant registration speedup can be achieved, as compared with the case where no prediction of initial deformation is performed.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Measuring and comparing brain cortical surface area and other areal quantities

Structural analysis of MRI data on the cortical surface usually focuses on cortical thickness. Cortical surface area, when considered, has been measured only over gross regions or approached indirectly via comparisons with a standard brain. Here we demonstrate that direct measurement and comparison of the surface area of the cerebral cortex at a fine scale is possible using mass conservative interpolation methods. We present a framework for analyses of the cortical surface area, as well as for any other measurement distributed across the cortex that is areal by nature. The method consists of the construction of a mesh representation of the cortex, registration to a common coordinate system and, crucially, interpolation using a pycnophylactic method. Statistical analysis of surface area is done with power-transformed data to address lognormality, and inference is done with permutation methods. We introduce the concept of facewise analysis, discuss its interpretation and potential applications.

FreeSurfer

FreeSurfer is a suite of tools for the analysis of neuroimaging data that provides an array of algorithms to quantify the functional, connectional and structural properties of the human brain. It has evolved from a package primarily aimed at generating surface representations of the cerebral cortex into one that automatically creates models of most macroscopically visible structures in the human brain given any reasonable T1-weighted input image. It is freely available, runs on a wide variety of hardware and software platforms, and is open source.

The organization of the human cerebellum estimated by intrinsic functional connectivity

The cerebral cortex communicates with the cerebellum via polysynaptic circuits. Separate regions of the cerebellum are connected to distinct cerebral areas, forming a complex topography. In this study we explored the organization of cerebrocerebellar circuits in the human using resting-state functional connectivity MRI (fcMRI). Data from 1,000 subjects were registered using nonlinear deformation of the cerebellum in combination with surface-based alignment of the cerebral cortex. The foot, hand, and tongue representations were localized in subjects performing movements. fcMRI maps derived from seed regions placed in different parts of the motor body representation yielded the expected inverted map of somatomotor topography in the anterior lobe and the upright map in the posterior lobe. Next, we mapped the complete topography of the cerebellum by estimating the principal cerebral target for each point in the cerebellum in a discovery sample of 500 subjects and replicated the topography in 500 independent subjects. The majority of the human cerebellum maps to association areas. Quantitative analysis of 17 distinct cerebral networks revealed that the extent of the cerebellum dedicated to each network is proportional to the network's extent in the cerebrum with a few exceptions, including primary visual cortex, which is not represented in the cerebellum. Like somatomotor representations, cerebellar regions linked to association cortex have separate anterior and posterior representations that are oriented as mirror images of one another. The orderly topography of the representations suggests that the cerebellum possesses at least two large, homotopic maps of the full cerebrum and possibly a smaller third map.

Probabilistic inference of regularisation in non-rigid registration

A long-standing issue in non-rigid image registration is the choice of the level of regularisation. Regularisation is necessary to preserve the smoothness of the registration and penalise against unnecessary complexity. The vast majority of existing registration methods use a fixed level of regularisation, which is typically hand-tuned by a user to provide "nice" results. However, the optimal level of regularisation will depend on the data which is being processed; lower signal-to-noise ratios require higher regularisation to avoid registering image noise as well as features, and different pairs of images require registrations of varying complexity depending on their anatomical similarity. In this paper we present a probabilistic registration framework that infers the level of regularisation from the data. An additional benefit of this proposed probabilistic framework is that estimates of the registration uncertainty are obtained. This framework has been implemented using a free-form deformation transformation model, although it would be generically applicable to a range of transformation models. We demonstrate our registration framework on the application of inter-subject brain registration of healthy control subjects from the NIREP database. In our results we show that our framework appropriately adapts the level of regularisation in the presence of noise, and that inferring regularisation on an individual basis leads to a reduction in model over-fitting as measured by image folding while providing a similar level of overlap.

The organization of the human cerebral cortex estimated by intrinsic functional connectivity

Information processing in the cerebral cortex involves interactions among distributed areas. Anatomical connectivity suggests that certain areas form local hierarchical relations such as within the visual system. Other connectivity patterns, particularly among association areas, suggest the presence of large-scale circuits without clear hierarchical relations. In this study the organization of networks in the human cerebrum was explored using resting-state functional connectivity MRI. Data from 1,000 subjects were registered using surface-based alignment. A clustering approach was employed to identify and replicate networks of functionally coupled regions across the cerebral cortex. The results revealed local networks confined to sensory and motor cortices as well as distributed networks of association regions. Within the sensory and motor cortices, functional connectivity followed topographic representations across adjacent areas. In association cortex, the connectivity patterns often showed abrupt transitions between network boundaries. Focused analyses were performed to better understand properties of network connectivity. A canonical sensory-motor pathway involving primary visual area, putative middle temporal area complex (MT+), lateral intraparietal area, and frontal eye field was analyzed to explore how interactions might arise within and between networks. Results showed that adjacent regions of the MT+ complex demonstrate differential connectivity consistent with a hierarchical pathway that spans networks. The functional connectivity of parietal and prefrontal association cortices was next explored. Distinct connectivity profiles of neighboring regions suggest they participate in distributed networks that, while showing evidence for interactions, are embedded within largely parallel, interdigitated circuits. We conclude by discussing the organization of these large-scale cerebral networks in relation to monkey anatomy and their potential evolutionary expansion in humans to support cognition.

The organization of the human cerebral cortex estimated by intrinsic functional connectivity

Information processing in the cerebral cortex involves interactions among distributed areas. Anatomical connectivity suggests that certain areas form local hierarchical relations such as within the visual system. Other connectivity patterns, particularly among association areas, suggest the presence of large-scale circuits without clear hierarchical relations. In this study the organization of networks in the human cerebrum was explored using resting-state functional connectivity MRI. Data from 1,000 subjects were registered using surface-based alignment. A clustering approach was employed to identify and replicate networks of functionally coupled regions across the cerebral cortex. The results revealed local networks confined to sensory and motor cortices as well as distributed networks of association regions. Within the sensory and motor cortices, functional connectivity followed topographic representations across adjacent areas. In association cortex, the connectivity patterns often showed abrupt transitions between network boundaries. Focused analyses were performed to better understand properties of network connectivity. A canonical sensory-motor pathway involving primary visual area, putative middle temporal area complex (MT+), lateral intraparietal area, and frontal eye field was analyzed to explore how interactions might arise within and between networks. Results showed that adjacent regions of the MT+ complex demonstrate differential connectivity consistent with a hierarchical pathway that spans networks. The functional connectivity of parietal and prefrontal association cortices was next explored. Distinct connectivity profiles of neighboring regions suggest they participate in distributed networks that, while showing evidence for interactions, are embedded within largely parallel, interdigitated circuits. We conclude by discussing the organization of these large-scale cerebral networks in relation to monkey anatomy and their potential evolutionary expansion in humans to support cognition.

The organization of the human cerebral cortex estimated by intrinsic functional connectivity

Information processing in the cerebral cortex involves interactions among distributed areas. Anatomical connectivity suggests that certain areas form local hierarchical relations such as within the visual system. Other connectivity patterns, particularly among association areas, suggest the presence of large-scale circuits without clear hierarchical relations. In this study the organization of networks in the human cerebrum was explored using resting-state functional connectivity MRI. Data from 1,000 subjects were registered using surface-based alignment. A clustering approach was employed to identify and replicate networks of functionally coupled regions across the cerebral cortex. The results revealed local networks confined to sensory and motor cortices as well as distributed networks of association regions. Within the sensory and motor cortices, functional connectivity followed topographic representations across adjacent areas. In association cortex, the connectivity patterns often showed abrupt transitions between network boundaries. Focused analyses were performed to better understand properties of network connectivity. A canonical sensory-motor pathway involving primary visual area, putative middle temporal area complex (MT+), lateral intraparietal area, and frontal eye field was analyzed to explore how interactions might arise within and between networks. Results showed that adjacent regions of the MT+ complex demonstrate differential connectivity consistent with a hierarchical pathway that spans networks. The functional connectivity of parietal and prefrontal association cortices was next explored. Distinct connectivity profiles of neighboring regions suggest they participate in distributed networks that, while showing evidence for interactions, are embedded within largely parallel, interdigitated circuits. We conclude by discussing the organization of these large-scale cerebral networks in relation to monkey anatomy and their potential evolutionary expansion in humans to support cognition.

Mapping cortical representations of the rodent forepaw and hindpaw with BOLD fMRI reveals two spatial boundaries

Electrical stimulation of the rat forepaw and hindpaw was employed to study the spatial distribution of BOLD fMRI. Averaging of multiple fMRI sessions significantly improved the spatial stability of the BOLD signal and enabled quantitative determination of the boundaries of the BOLD fMRI maps. The averaged BOLD fMRI signal was distributed unevenly over the extent of the map and the data at the boundaries could be modeled with major and minor spatial components. Comparison of three-dimensional echo-planar imaging (EPI) fMRI at isotropic 300 μm resolution demonstrated that the border locations of the major spatial component of BOLD signal did not overlap between the forepaw and hindpaw maps. Interestingly, the border positions of the minor BOLD fMRI spatial components extended significantly into neighboring representations. Similar results were found for cerebral blood volume (CBV) weighted fMRI obtained using iron oxide particles, suggesting that the minor spatial components may not be due to vascular mislocalization typically associated with BOLD fMRI. Comparison of the BOLD fMRI maps of the forepaw and hindpaw to histological determination of these representations using cytochrome oxidase (CO) staining demonstrated that the major spatial component of the BOLD fMRI activation maps accurately localizes the borders. Finally, 2-3 weeks following peripheral nerve denervation, cortical reorganization/plasticity at the boundaries of somatosensory limb representations in adult rat brain was studied. Denervation of the hindpaw caused a growth in the major component of forepaw representation into the adjacent border of hindpaw representation, such that fitting to two components no longer led to a better fit as compared to using one major component. The border of the representation after plasticity was the same as the border of its minor component in the absence of any plasticity. It is possible that the minor components represent either vascular effects that extend from the real neuronal representations or the neuronal communication between neighboring regions. Either way the results will be useful for studying mechanisms of plasticity that cause alterations in the boundaries of neuronal representations.