4D multi-modality tissue segmentation of serial infant images

Subject-specific atlas for automatic brain tissue segmentation of neonatal magnetic resonance images

Developing advanced systems for 3D brain tissue segmentation from neonatal magnetic resonance (MR) images is vital for newborn structural analysis. However, automatic segmentation of neonatal brain tissues is challenging due to smaller head size and inverted T1/T2 tissue contrast compared to adults. In this work, a subject-specific atlas based technique is presented for segmentation of gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) from neonatal MR images. It involves atlas selection, subject-specific atlas creation using random forest (RF) classifier, and brain tissue segmentation using the expectation maximization-Markov random field (EM-MRF) method. To increase the segmentation accuracy, different tissue intensity- and gradient-based features were used. Evaluation on 40 neonatal MR images (gestational age of 37-44 weeks) demonstrated an overall accuracy of 94.3% and an average Dice similarity coefficient (DSC) of 0.945 (GM), 0.947 (WM), and 0.912 (CSF). Compared to multi-atlas segmentation methods like SEGMA and EM-MRF with multiple atlases, our method improved accuracy by up to 4%, particularly in complex tissue regions. Our proposed method allows accurate brain tissue segmentation, a crucial step in brain magnetic resonance imaging (MRI) applications including brain surface reconstruction and realistic head model creation in neonates.

The past, current, and future of neonatal intensive care units with artificial intelligence: a systematic review

Machine learning and deep learning are two subsets of artificial intelligence that involve teaching computers to learn and make decisions from any sort of data. Most recent developments in artificial intelligence are coming from deep learning, which has proven revolutionary in almost all fields, from computer vision to health sciences. The effects of deep learning in medicine have changed the conventional ways of clinical application significantly. Although some sub-fields of medicine, such as pediatrics, have been relatively slow in receiving the critical benefits of deep learning, related research in pediatrics has started to accumulate to a significant level, too. Hence, in this paper, we review recently developed machine learning and deep learning-based solutions for neonatology applications. We systematically evaluate the roles of both classical machine learning and deep learning in neonatology applications, define the methodologies, including algorithmic developments, and describe the remaining challenges in the assessment of neonatal diseases by using PRISMA 2020 guidelines. To date, the primary areas of focus in neonatology regarding AI applications have included survival analysis, neuroimaging, analysis of vital parameters and biosignals, and retinopathy of prematurity diagnosis. We have categorically summarized 106 research articles from 1996 to 2022 and discussed their pros and cons, respectively. In this systematic review, we aimed to further enhance the comprehensiveness of the study. We also discuss possible directions for new AI models and the future of neonatology with the rising power of AI, suggesting roadmaps for the integration of AI into neonatal intensive care units.

3D-MASNet: 3D mixed-scale asymmetric convolutional segmentation network for 6-month-old infant brain MR images

Precise segmentation of infant brain magnetic resonance (MR) images into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) are essential for studying neuroanatomical hallmarks of early brain development. However, for 6-month-old infants, the extremely low-intensity contrast caused by inherent myelination hinders accurate tissue segmentation. Existing convolutional neural networks (CNNs) based segmentation models for this task generally employ single-scale symmetric convolutions, which are inefficient for encoding the isointense tissue boundaries in baby brain images. Here, we propose a 3D mixed-scale asymmetric convolutional segmentation network (3D-MASNet) framework for brain MR images of 6-month-old infants. We replaced the traditional convolutional layer of an existing to-be-trained network with a 3D mixed-scale convolution block consisting of asymmetric kernels (MixACB) during the training phase and then equivalently converted it into the original network. Five canonical CNN segmentation models were evaluated using both T1- and T2-weighted images of 23 6-month-old infants from iSeg-2019 datasets, which contained manual labels as ground truth. MixACB significantly enhanced the average accuracy of all five models and obtained the most considerable improvement in the fully convolutional network model (CC-3D-FCN) and the highest performance in the Dense U-Net model. This approach further obtained Dice coefficient accuracies of 0.931, 0.912, and 0.961 in GM, WM, and CSF, respectively, ranking first among 30 teams on the validation dataset of the iSeg-2019 Grand Challenge. Thus, the proposed 3D-MASNet can improve the accuracy of existing CNNs-based segmentation models as a plug-and-play solution that offers a promising technique for future infant brain MRI studies.

Segmentation of Infant Brain Using Nonnegative Matrix Factorization

This study develops an atlas-based automated framework for segmenting infants’ brains from magnetic resonance imaging (MRI). For the accurate segmentation of different structures of an infant’s brain at the isointense age (6–12 months), our framework integrates features of diffusion tensor imaging (DTI) (e.g., the fractional anisotropy (FA)). A brain diffusion tensor (DT) image and its region map are considered samples of a Markov–Gibbs random field (MGRF) that jointly models visual appearance, shape, and spatial homogeneity of a goal structure. The visual appearance is modeled with an empirical distribution of the probability of the DTI features, fused by their nonnegative matrix factorization (NMF) and allocation to data clusters. Projecting an initial high-dimensional feature space onto a low-dimensional space of the significant fused features with the NMF allows for better separation of the goal structure and its background. The cluster centers in the latter space are determined at the training stage by the K-means clustering. In order to adapt to large infant brain inhomogeneities and segment the brain images more accurately, appearance descriptors of both the first-order and second-order are taken into account in the fused NMF feature space. Additionally, a second-order MGRF model is used to describe the appearance based on the voxel intensities and their pairwise spatial dependencies. An adaptive shape prior that is spatially variant is constructed from a training set of co-aligned images, forming an atlas database. Moreover, the spatial homogeneity of the shape is described with a spatially uniform 3D MGRF of the second-order for region labels. In vivo experiments on nine infant datasets showed promising results in terms of the accuracy, which was computed using three metrics: the 95-percentile modified Hausdorff distance (MHD), the Dice similarity coefficient (DSC), and the absolute volume difference (AVD). Both the quantitative and visual assessments confirm that integrating the proposed NMF-fused DTI feature and intensity MGRF models of visual appearance, the adaptive shape prior, and the shape homogeneity MGRF model is promising in segmenting the infant brain DTI.

Novel User-Friendly Application for MRI Segmentation of Brain Resection following Epilepsy Surgery

Delineation of resected brain cavities on magnetic resonance images (MRIs) of epilepsy surgery patients is essential for neuroimaging/neurophysiology studies investigating biomarkers of the epileptogenic zone. The gold standard to delineate the resection on MRI remains manual slice-by-slice tracing by experts. Here, we proposed and validated a semiautomated MRI segmentation pipeline, generating an accurate model of the resection and its anatomical labeling, and developed a graphical user interface (GUI) for user-friendly usage. We retrieved pre- and postoperative MRIs from 35 patients who had focal epilepsy surgery, implemented a region-growing algorithm to delineate the resection on postoperative MRIs and tested its performance while varying different tuning parameters. Similarity between our output and hand-drawn gold standards was evaluated via dice similarity coefficient (DSC; range: 0-1). Additionally, the best segmentation pipeline was trained to provide an automated anatomical report of the resection (based on presurgical brain atlas). We found that the best-performing set of parameters presented DSC of 0.83 (0.72-0.85), high robustness to seed-selection variability and anatomical accuracy of 90% to the clinical postoperative MRI report. We presented a novel user-friendly open-source GUI that implements a semiautomated segmentation pipeline specifically optimized to generate resection models and their anatomical reports from epilepsy surgery patients, while minimizing user interaction.

Infant and Child MRI: A Review of Scanning Procedures

Magnetic resonance imaging (MRI) is a safe method to examine human brain. However, a typical MR scan is very sensitive to motion, and it requires the subject to lie still during the acquisition, which is a major challenge for pediatric scans. Consequently, in a clinical setting, sedation or general anesthesia is often used. In the research setting including healthy subjects anesthetics are not recommended for ethical reasons and potential longer-term harm. Here we review the methods used to prepare a child for an MRI scan, but also on the techniques and tools used during the scanning to enable a successful scan. Additionally, we critically evaluate how studies have reported the scanning procedure and success of scanning. We searched articles based on special subject headings from PubMed and identified 86 studies using brain MRI in healthy subjects between 0 and 6 years of age. Scan preparations expectedly depended on subject's age; infants and young children were scanned asleep after feeding and swaddling and older children were scanned awake. Comparing the efficiency of different procedures was difficult because of the heterogeneous reporting of the used methods and the success rates. Based on this review, we recommend more detailed reporting of scanning procedure to help find out which are the factors affecting the success of scanning. In the long term, this could help the research field to get high quality data, but also the clinical field to reduce the use of anesthetics. Finally, we introduce the protocol used in scanning 2 to 5-week-old infants in the FinnBrain Birth Cohort Study, and tips for calming neonates during the scans.

Using Deep Convolutional Neural Networks for Neonatal Brain Image Segmentation

INTRODUCTION

Deep learning neural networks are especially potent at dealing with structured data, such as images and volumes. Both modified LiviaNET and HyperDense-Net performed well at a prior competition segmenting 6-month-old infant magnetic resonance images, but neonatal cerebral tissue type identification is challenging given its uniquely inverted tissue contrasts. The current study aims to evaluate the two architectures to segment neonatal brain tissue types at term equivalent age.

METHODS

Both networks were retrained over 24 pairs of neonatal T1 and T2 data from the Developing Human Connectome Project public data set and validated on another eight pairs against ground truth. We then reported the best-performing model from training and its performance by computing the Dice similarity coefficient (DSC) for each tissue type against eight test subjects.

RESULTS

During the testing phase, among the segmentation approaches tested, the dual-modality HyperDense-Net achieved the best statistically significantly test mean DSC values, obtaining 0.94/0.95/0.92 for the tissue types and took 80 h to train and 10 min to segment, including preprocessing. The single-modality LiviaNET was better at processing T2-weighted images than processing T1-weighted images across all tissue types, achieving mean DSC values of 0.90/0.90/0.88 for gray matter, white matter, and cerebrospinal fluid, respectively, while requiring 30 h to train and 8 min to segment each brain, including preprocessing.

DISCUSSION

Our evaluation demonstrates that both neural networks can segment neonatal brains, achieving previously reported performance. Both networks will be continuously retrained over an increasingly larger repertoire of neonatal brain data and be made available through the Canadian Neonatal Brain Platform to better serve the neonatal brain imaging research community.

Parcellation of the neonatal cortex using Surface-based Melbourne Children's Regional Infant Brain atlases (M-CRIB-S)

Longitudinal studies measuring changes in cortical morphology over time are best facilitated by parcellation schemes compatible across all life stages. The Melbourne Children’s Regional Infant Brain (M-CRIB) and M-CRIB 2.0 atlases provide voxel-based parcellations of the cerebral cortex compatible with the Desikan-Killiany (DK) and the Desikan-Killiany-Tourville (DKT) cortical labelling schemes. This study introduces surface-based versions of the M-CRIB and M-CRIB 2.0 atlases, termed M-CRIB-S(DK) and M-CRIB-S(DKT), with a pipeline for automated parcellation utilizing FreeSurfer and developing Human Connectome Project (dHCP) tools. Using T₂-weighted magnetic resonance images of healthy neonates (n = 58), we created average spherical templates of cortical curvature and sulcal depth. Manually labelled regions in a subset (n = 10) were encoded into the spherical template space to construct M-CRIB-S(DK) and M-CRIB-S(DKT) atlases. Labelling accuracy was assessed using Dice overlap and boundary discrepancy measures with leave-one-out cross-validation. Cross-validated labelling accuracy was high for both atlases (average regional Dice = 0.79–0.83). Worst-case boundary discrepancy instances ranged from 9.96–10.22 mm, which appeared to be driven by variability in anatomy for some cases. The M-CRIB-S atlas data and automatic pipeline allow extraction of neonatal cortical surfaces labelled according to the DK or DKT parcellation schemes.

Role of deep learning in infant brain MRI analysis

Deep learning algorithms and in particular convolutional networks have shown tremendous success in medical image analysis applications, though relatively few methods have been applied to infant MRI data due numerous inherent challenges such as inhomogenous tissue appearance across the image, considerable image intensity variability across the first year of life, and a low signal to noise setting. This paper presents methods addressing these challenges in two selected applications, specifically infant brain tissue segmentation at the isointense stage and presymptomatic disease prediction in neurodevelopmental disorders. Corresponding methods are reviewed and compared, and open issues are identified, namely low data size restrictions, class imbalance problems, and lack of interpretation of the resulting deep learning solutions. We discuss how existing solutions can be adapted to approach these issues as well as how generative models seem to be a particularly strong contender to address them.

Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation

Precise 3D segmentation of infant brain tissues is an essential step towards comprehensive volumetric studies and quantitative analysis of early brain development. However, computing such segmentations is very challenging, especially for 6-month infant brain, due to the poor image quality, among other difficulties inherent to infant brain MRI, e.g., the isointense contrast between white and gray matter and the severe partial volume effect due to small brain sizes. This study investigates the problem with an ensemble of semi-dense fully convolutional neural networks (CNNs), which employs T1-weighted and T2-weighted MR images as input. We demonstrate that the ensemble agreement is highly correlated with the segmentation errors. Therefore, our method provides measures that can guide local user corrections. To the best of our knowledge, this work is the first ensemble of 3D CNNs for suggesting annotations within images. Our quasi-dense architecture allows the efficient propagation of gradients during training, while limiting the number of parameters, requiring one order of magnitude less parameters than popular medical image segmentation networks such as 3D U-Net (Çiçek, et al.). We also investigated the impact that early or late fusions of multiple image modalities might have on the performances of deep architectures. We report evaluations of our method on the public data of the MICCAI iSEG-2017 Challenge on 6-month infant brain MRI segmentation, and show very competitive results among 21 teams, ranking first or second in most metrics.

A Novel Approach for Manual Segmentation of the Amygdala and Hippocampus in Neonate MRI

The gross anatomy of the infant brain at term is fairly similar to that of the adult brain, but structures are immature, and the brain undergoes rapid growth during the first 2 years of life. Neonate magnetic resonance (MR) images have different contrasts compared to adult images, and automated segmentation of brain magnetic resonance imaging (MRI) can thus be considered challenging as less software options are available. Despite this, most anatomical regions are identifiable and thus amenable to manual segmentation. In the current study, we developed a protocol for segmenting the amygdala and hippocampus in T2-weighted neonatal MR images. The participants were 31 healthy infants between 2 and 5 weeks of age. Intra-rater reliability was measured in 12 randomly selected MR images, where 6 MR images were segmented at 1-month intervals between the delineations, and another 6 MR images at 6-month intervals. The protocol was also tested by two independent raters in 20 randomly selected T2-weighted images, and finally with T1 images. Intraclass correlation coefficient (ICC) and Dice similarity coefficient (DSC) for intra-rater, inter-rater, and T1 vs. T2 comparisons were computed. Moreover, manual segmentations were compared to automated segmentations performed by iBEAT toolbox in 10 T2-weighted MR images. The intra-rater reliability was high ICC ≥ 0.91, DSC ≥ 0.89, the inter-rater reliabilities were satisfactory ICC ≥ 0.90, DSC ≥ 0.75 for hippocampus and DSC ≥ 0.52 for amygdalae. Segmentations for T1 vs. T2-weighted images showed high consistency ICC ≥ 0.90, DSC ≥ 0.74. The manual and iBEAT segmentations showed no agreement, DSC ≥ 0.39. In conclusion, there is a clear need to improve and develop the procedures for automated segmentation of infant brain MR images.

Construction of 4D infant cortical surface atlases with sharp folding patterns via spherical patch-based group-wise sparse representation

4D (spatial + temporal) infant cortical surface atlases covering dense time points are highly needed for understanding dynamic early brain development. In this article, we construct a set of 4D infant cortical surface atlases with longitudinally consistent and sharp cortical attribute patterns at 11 time points in the first six postnatal years, that is, at 1, 3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months of age, which is targeted for better normalization of the dynamic changing early brain cortical surfaces. To ensure longitudinal consistency and unbiasedness, we adopt a two-stage group-wise surface registration. To preserve sharp cortical attribute patterns on the atlas, instead of simply averaging over the coregistered cortical surfaces, we leverage a spherical patch-based sparse representation using the augmented dictionary to overcome the potential registration errors. Our atlases provide not only geometric attributes of the cortical folding, but also cortical thickness and myelin content. Therefore, to address the consistency across different cortical attributes on the atlas, instead of sparsely representing each attribute independently, we jointly represent all cortical attributes with a group-wise sparsity constraint. In addition, to further facilitate region-based analysis using our atlases, we have also provided two widely used parcellations, that is, FreeSurfer parcellation and multimodal parcellation, on our 4D infant cortical surface atlases. Compared to cortical surface atlases constructed with other methods, our cortical surface atlases preserve sharper cortical folding attribute patterns, thus leading to better accuracy in registration of individual infant cortical surfaces to the atlas.

3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.

Hierarchical Rough-to-Fine Model for Infant Age Prediction Based on Cortical Features

Prediction of the chronological age based on neuroimaging data is important for brain development analysis and brain disease diagnosis. Although many researches have been conducted for age prediction of older children and adults, little work has been dedicated to infants. To this end, this paper focuses on predicting infant age from birth to 2-year old using brain MR images, as well as identifying some related biomarkers. However, brain development during infancy is too rapid and heterogeneous to be accurately modeled by the conventional regression models. To address this issue, a two-stage prediction method is proposed. Specifically, our method first roughly predicts the age range of an infant and then finely predicts the accurate chronological age based on a learned, age-group-specific regression model. Combining this two-stage prediction method with another complementary one-stage prediction method, a hierarchical rough-to-fine (HRtoF) model is built. HRtoF effectively splits the rapid and heterogeneous changes during a long time period into several short time ranges and further mines the discrimination capability of cortical features, thus reaching high accuracy in infant age prediction. Taking 8 types of cortical morphometric features from structural MRI as predictors, the effectiveness of our proposed HRtoF model is validated using an infant dataset including 50 healthy subjects with 251 longitudinal MRI scans from 14 to 797 days. Comparing with five state-of-the-art regression methods, HRtoF model reduces the mean absolute error of the prediction from >48 days to 32.1 days. The correlation coefficient of the predicted age and the chronological age reaches 0.963. Moreover, based on HRtoF, the relative contributions of the eight types of cortical features for age prediction are also studied.

First-year development of modules and hubs in infant brain functional networks

The human brain develops rapidly in the first postnatal year, in which rewired functional brain networks could shape later behavioral and cognitive performance. Resting-state functional magnetic resonances imaging (rs-fMRI) and complex network analysis have been widely used for characterizing the developmental brain functional connectome. Yet, such studies focusing on the first year of postnatal life are still very limited. Leveraging normally developing longitudinal infant rs-fMRI scans from neonate to one year of age, we investigated how brain functional networks develop at a fine temporal scale (every 3 months). Considering challenges in the infant fMRI-based network analysis, we developed a novel algorithm to construct the robust, temporally consistent and modular structure augmented group-level network based on which functional modules were detected at each age. Our study reveals that the brain functional network is gradually subdivided into an increasing number of functional modules accompanied by the strengthened intra- and inter-modular connectivities. Based on the developing modules, we found connector hubs (the high-centrality regions connecting different modules) emerging and increasing, while provincial hubs (the high-centrality regions connecting regions in the same module) diminishing. Further region-wise longitudinal analysis validates that different hubs have distinct developmental trajectories of the intra- and inter-modular connections suggesting different types of role transitions in network, such as non-hubs to hubs or provincial hubs to connector hubs et al. All findings indicate that functional segregation and integration are both increased in the first year of postnatal life. The module reorganization and hub transition lead to more efficient brain networks, featuring increasingly segregated modular structure and more connector hubs. This study provides the first comprehensive report of the development of functional brain networks at a 3-month interval throughout the first postnatal year of life, which provides essential information to the future neurodevelopmental and developmental disorder studies.

Anatomy-guided joint tissue segmentation and topological correction for 6-month infant brain MRI with risk of autism

Tissue segmentation of infant brain MRIs with risk of autism is critically important for characterizing early brain development and identifying biomarkers. However, it is challenging due to low tissue contrast caused by inherent ongoing myelination and maturation. In particular, at around 6 months of age, the voxel intensities in both gray matter and white matter are within similar ranges, thus leading to the lowest image contrast in the first postnatal year. Previous studies typically employed intensity images and tentatively estimated tissue probabilities to train a sequence of classifiers for tissue segmentation. However, the important prior knowledge of brain anatomy is largely ignored during the segmentation. Consequently, the segmentation accuracy is still limited and topological errors frequently exist, which will significantly degrade the performance of subsequent analyses. Although topological errors could be partially handled by retrospective topological correction methods, their results may still be anatomically incorrect. To address these challenges, in this article, we propose an anatomy-guided joint tissue segmentation and topological correction framework for isointense infant MRI. Particularly, we adopt a signed distance map with respect to the outer cortical surface as anatomical prior knowledge, and incorporate such prior information into the proposed framework to guide segmentation in ambiguous regions. Experimental results on the subjects acquired from National Database for Autism Research demonstrate the effectiveness to topological errors and also some levels of robustness to motion. Comparisons with the state-of-the-art methods further demonstrate the advantages of the proposed method in terms of both segmentation accuracy and topological correctness.

A Kinect-Based Segmentation of Touching-Pigs for Real-Time Monitoring

Segmenting touching-pigs in real-time is an important issue for surveillance cameras intended for the 24-h tracking of individual pigs. However, methods to do so have not yet been reported. We particularly focus on the segmentation of touching-pigs in a crowded pig room with low-contrast images obtained using a Kinect depth sensor. We reduce the execution time by combining object detection techniques based on a convolutional neural network (CNN) with image processing techniques instead of applying time-consuming operations, such as optimization-based segmentation. We first apply the fastest CNN-based object detection technique (i.e., You Only Look Once, YOLO) to solve the separation problem for touching-pigs. If the quality of the YOLO output is not satisfied, then we try to find the possible boundary line between the touching-pigs by analyzing the shape. Our experimental results show that this method is effective to separate touching-pigs in terms of both accuracy (i.e., 91.96%) and execution time (i.e., real-time execution), even with low-contrast images obtained using a Kinect depth sensor.

A Label-fusion-aided Convolutional Neural Network for Isointense Infant Brain Tissue Segmentation

The extremely low tissue contrast in white matter during an infant's isointense stage (6-8 months) of brain development presents major difficulty when segmenting brain image regions for analysis. We sought to develop a label-fusion-aided deep-learning approach for automatically segmenting isointense infant brain images into white matter, gray matter and cerebrospinal fluid using T1- and T2-weighted magnetic resonance images. A key idea of our approach is to apply the fully convolutional neural network (FCNN) to individual brain regions determined by a traditional registration-based segmentation method instead of training a single model for the whole brain. This provides more refined segmentation results by capturing more region-specific features. We show that this method outperforms traditional joint label fusion and FCNN-only methods in terms of Dice coefficients using the dataset from iSEG MICCAI Grand Challenge 2017.

Computational neuroanatomy of baby brains: A review

The first postnatal years are an exceptionally dynamic and critical period of structural, functional and connectivity development of the human brain. The increasing availability of non-invasive infant brain MR images provides unprecedented opportunities for accurate and reliable charting of dynamic early brain developmental trajectories in understanding normative and aberrant growth. However, infant brain MR images typically exhibit reduced tissue contrast (especially around 6 months of age), large within-tissue intensity variations, and regionally-heterogeneous, dynamic changes, in comparison with adult brain MR images. Consequently, the existing computational tools developed typically for adult brains are not suitable for infant brain MR image processing. To address these challenges, many infant-tailored computational methods have been proposed for computational neuroanatomy of infant brains. In this review paper, we provide a comprehensive review of the state-of-the-art computational methods for infant brain MRI processing and analysis, which have advanced our understanding of early postnatal brain development. We also summarize publically available infant-dedicated resources, including MRI datasets, computational tools, grand challenges, and brain atlases. Finally, we discuss the limitations in current research and suggest potential future research directions.

Learning-based subject-specific estimation of dynamic maps of cortical morphology at missing time points in longitudinal infant studies

Longitudinal neuroimaging analysis of the dynamic brain development in infants has received increasing attention recently. Many studies expect a complete longitudinal dataset in order to accurately chart the brain developmental trajectories. However, in practice, a large portion of subjects in longitudinal studies often have missing data at certain time points, due to various reasons such as the absence of scan or poor image quality. To make better use of these incomplete longitudinal data, in this paper, we propose a novel machine learning-based method to estimate the subject-specific, vertex-wise cortical morphological attributes at the missing time points in longitudinal infant studies. Specifically, we develop a customized regression forest, named dynamically assembled regression forest (DARF), as the core regression tool. DARF ensures the spatial smoothness of the estimated maps for vertex-wise cortical morphological attributes and also greatly reduces the computational cost. By employing a pairwise estimation followed by a joint refinement, our method is able to fully exploit the available information from both subjects with complete scans and subjects with missing scans for estimation of the missing cortical attribute maps. The proposed method has been applied to estimating the dynamic cortical thickness maps at missing time points in an incomplete longitudinal infant dataset, which includes 31 healthy infant subjects, each having up to five time points in the first postnatal year. The experimental results indicate that our proposed framework can accurately estimate the subject-specific vertex-wise cortical thickness maps at missing time points, with the average error less than 0.23 mm. Hum Brain Mapp 37:4129-4147, 2016. © 2016 Wiley Periodicals, Inc.

Can we predict subject-specific dynamic cortical thickness maps during infancy from birth?

Understanding the early dynamic development of the human cerebral cortex remains a challenging problem. Cortical thickness, as one of the most important morphological attributes of the cerebral cortex, is a sensitive indicator for both normal neurodevelopment and neuropsychiatric disorders, but its early postnatal development remains largely unexplored. In this study, we investigate a key question in neurodevelopmental science: can we predict the future dynamic development of cortical thickness map in an individual infant based on its available MRI data at birth? If this is possible, we might be able to better model and understand the early brain development and also early detect abnormal brain development during infancy. To this end, we develop a novel learning-based method, called Dynamically-Assembled Regression Forest (DARF), to predict the development of the cortical thickness map during the first postnatal year, based on neonatal MRI features. We applied our method to 15 healthy infants and predicted their cortical thickness maps at 3, 6, 9, and 12 months of age, with respectively mean absolute errors of 0.209 mm, 0.332 mm, 0.340 mm, and 0.321 mm. Moreover, we found that the prediction precision is region-specific, with high precision in the unimodal cortex and relatively low precision in the high-order association cortex, which may be associated with their differential developmental patterns. Additional experiments also suggest that using more early time points for prediction can further significantly improve the prediction accuracy. Hum Brain Mapp 38:2865-2874, 2017. © 2017 Wiley Periodicals, Inc.

Scalable Joint Segmentation and Registration Framework for Infant Brain Images

The first year of life is the most dynamic and perhaps the most critical phase of postnatal brain development. The ability to accurately measure structure changes is critical in early brain development study, which highly relies on the performances of image segmentation and registration techniques. However, either infant image segmentation or registration, if deployed independently, encounters much more challenges than segmentation/registration of adult brains due to dynamic appearance change with rapid brain development. In fact, image segmentation and registration of infant images can assists each other to overcome the above challenges by using the growth trajectories (i.e., temporal correspondences) learned from a large set of training subjects with complete longitudinal data. Specifically, a one-year-old image with ground-truth tissue segmentation can be first set as the reference domain. Then, to register the infant image of a new subject at earlier age, we can estimate its tissue probability maps, i.e., with sparse patch-based multi-atlas label fusion technique, where only the training images at the respective age are considered as atlases since they have similar image appearance. Next, these probability maps can be fused as a good initialization to guide the level set segmentation. Thus, image registration between the new infant image and the reference image is free of difficulty of appearance changes, by establishing correspondences upon the reasonably segmented images. Importantly, the segmentation of new infant image can be further enhanced by propagating the much more reliable label fusion heuristics at the reference domain to the corresponding location of the new infant image via the learned growth trajectories, which brings image segmentation and registration to assist each other. It is worth noting that our joint segmentation and registration framework is also flexible to handle the registration of any two infant images even with significant age gap in the first year of life, by linking their joint segmentation and registration through the reference domain. Thus, our proposed joint segmentation and registration method is scalable to various registration tasks in early brain development studies. Promising segmentation and registration results have been achieved for infant brain MR images aged from 2-week-old to 1-year-old, indicating the applicability of our method in early brain development study.

Learning-based deformable image registration for infant MR images in the first year of life

PURPOSE

Many brain development studies have been devoted to investigate dynamic structural and functional changes in the first year of life. To quantitatively measure brain development in such a dynamic period, accurate image registration for different infant subjects with possible large age gap is of high demand. Although many state-of-the-art image registration methods have been proposed for young and elderly brain images, very few registration methods work for infant brain images acquired in the first year of life, because of (a) large anatomical changes due to fast brain development and (b) dynamic appearance changes due to white-matter myelination.

METHODS

To address these two difficulties, we propose a learning-based registration method to not only align the anatomical structures but also alleviate the appearance differences between two arbitrary infant MR images (with large age gap) by leveraging the regression forest to predict both the initial displacement vector and appearance changes. Specifically, in the training stage, two regression models are trained separately, with (a) one model learning the relationship between local image appearance (of one development phase) and its displacement toward the template (of another development phase) and (b) another model learning the local appearance changes between the two brain development phases. Then, in the testing stage, to register a new infant image to the template, we first predict both its voxel-wise displacement and appearance changes by the two learned regression models. Since such initializations can alleviate significant appearance and shape differences between new infant image and the template, it is easy to just use a conventional registration method to refine the remaining registration.

RESULTS

We apply our proposed registration method to align 24 infant subjects at five different time points (i.e., 2-week-old, 3-month-old, 6-month-old, 9-month-old, and 12-month-old), and achieve more accurate and robust registration results, compared to the state-of-the-art registration methods.

CONCLUSIONS

The proposed learning-based registration method addresses the challenging task of registering infant brain images and achieves higher registration accuracy compared with other counterpart registration methods.

White Matter Injury and General Movements in High-Risk Preterm Infants

BACKGROUND AND PURPOSE

Very preterm infants (birth weight, <1500 g) are at increased risk of cognitive and motor impairment, including cerebral palsy. These adverse neurodevelopmental outcomes are associated with white matter abnormalities on MR imaging at term-equivalent age. Cerebral palsy has been predicted by analysis of spontaneous movements in the infant termed "General Movement Assessment." The goal of this study was to determine the utility of General Movement Assessment in predicting adverse cognitive, language, and motor outcomes in very preterm infants and to identify brain imaging markers associated with both adverse outcomes and aberrant general movements.

MATERIALS AND METHODS

In this prospective study of 47 preterm infants of 24-30 weeks' gestation, brain MR imaging was performed at term-equivalent age. Infants underwent T1- and T2-weighted imaging for volumetric analysis and DTI. General movements were assessed at 10-15 weeks' postterm age, and neurodevelopmental outcomes were evaluated at 2 years by using the Bayley Scales of Infant and Toddler Development III.

RESULTS

Nine infants had aberrant general movements and were more likely to have adverse neurodevelopmental outcomes, compared with infants with normal movements. In infants with aberrant movements, Tract-Based Spatial Statistics analysis identified significantly lower fractional anisotropy in widespread white matter tracts, including the corpus callosum, inferior longitudinal and fronto-occipital fasciculi, internal capsule, and optic radiation. The subset of infants having both aberrant movements and abnormal neurodevelopmental outcomes in cognitive, language, and motor skills had significantly lower fractional anisotropy in specific brain regions.

CONCLUSIONS

Aberrant general movements at 10-15 weeks' postterm are associated with adverse neurodevelopmental outcomes and specific white matter microstructure abnormalities for cognitive, language, and motor delays.

Biomechanical Analysis of Normal Brain Development during the First Year of Life Using Finite Strain Theory

The first year of life is the most critical time period for structural and functional development of the human brain. Combining longitudinal MR imaging and finite strain theory, this study aimed to provide new insights into normal brain development through a biomechanical framework. Thirty-three normal infants were longitudinally imaged using MRI from 2 weeks to 1 year of age. Voxel-wise Jacobian determinant was estimated to elucidate volumetric changes while Lagrange strains (both normal and shear strains) were measured to reveal directional growth information every 3 months during the first year of life. Directional normal strain maps revealed that, during the first 6 months, the growth pattern of gray matter is anisotropic and spatially inhomogeneous with higher left-right stretch around the temporal lobe and interhemispheric fissure, anterior-posterior stretch in the frontal and occipital lobes, and superior-inferior stretch in right inferior occipital and right inferior temporal gyri. In contrast, anterior lateral ventricles and insula showed an isotropic stretch pattern. Volumetric and directional growth rates were linearly decreased with age for most of the cortical regions. Our results revealed anisotropic and inhomogeneous brain growth patterns of the human brain during the first year of life using longitudinal MRI and a biomechanical framework.

Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views

Segmentation of the placenta from fetal MRI is challenging due to sparse acquisition, inter-slice motion, and the widely varying position and shape of the placenta between pregnant women. We propose a minimally interactive framework that combines multiple volumes acquired in different views to obtain accurate segmentation of the placenta. In the first phase, a minimally interactive slice-by-slice propagation method called Slic-Seg is used to obtain an initial segmentation from a single motion-corrupted sparse volume image. It combines high-level features, online Random Forests and Conditional Random Fields, and only needs user interactions in a single slice. In the second phase, to take advantage of the complementary resolution in multiple volumes acquired in different views, we further propose a probability-based 4D Graph Cuts method to refine the initial segmentations using inter-slice and inter-image consistency. We used our minimally interactive framework to examine the placentas of 16 mid-gestation patients from MRI acquired in axial and sagittal views respectively. The results show the proposed method has 1) a good performance even in cases where sparse scribbles provided by the user lead to poor results with the competitive propagation approaches; 2) a good interactivity with low intra- and inter-operator variability; 3) higher accuracy than state-of-the-art interactive segmentation methods; and 4) an improved accuracy due to the co-segmentation based refinement, which outperforms single volume or intensity-based Graph Cuts.

Consistent Spatial-Temporal Longitudinal Atlas Construction for Developing Infant Brains

Brain atlases are an essential component in understanding the dynamic cerebral development, especially for the early postnatal period. However, longitudinal atlases are rare for infants, and the existing ones are generally limited by their fuzzy appearance. Moreover, since longitudinal atlas construction is typically performed independently over time, the constructed atlases often fail to preserve temporal consistency. This problem is further aggravated for infant images since they typically have low spatial resolution and insufficient tissue contrast. In this paper, we propose a novel framework for consistent spatial-temporal construction of longitudinal atlases for developing infant brain MR images. Specifically, for preserving structural details, the atlas construction is performed in spatial-temporal wavelet domain simultaneously. This is achieved by a patch-based combination of results from each frequency subband. Compared with the existing infant longitudinal atlases, our experimental results indicate that our approach is able to produce longitudinal atlases with richer structural details and also better longitudinal consistency, thus leading to higher performance when used for spatial normalization of a group of infant brain images.

FULLY CONVOLUTIONAL NETWORKS FOR MULTI-MODALITY ISOINTENSE INFANT BRAIN IMAGE SEGMENTATION

The segmentation of infant brain tissue images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) plays an important role in studying early brain development. In the isointense phase (approximately 6-8 months of age), WM and GM exhibit similar levels of intensity in both T1 and T2 MR images, resulting in extremely low tissue contrast and thus making the tissue segmentation very challenging. The existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single T1, T2 or fractional anisotropy (FA) modality or their simply-stacked combinations without fully exploring the multi-modality information. To address the challenge, in this paper, we propose to use fully convolutional networks (FCNs) for the segmentation of isointense phase brain MR images. Instead of simply stacking the three modalities, we train one network for each modality image, and then fuse their high-layer features together for final segmentation. Specifically, we conduct a convolution-pooling stream for multimodality information from T1, T2, and FA images separately, and then combine them in high-layer for finally generating the segmentation maps as the outputs. We compared the performance of our approach with that of the commonly used segmentation methods on a set of manually segmented isointense phase brain images. Results showed that our proposed model significantly outperformed previous methods in terms of accuracy. In addition, our results also indicated a better way of integrating multi-modality images, which leads to performance improvement.

Hierarchical and symmetric infant image registration by robust longitudinal-example-guided correspondence detection

PURPOSE

To investigate anatomical differences across individual subjects, or longitudinal changes in early brain development, it is important to perform accurate image registration. However, due to fast brain development and dynamic tissue appearance changes, it is very difficult to align infant brain images acquired from birth to 1-yr-old.

METHODS

To solve this challenging problem, a novel image registration method is proposed to align two infant brain images, regardless of age at acquisition. The main idea is to utilize the growth trajectories, or spatial-temporal correspondences, learned from a set of longitudinal training images, for guiding the registration of two different time-point images with different image appearances. Specifically, in the training stage, an intrinsic growth trajectory is first estimated for each training subject using the longitudinal images. To register two new infant images with potentially a large age gap, the corresponding images patches between each new image and its respective training images with similar age are identified. Finally, the registration between the two new images can be assisted by the learned growth trajectories from one time point to another time point that have been established in the training stage. To further improve registration accuracy, the proposed method is combined with a hierarchical and symmetric registration framework that can iteratively add new key points in both images to steer the estimation of the deformation between the two infant brain images under registration.

RESULTS

To evaluate image registration accuracy, the proposed method is used to align 24 infant subjects at five different time points (2-week-old, 3-month-old, 6-month-old, 9-month-old, and 12-month-old). Compared to the state-of-the-art methods, the proposed method demonstrated superior registration performance.

CONCLUSIONS

The proposed method addresses the difficulties in the infant brain registration and produces better results compared to existing state-of-the-art registration methods.

Detail-preserving construction of neonatal brain atlases in space-frequency domain

Brain atlases are commonly utilized in neuroimaging studies. However, most brain atlases are fuzzy and lack structural details, especially in the cortical regions. This is mainly caused by the image averaging process involved in atlas construction, which often smoothes out high-frequency contents that capture fine anatomical details. Brain atlas construction for neonatal images is even more challenging due to insufficient spatial resolution and low tissue contrast. In this paper, we propose a novel framework for detail-preserving construction of population-representative atlases. Our approach combines spatial and frequency information to better preserve image details. This is achieved by performing atlas construction in the space-frequency domain given by wavelet transform. In particular, sparse patch-based atlas construction is performed in all frequency subbands, and the results are combined to give a final atlas. For enhancing anatomical details, tissue probability maps are also used to guide atlas construction. Experimental results show that our approach can produce atlases with greater structural details than existing atlases. Hum Brain Mapp 37:2133-2150, 2016. © 2016 Wiley Periodicals, Inc.

Multi-Atlas and Multi-Modal Hippocampus Segmentation for Infant MR Brain Images by Propagating Anatomical Labels on Hypergraph

Accurate segmentation of hippocampus from infant magnetic resonance (MR) images is very important in the study of early brain development and neurological disorder. Recently, multi-atlas patch-based label fusion methods have shown a great success in segmenting anatomical structures from medical images. However, the dramatic appearance change from birth to 1-year-old and the poor image contrast make the existing label fusion methods less competitive to handle infant brain images. To alleviate these difficulties, we propose a novel multi-atlas and multi-modal label fusion method, which can unanimously label for all voxels by propagating the anatomical labels on a hypergraph. Specifically, we consider not only all voxels within the target image but also voxels across the atlas images as the vertexes in the hypergraph. Each hyperedge encodes a high-order correlation, among a set of vertexes, in different perspectives which incorporate 1) feature affinity within the multi-modal feature space, 2) spatial coherence within target image, and 3) population heuristics from multiple atlases. In addition, our label fusion method further allows those reliable voxels to supervise the label estimation on other difficult-to-label voxels, based on the established hyperedges, until all the target image voxels reach the unanimous labeling result. We evaluate our proposed label fusion method in segmenting hippocampus from T1 and T2 weighted MR images acquired from at 2-week-old, 3-month-old, 6-month-old, 9-month-old, and 12-month-old. Our segmentation results achieves improvement of labeling accuracy over the conventional state-of-the-art label fusion methods, which shows a great potential to facilitate the early infant brain studies.

Brain extraction in pediatric ADC maps, toward characterizing neuro-development in multi-platform and multi-institution clinical images

Apparent Diffusion Coefficient (ADC) maps can be used to characterize myelination and to detect abnormalities in the developing brain. However, given the normal variation in regional ADC with myelination, detection of abnormalities is difficult when based on visual assessment. Quantitative and automated analysis of pediatric ADC maps is thus desired but requires accurate brain extraction as the first step. Currently, most existing brain extraction methods are optimized for structural T1-weighted MR images of fully myelinated brains. Due to differences in age and image contrast, these approaches do not translate well to pediatric ADC maps. To address this problem, we present a multi-atlas brain extraction framework that has 1) specificity: designed and optimized specifically for pediatric ADC maps; 2) generality: applicable to multi-platform and multi-institution data, and to subjects at various neuro-developmental stages across the first 6 years of life; 3) accuracy: highly accurate compared to expert annotations; and 4) consistency: consistently accurate regardless of sources of data and ages of subjects. We show how we achieve these goals, via optimizing major components in a multi-atlas brain extraction framework, and via developing and evaluating new criteria for its atlas ranking component. Moreover, we demonstrate that these goals can be achieved with a fixed set of atlases and a fixed set of parameters, which opens doors for our optimized framework to be used in large-scale and multi-institution neuro-developmental and clinical studies. In a pilot study, we use this framework in a dataset containing scanner-generated ADC maps from 308 pediatric patients collected during the course of routine clinical care. Our framework leads to successful quantifications of the changes in whole-brain volumes and mean ADC values across the first 6 years of life.

Regional growth and atlasing of the developing human brain

Detailed morphometric analysis of the neonatal brain is required to characterise brain development and define neuroimaging biomarkers related to impaired brain growth. Accurate automatic segmentation of neonatal brain MRI is a prerequisite to analyse large datasets. We have previously presented an accurate and robust automatic segmentation technique for parcellating the neonatal brain into multiple cortical and subcortical regions. In this study, we further extend our segmentation method to detect cortical sulci and provide a detailed delineation of the cortical ribbon. These detailed segmentations are used to build a 4-dimensional spatio-temporal structural atlas of the brain for 82 cortical and subcortical structures throughout this developmental period. We employ the algorithm to segment an extensive database of 420 MR images of the developing brain, from 27 to 45 weeks post-menstrual age at imaging. Regional volumetric and cortical surface measurements are derived and used to investigate brain growth and development during this critical period and to assess the impact of immaturity at birth. Whole brain volume, the absolute volume of all structures studied, cortical curvature and cortical surface area increased with increasing age at scan. Relative volumes of cortical grey matter, cerebellum and cerebrospinal fluid increased with age at scan, while relative volumes of white matter, ventricles, brainstem and basal ganglia and thalami decreased. Preterm infants at term had smaller whole brain volumes, reduced regional white matter and cortical and subcortical grey matter volumes, and reduced cortical surface area compared with term born controls, while ventricular volume was greater in the preterm group. Increasing prematurity at birth was associated with a reduction in total and regional white matter, cortical and subcortical grey matter volume, an increase in ventricular volume, and reduced cortical surface area.

•

A novel methodology is proposed for delineating the cortical ribbon.

•

Regional brain growth is assessed in the developing preterm brain.

•

We investigate the effect of prematurity on brain growth and cortical development.

•

A spatio-temporal neonatal atlas is constructed with 82 brain structures.

Parcellation of Infant Surface Atlas Using Developmental Trajectories of Multidimensional Cortical Attributes

Cortical surface atlases, equipped with anatomically and functionally defined parcellations, are of fundamental importance in neuroimaging studies. Typically, parcellations of surface atlases are derived based on the sulcal-gyral landmarks, which are extremely variable across individuals and poorly matched with microstructural and functional boundaries. Cortical developmental trajectories in infants reflect underlying changes of microstructures, which essentially determines the molecular organization and functional principles of the cortex, thus allowing better definition of developmentally, microstructurally, and functionally distinct regions, compared to conventional sulcal-gyral landmarks. Accordingly, a parcellation of infant cortical surface atlas was proposed, based on the developmental trajectories of cortical thickness in infants, revealing regional patterning of cortical growth. However, cortical anatomy is jointly characterized by biologically-distinct, multidimensional cortical attributes, i.e., cortical thickness, surface area, and local gyrification, each with its distinct genetic underpinning, cellular mechanism, and developmental trajectories. To date, the parcellations based on the development of surface area and local gyrification is still missing. To bridge this critical gap, for the first time, we parcellate an infant cortical surface atlas into distinct regions based solely on developmental trajectories of surface area and local gyrification, respectively. For each cortical attribute, we first nonlinearly fuse the subject-specific similarity matrices of vertices' developmental trajectories of all subjects into a single matrix, which helps better capture common and complementary information of the population than the conventional method of simple averaging of all subjects' matrices. Then, we perform spectral clustering based on this fused matrix. We have applied our method to parcellate an infant surface atlas using the developmental trajectories of surface area and local gyrification from 35 healthy infants, each with up to 7 time points in the first two postnatal years, revealing biologically more meaningful growth patterning than the conventional method.

Construction of 4D high-definition cortical surface atlases of infants: Methods and applications

In neuroimaging, cortical surface atlases play a fundamental role for spatial normalization, analysis, visualization, and comparison of results across individuals and different studies. However, existing cortical surface atlases created for adults are not suitable for infant brains during the first two postnatal years, which is the most dynamic period of postnatal structural and functional development of the highly-folded cerebral cortex. Therefore, spatiotemporal cortical surface atlases for infant brains are highly desired yet still lacking for accurate mapping of early dynamic brain development. To bridge this significant gap, leveraging our infant-dedicated computational pipeline for cortical surface-based analysis and the unique longitudinal infant MRI dataset acquired in our research center, in this paper, we construct the first spatiotemporal (4D) high-definition cortical surface atlases for the dynamic developing infant cortical structures at seven time points, including 1, 3, 6, 9, 12, 18, and 24 months of age, based on 202 serial MRI scans from 35 healthy infants. For this purpose, we develop a novel method to ensure the longitudinal consistency and unbiasedness to any specific subject and age in our 4D infant cortical surface atlases. Specifically, we first compute the within-subject mean cortical folding by unbiased groupwise registration of longitudinal cortical surfaces of each infant. Then we establish longitudinally-consistent and unbiased inter-subject cortical correspondences by groupwise registration of the geometric features of within-subject mean cortical folding across all infants. Our 4D surface atlases capture both longitudinally-consistent dynamic mean shape changes and the individual variability of cortical folding during early brain development. Experimental results on two independent infant MRI datasets show that using our 4D infant cortical surface atlases as templates leads to significantly improved accuracy for spatial normalization of cortical surfaces across infant individuals, in comparison to the infant surface atlases constructed without longitudinal consistency and also the FreeSurfer adult surface atlas. Moreover, based on our 4D infant surface atlases, for the first time, we reveal the spatially-detailed, region-specific correlation patterns of the dynamic cortical developmental trajectories between different cortical regions during early brain development.

Structural growth trajectories and rates of change in the first 3 months of infant brain development

IMPORTANCE

The very early postnatal period witnesses extraordinary rates of growth, but structural brain development in this period has largely not been explored longitudinally. Such assessment may be key in detecting and treating the earliest signs of neurodevelopmental disorders.

OBJECTIVE

To assess structural growth trajectories and rates of change in the whole brain and regions of interest in infants during the first 3 months after birth.

DESIGN, SETTING, AND PARTICIPANTS

Serial structural T1-weighted and/or T2-weighted magnetic resonance images were obtained for 211 time points from 87 healthy term-born or term-equivalent preterm-born infants, aged 2 to 90 days, between October 5, 2007, and June 12, 2013.

MAIN OUTCOMES AND MEASURES

We segmented whole-brain and multiple subcortical regions of interest using a novel application of Bayesian-based methods. We modeled growth and rate of growth trajectories nonparametrically and assessed left-right asymmetries and sexual dimorphisms.

RESULTS

Whole-brain volume at birth was approximately one-third of healthy elderly brain volume, and did not differ significantly between male and female infants (347 388 mm3 and 335 509 mm3, respectively, P = .12). The growth rate was approximately 1%/d, slowing to 0.4%/d by the end of the first 3 months, when the brain reached just more than half of elderly adult brain volume. Overall growth in the first 90 days was 64%. There was a significant age-by-sex effect leading to widening separation in brain sizes with age between male and female infants (with male infants growing faster than females by 200.4 mm3/d, SE = 67.2, P = .003). Longer gestation was associated with larger brain size (2215 mm3/d, SE = 284, P = 4×10-13). The expected brain size of an infant born one week earlier than average was 5% smaller than average; at 90 days it will not have caught up, being 2% smaller than average. The cerebellum grew at the highest rate, more than doubling in 90 days, and the hippocampus grew at the slowest rate, increasing by 47% in 90 days. There was left-right asymmetry in multiple regions of interest, particularly the lateral ventricles where the left was larger than the right by 462 mm3 on average (approximately 5% of lateral ventricular volume at 2 months). We calculated volume-by-age percentile plots for assessing individual development.

CONCLUSIONS AND RELEVANCE

Normative trajectories for early postnatal brain structural development can be determined from magnetic resonance imaging and could be used to improve the detection of deviant maturational patterns indicative of neurodevelopmental disorders.

Deep convolutional neural networks for multi-modality isointense infant brain image segmentation

The segmentation of infant brain tissue images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) plays an important role in studying early brain development in health and disease. In the isointense stage (approximately 6-8 months of age), WM and GM exhibit similar levels of intensity in both T1 and T2 MR images, making the tissue segmentation very challenging. Only a small number of existing methods have been designed for tissue segmentation in this isointense stage; however, they only used a single T1 or T2 images, or the combination of T1 and T2 images. In this paper, we propose to use deep convolutional neural networks (CNNs) for segmenting isointense stage brain tissues using multi-modality MR images. CNNs are a type of deep models in which trainable filters and local neighborhood pooling operations are applied alternatingly on the raw input images, resulting in a hierarchy of increasingly complex features. Specifically, we used multi-modality information from T1, T2, and fractional anisotropy (FA) images as inputs and then generated the segmentation maps as outputs. The multiple intermediate layers applied convolution, pooling, normalization, and other operations to capture the highly nonlinear mappings between inputs and outputs. We compared the performance of our approach with that of the commonly used segmentation methods on a set of manually segmented isointense stage brain images. Results showed that our proposed model significantly outperformed prior methods on infant brain tissue segmentation. In addition, our results indicated that integration of multi-modality images led to significant performance improvement.

LINKS: learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images

Segmentation of infant brain MR images is challenging due to insufficient image quality, severe partial volume effect, and ongoing maturation and myelination processes. In the first year of life, the image contrast between white and gray matters of the infant brain undergoes dramatic changes. In particular, the image contrast is inverted around 6-8months of age, and the white and gray matter tissues are isointense in both T1- and T2-weighted MR images and thus exhibit the extremely low tissue contrast, which poses significant challenges for automated segmentation. Most previous studies used multi-atlas label fusion strategy, which has the limitation of equally treating the different available image modalities and is often computationally expensive. To cope with these limitations, in this paper, we propose a novel learning-based multi-source integration framework for segmentation of infant brain images. Specifically, we employ the random forest technique to effectively integrate features from multi-source images together for tissue segmentation. Here, the multi-source images include initially only the multi-modality (T1, T2 and FA) images and later also the iteratively estimated and refined tissue probability maps of gray matter, white matter, and cerebrospinal fluid. Experimental results on 119 infants show that the proposed method achieves better performance than other state-of-the-art automated segmentation methods. Further validation was performed on the MICCAI grand challenge and the proposed method was ranked top among all competing methods. Moreover, to alleviate the possible anatomical errors, our method can also be combined with an anatomically-constrained multi-atlas labeling approach for further improving the segmentation accuracy.

Simultaneous and consistent labeling of longitudinal dynamic developing cortical surfaces in infants

The human cerebral cortex develops extremely dynamically in the first 2years of life. Accurate and consistent parcellation of longitudinal dynamic cortical surfaces during this critical stage is essential to understand the early development of cortical structure and function in both normal and high-risk infant brains. However, directly applying the existing methods developed for the cross-sectional studies often generates longitudinally-inconsistent results, thus leading to inaccurate measurements of the cortex development. In this paper, we propose a new method for accurate, consistent, and simultaneous labeling of longitudinal cortical surfaces in the serial infant brain MR images. The proposed method is explicitly formulated as a minimization problem with an energy function that includes a data fitting term, a spatial smoothness term, and a temporal consistency term. Specifically, inspired by multi-atlas based label fusion, the data fitting term is designed to integrate the contributions from multi-atlas surfaces adaptively, according to the similarities of their local cortical folding with that of the subject cortical surface. The spatial smoothness term is then designed to adaptively encourage label smoothness based on the local cortical folding geometries, i.e., allowing label discontinuity at sulcal bottoms (which often are the boundaries of cytoarchitecturally and functionally distinct regions). The temporal consistency term is to adaptively encourage the label consistency among the temporally-corresponding vertices, based on their similarity of local cortical folding. Finally, the entire energy function is efficiently minimized by a graph cuts method. The proposed method has been applied to the parcellation of longitudinal cortical surfaces of 13 healthy infants, each with 6 serial MRI scans acquired at 0, 3, 6, 9, 12 and 18months of age. Qualitative and quantitative evaluations demonstrated both accuracy and longitudinal consistency of the proposed method. By using our method, for the first time, we reveal several hitherto unseen properties of the dynamic and regionally heterogeneous development of the cortical surface area in the first 18months of life.

Constructing 4D infant cortical surface atlases based on dynamic developmental trajectories of the cortex

Cortical surface atlases play an increasingly important role for analysis, visualization, and comparison of results across different neuroimaging studies. As the first two years of life is the most dynamic period of postnatal structural and functional development of the highly-folded cerebral cortex, longitudinal (4D) cortical surface atlases for the infant brains during this period is highly desired yet still lacking for early brain development studies. In this paper, we construct the first longitudinal (4D) cortical surface atlases for the dynamic developing infant cortical structures at 1, 3, 6, 9, 12, 18 and 24 months of age, based on 202 serial MRI scans from 35 healthy infants. To ensure longitudinal consistency and unbiasedness of the 4D infant cortical surface atlases, we first compute the within-subject mean cortical folding geometries by groupwise registration of longitudinal surfaces of each infant. Then we establish intersubject cortical correspondences by groupwise registration of the within-subject mean cortical folding geometries of all infants. More importantly, for the first time, we further parcellate the 4D infant surface atlases into developmentally and functionally distinctive regions based solely on the dynamic developmental trajectories of the cortical thickness, by using the spectral clustering method. Specifically, to deal with the problem that each infant has different number of scans, we first compute the within-subject affinity matrix of vertices' cortical thickness trajectories of each infant, and then we use the averaged affinity matrix of all infants for parcellation. Our constructed 4D infant cortical surface atlases with developmental trajectories based parcellation will greatly facilitate the surface-based analysis of dynamic brain development in infants.

Automatic whole brain MRI segmentation of the developing neonatal brain

Magnetic resonance (MR) imaging is increasingly being used to assess brain growth and development in infants. Such studies are often based on quantitative analysis of anatomical segmentations of brain MR images. However, the large changes in brain shape and appearance associated with development, the lower signal to noise ratio and partial volume effects in the neonatal brain present challenges for automatic segmentation of neonatal MR imaging data. In this study, we propose a framework for accurate intensity-based segmentation of the developing neonatal brain, from the early preterm period to term-equivalent age, into 50 brain regions. We present a novel segmentation algorithm that models the intensities across the whole brain by introducing a structural hierarchy and anatomical constraints. The proposed method is compared to standard atlas-based techniques and improves label overlaps with respect to manual reference segmentations. We demonstrate that the proposed technique achieves highly accurate results and is very robust across a wide range of gestational ages, from 24 weeks gestational age to term-equivalent age.

Compounding local invariant features and global deformable geometry for medical image registration

Using deformable models to register medical images can result in problems of initialization of deformable models and robustness and accuracy of matching of inter-subject anatomical variability. To tackle these problems, a novel model is proposed in this paper by compounding local invariant features and global deformable geometry. This model has four steps. First, a set of highly-repeatable and highly-robust local invariant features, called Key Features Model (KFM), are extracted by an effective matching strategy. Second, local features can be matched more accurately through the KFM for the purpose of initializing a global deformable model. Third, the positional relationship between the KFM and the global deformable model can be used to precisely pinpoint all landmarks after initialization. And fourth, the final pose of the global deformable model is determined by an iterative process with a lower time cost. Through the practical experiments, the paper finds three important conclusions. First, it proves that the KFM can detect the matching feature points well. Second, the precision of landmark locations adjusted by the modeled relationship between KFM and global deformable model is greatly improved. Third, regarding the fitting accuracy and efficiency, by observation from the practical experiments, it is found that the proposed method can improve % of the fitting accuracy and reduce around 50% of the computational time compared with state-of-the-art methods.

A JOINT FRAMEWORK FOR 4D SEGMENTATION AND ESTIMATION OF SMOOTH TEMPORAL APPEARANCE CHANGES

Medical imaging studies increasingly use longitudinal images of individual subjects in order to follow-up changes due to development, degeneration, disease progression or efficacy of therapeutic intervention. Repeated image data of individuals are highly correlated, and the strong causality of information over time lead to the development of procedures for joint segmentation of the series of scans, called 4D segmentation. A main aim was improved consistency of quantitative analysis, most often solved via patient-specific atlases. Challenging open problems are contrast changes and occurance of subclasses within tissue as observed in multimodal MRI of infant development, neurodegeneration and disease. This paper proposes a new 4D segmentation framework that enforces continuous dynamic changes of tissue contrast patterns over time as observed in such data. Moreover, our model includes the capability to segment different contrast patterns within a specific tissue class, for example as seen in myelinated and unmyelinated white matter regions in early brain development. Proof of concept is shown with validation on synthetic image data and with 4D segmentation of longitudinal, multimodal pediatric MRI taken at 6, 12 and 24 months of age, but the methodology is generic w.r.t. different application domains using serial imaging.

Integration of sparse multi-modality representation and anatomical constraint for isointense infant brain MR image segmentation

Segmentation of infant brain MR images is challenging due to poor spatial resolution, severe partial volume effect, and the ongoing maturation and myelination processes. During the first year of life, the brain image contrast between white and gray matters undergoes dramatic changes. In particular, the image contrast inverses around 6-8months of age, where the white and gray matter tissues are isointense in T1 and T2 weighted images and hence exhibit the extremely low tissue contrast, posing significant challenges for automated segmentation. In this paper, we propose a general framework that adopts sparse representation to fuse the multi-modality image information and further incorporate the anatomical constraints for brain tissue segmentation. Specifically, we first derive an initial segmentation from a library of aligned images with ground-truth segmentations by using sparse representation in a patch-based fashion for the multi-modality T1, T2 and FA images. The segmentation result is further iteratively refined by integration of the anatomical constraint. The proposed method was evaluated on 22 infant brain MR images acquired at around 6months of age by using a leave-one-out cross-validation, as well as other 10 unseen testing subjects. Our method achieved a high accuracy for the Dice ratios that measure the volume overlap between automated and manual segmentations, i.e., 0.889±0.008 for white matter and 0.870±0.006 for gray matter.

Segmentation of neuronal structures using SARSA (λ)-based boundary amendment with reinforced gradient-descent curve shape fitting

The segmentation of structures in electron microscopy (EM) images is very important for neurobiological research. The low resolution neuronal EM images contain noise and generally few features are available for segmentation, therefore application of the conventional approaches to identify the neuron structure from EM images is not successful. We therefore present a multi-scale fused structure boundary detection algorithm in this study. In the algorithm, we generate an EM image Gaussian pyramid first, then at each level of the pyramid, we utilize Laplacian of Gaussian function (LoG) to attain structure boundary, we finally assemble the detected boundaries by using fusion algorithm to attain a combined neuron structure image. Since the obtained neuron structures usually have gaps, we put forward a reinforcement learning-based boundary amendment method to connect the gaps in the detected boundaries. We use a SARSA (λ)-based curve traveling and amendment approach derived from reinforcement learning to repair the incomplete curves. Using this algorithm, a moving point starts from one end of the incomplete curve and walks through the image where the decisions are supervised by the approximated curve model, with the aim of minimizing the connection cost until the gap is closed. Our approach provided stable and efficient structure segmentation. The test results using 30 EM images from ISBI 2012 indicated that both of our approaches, i.e., with or without boundary amendment, performed better than six conventional boundary detection approaches. In particular, after amendment, the Rand error and warping error, which are the most important performance measurements during structure segmentation, were reduced to very low values. The comparison with the benchmark method of ISBI 2012 and the recent developed methods also indicates that our method performs better for the accurate identification of substructures in EM images and therefore useful for the identification of imaging features related to brain diseases.

Integration of sparse multi-modality representation and geometrical constraint for isointense infant brain segmentation

Segmentation of infant brain MR images is challenging due to insufficient image quality, severe partial volume effect, and ongoing maturation and myelination process. During the first year of life, the signal contrast between white matter (WM) and gray matter (GM) in MR images undergoes inverse changes. In particular, the inversion of WM/GM signal contrast appears around 6-8 months of age, where brain tissues appear isointense and hence exhibit extremely low tissue contrast, posing significant challenges for automated segmentation. In this paper, we propose a novel segmentation method to address the above-mentioned challenge based on the sparse representation of the complementary tissue distribution information from T1, T2 and diffusion-weighted images. Specifically, we first derive an initial segmentation from a library of aligned multi-modality images with ground-truth segmentations by using sparse representation in a patch-based fashion. The segmentation is further refined by the integration of the geometrical constraint information. The proposed method was evaluated on 22 6-month-old training subjects using leave-one-out cross-validation, as well as 10 additional infant testing subjects, showing superior results in comparison to other state-of-the-art methods.

Measuring the dynamic longitudinal cortex development in infants by reconstruction of temporally consistent cortical surfaces

Quantitative measurement of the dynamic longitudinal cortex development during early postnatal stages is of great importance to understand the early cortical structural and functional development. Conventional methods usually reconstruct the cortical surfaces of longitudinal images from the same subject independently, which often generate longitudinally-inconsistent cortical surfaces and thus lead to inaccurate measurement of cortical changes, especially for vertex-wise mapping of cortical development. This paper aims to address this problem by presenting a method to reconstruct temporally-consistent cortical surfaces from longitudinal infant brain MR images, for accurate and consistent measurement of the dynamic cortex development in infants. Specifically, the longitudinal development of the inner cortical surface is first modeled by a deformable growth sheet with elasto-plasticity property to establish longitudinally smooth correspondences of the inner cortical surfaces. Then, the modeled longitudinal inner cortical surfaces are jointly deformed to locate both inner and outer cortical surfaces with a spatial-temporal deformable surface method. The method has been applied to 13 healthy infants, each with 6 serial MR scans acquired at 2 weeks, 3 months, 6 months, 9 months, 12 months and 18 months of age. Experimental results showed that our method with the incorporated longitudinal constraints can reconstruct the longitudinally-dynamic cortical surfaces from serial infant MR images more consistently and accurately than the previously published methods. By using our method, for the first time, we can characterize the vertex-wise longitudinal cortical thickness development trajectory at multiple time points in the first 18 months of life. Specifically, we found the highly age-related and regionally-heterogeneous developmental trajectories of the cortical thickness during this period, with the cortical thickness increased most from 3 to 6 months (16.2%) and least from 9 to 12 months (less than 0.1%). Specifically, the central sulcus only underwent significant increase of cortical thickness from 6 to 9 months and the occipital cortex underwent significant increase from 0 to 9 months, while the frontal, temporal and parietal cortices grew continuously in this first 18 months of life. The adult-like spatial patterns of cortical thickness were generally present at 18 months of age. These results provided detailed insights into the dynamic trajectory of the cortical thickness development in infants.

LONGITUDINAL INTENSITY NORMALIZATION IN THE PRESENCE OF MULTIPLE SCLEROSIS LESIONS

This paper proposes a longitudinal intensity normalization algorithm for T₁-weighted magnetic resonance images of human brains in the presence of multiple sclerosis lesions, aiming towards stable and consistent longitudinal segmentations. Unlike previous longitudinal segmentation methods, we propose a 4D intensity normalization that can be used as a preprocessing step to any segmentation method. The variability in intensities arising from the relapsing and remitting nature of the multiple sclerosis lesions is modeled into an otherwise smooth intensity transform based on first order autoregressive models, resulting in smooth changes in segmentation statistics of normal tissues, while keeping the lesion information unaffected. We validated our method on both simulated and real longitudinal normal subjects and on multiple sclerosis subjects.