Deep learning based segmentation of brain tissue from diffusion MRI

Fine-scale striatal parcellation using diffusion MRI tractography and graph neural networks

The striatum, a crucial part of the basal ganglia, plays a key role in various brain functions through its interactions with the cortex. The complex structural and functional diversity across subdivisions within the striatum highlights the necessity for precise striatal segmentation. In this study, we introduce a novel deep clustering pipeline for automated, fine-scale parcellation of the striatum using diffusion MRI (dMRI) tractography. Initially, we employ a voxel-based probabilistic fiber tractography algorithm combined with a fiber-tract embedding technique to capture intricate dMRI connectivity patterns. To maintain critical inter-voxel relationships, our approach employs Graph Neural Networks (GNNs) to create accurate graph representations of the striatum. This involves encoding probabilistic fiber bundle characteristics as node attributes and refining edge weights using activation functions to enhance the graph's interpretability and accuracy. The methodology incorporates a Transformer-based GraphConv autoencoder in the pre-training phase to extract critical spatial features while minimizing reconstruction loss. In the fine-tuning phase, a novel joint loss mechanism markedly improves segmentation precision and anatomical fidelity. Integration of traditional clustering techniques with multi-head self-attention mechanisms further elevates the accuracy and robustness of our segmentation approach. This methodology provides new insights into the striatum's role in cognition and behavior and offers potential clinical applications for neurological disorders.

Artificial intelligence in medical imaging: From task-specific models to large-scale foundation models

ABSTRACT

Artificial intelligence (AI), particularly deep learning, has demonstrated remarkable performance in medical imaging across a variety of modalities, including X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, positron emission tomography (PET), and pathological imaging. However, most existing state-of-the-art AI techniques are task-specific and focus on a limited range of imaging modalities. Compared to these task-specific models, emerging foundation models represent a significant milestone in AI development. These models can learn generalized representations of medical images and apply them to downstream tasks through zero-shot or few-shot fine-tuning. Foundation models have the potential to address the comprehensive and multifactorial challenges encountered in clinical practice. This article reviews the clinical applications of both task-specific and foundation models, highlighting their differences, complementarities, and clinical relevance. We also examine their future research directions and potential challenges. Unlike the replacement relationship seen between deep learning and traditional machine learning, task-specific and foundation models are complementary, despite inherent differences. While foundation models primarily focus on segmentation and classification, task-specific models are integrated into nearly all medical image analyses. However, with further advancements, foundation models could be applied to other clinical scenarios. In conclusion, all indications suggest that task-specific and foundation models, especially the latter, have the potential to drive breakthroughs in medical imaging, from image processing to clinical workflows.

Towards an informed choice of diffusion MRI image contrasts for cerebellar segmentation

The fine-grained segmentation of cerebellar structures is an essential step towards supplying increasingly accurate anatomically informed analyses, including, for example, white matter diffusion magnetic resonance imaging (MRI) tractography. Cerebellar tissue segmentation is typically performed on structural magnetic resonance imaging data, such as T1-weighted data, while connectivity between segmented regions is mapped using diffusion MRI tractography data. Small deviations in structural to diffusion MRI data co-registration may negatively impact connectivity analyses. Reliable segmentation of brain tissue performed directly on diffusion MRI data helps to circumvent such inaccuracies. Diffusion MRI enables the computation of many image contrasts, including a variety of tissue microstructure maps. While multiple methods have been proposed for the segmentation of cerebellar structures using diffusion MRI, little attention has been paid to the systematic evaluation of the performance of different available input image contrasts for the segmentation task. In this work, we evaluate and compare the segmentation performance of diffusion MRI-derived contrasts on the cerebellar segmentation task. Specifically, we include spherical mean (diffusion-weighted image average) and b0 (non-diffusion-weighted image average) contrasts, local signal parameterization contrasts (diffusion tensor and kurtosis fit maps), and the structural T1-weighted MRI contrast that is most commonly employed for the task. We train a popular deep-learning architecture using a publicly available dataset (HCP-YA), leveraging cerebellar region labels from the atlas-based SUIT cerebellar segmentation pipeline. By training and testing using many diffusion-MRI-derived image inputs, we find that the spherical mean image computed from b=1000 s/mm2 shell data provides stable performance across different metrics and significantly outperforms the tissue microstructure contrasts that are traditionally used in machine learning segmentation methods for diffusion MRI.

DDEvENet: Evidence-based ensemble learning for uncertainty-aware brain parcellation using diffusion MRI

In this study, we developed an Evidential Ensemble Neural Network based on Deep learning and Diffusion MRI, namely DDEvENet, for anatomical brain parcellation. The key innovation of DDEvENet is the design of an evidential deep learning framework to quantify predictive uncertainty at each voxel during a single inference. To do so, we design an evidence-based ensemble learning framework for uncertainty-aware parcellation to leverage the multiple dMRI parameters derived from diffusion MRI. Using DDEvENet, we obtained accurate parcellation and uncertainty estimates across different datasets from healthy and clinical populations and with different imaging acquisitions. The overall network includes five parallel subnetworks, where each is dedicated to learning the FreeSurfer parcellation for a certain diffusion MRI parameter. An evidence-based ensemble methodology is then proposed to fuse the individual outputs. We perform experimental evaluations on large-scale datasets from multiple imaging sources, including high-quality diffusion MRI data from healthy adults and clinically diffusion MRI data from participants with various brain diseases (schizophrenia, bipolar disorder, attention-deficit/hyperactivity disorder, Parkinson's disease, cerebral small vessel disease, and neurosurgical patients with brain tumors). Compared to several state-of-the-art methods, our experimental results demonstrate highly improved parcellation accuracy across the multiple testing datasets despite the differences in dMRI acquisition protocols and health conditions. Furthermore, thanks to the uncertainty estimation, our DDEvENet approach demonstrates a good ability to detect abnormal brain regions in patients with lesions that are consistent with expert-drawn results, enhancing the interpretability and reliability of the segmentation results.

Automatic deep learning segmentation of the hippocampus on high-resolution diffusion magnetic resonance imaging and its application to the healthy lifespan

Diffusion tensor imaging (DTI) can provide unique contrast and insight into microstructural changes with age or disease of the hippocampus, although it is difficult to measure the hippocampus because of its comparatively small size, location, and shape. This has been markedly improved by the advent of a clinically feasible 1-mm isotropic resolution 6-min DTI protocol at 3 T of the hippocampus with limited brain coverage of 20 axial-oblique slices aligned along its long axis. However, manual segmentation is too laborious for large population studies, and it cannot be automatically segmented directly on the diffusion images using traditional T1 or T2 image-based methods because of the limited brain coverage and different contrast. An automatic method is proposed here that segments the hippocampus directly on high-resolution diffusion images based on an extension of well-known deep learning architectures like UNet and UNet++ by including additional dense residual connections. The method was trained on 100 healthy participants with previously performed manual segmentation on the 1-mm DTI, then evaluated on typical healthy participants (n = 53), yielding an excellent voxel overlap with a Dice score of ~ 0.90 with manual segmentation; notably, this was comparable with the inter-rater reliability of manually delineating the hippocampus on diffusion magnetic resonance imaging (MRI) (Dice score of 0.86). This method also generalized to a different DTI protocol with 36% fewer acquisitions. It was further validated by showing similar age trajectories of volumes, fractional anisotropy, and mean diffusivity from manual segmentations in one cohort (n = 153, age 5-74 years) with automatic segmentations from a second cohort without manual segmentations (n = 354, age 5-90 years). Automated high-resolution diffusion MRI segmentation of the hippocampus will facilitate large cohort analyses and, in future research, needs to be evaluated on patient groups.

Diffusion MRI with Machine Learning

Diffusion-weighted magnetic resonance imaging (dMRI) of the brain offers unique capabilities including noninvasive probing of tissue microstructure and structural connectivity. It is widely used for clinical assessment of disease and injury, and for neuroscience research. Analyzing the dMRI data to extract useful information for medical and scientific purposes can be challenging. The dMRI measurements may suffer from strong noise and artifacts, and may exhibit high inter-session and inter-scanner variability in the data, as well as inter-subject heterogeneity in brain structure. Moreover, the relationship between measurements and the phenomena of interest can be highly complex. Recent years have witnessed increasing use of machine learning methods for dMRI analysis. This manuscript aims to assess these efforts, with a focus on methods that have addressed data preprocessing and harmonization, microstructure mapping, tractography, and white matter tract analysis. We study the main findings, strengths, and weaknesses of the existing methods and suggest topics for future research. We find that machine learning may be exceptionally suited to tackle some of the difficult tasks in dMRI analysis. However, for this to happen, several shortcomings of existing methods and critical unresolved issues need to be addressed. There is a pressing need to improve evaluation practices, to increase the availability of rich training datasets and validation benchmarks, as well as model generalizability, reliability, and explainability concerns.

Excessive interstitial free-water in cortical gray matter preceding accelerated volume changes in individuals at clinical high risk for psychosis

Recent studies show that accelerated cortical gray matter (GM) volume reduction seen in anatomical MRI can help distinguish between individuals at clinical high risk (CHR) for psychosis who will develop psychosis and those who will not. This reduction is suggested to represent atypical developmental or degenerative changes accompanying an accumulation of microstructural changes, such as decreased spine density and dendritic arborization. Detecting the microstructural sources of these changes before they accumulate into volume loss is crucial. Our study aimed to detect these microstructural GM alterations using diffusion MRI (dMRI). We tested for baseline and longitudinal group differences in anatomical and dMRI data from 160 individuals at CHR and 96 healthy controls (HC) acquired in a single imaging site. Of the CHR individuals, 33 developed psychosis (CHR-P), while 127 did not (CHR-NP). Among all participants, longitudinal data was available for 45 HCs, 17 CHR-P, and 66 CHR-NP. Eight cortical lobes were examined for GM volume and GM microstructure. A novel dMRI measure, interstitial free water (iFW), was used to quantify GM microstructure by eliminating cerebrospinal fluid contribution. Additionally, we assessed whether these measures differentiated the CHR-P from the CHR-NP. In addition, for completeness, we also investigated changes in cortical thickness and in white matter (WM) microstructure. At baseline the CHR group had significantly higher iFW than HC in the prefrontal, temporal, parietal, and occipital lobes, while volume was reduced only in the temporal lobe. Neither iFW nor volume differentiated between the CHR-P and CHR-NP groups at baseline. However, in many brain areas, the CHR-P group demonstrated significantly accelerated changes (iFW increase and volume reduction) with time than the CHR-NP group. Cortical thickness provided similar results as volume, and there were no significant changes in WM microstructure. Our results demonstrate that microstructural GM changes in individuals at CHR have a wider extent than volumetric changes or microstructural WM changes, and they predate the acceleration of brain changes that occur around psychosis onset. Microstructural GM changes, as reflected by the increased iFW, are thus an early pathology at the prodromal stage of psychosis that may be useful for a better mechanistic understanding of psychosis development.

Deep learning enables accurate brain tissue microstructure analysis based on clinically feasible diffusion magnetic resonance imaging

Diffusion magnetic resonance imaging (dMRI) allows non-invasive assessment of brain tissue microstructure. Current model-based tissue microstructure reconstruction techniques require a large number of diffusion gradients, which is not clinically feasible due to imaging time constraints, and this has limited the use of tissue microstructure information in clinical settings. Recently, approaches based on deep learning (DL) have achieved promising tissue microstructure reconstruction results using clinically feasible dMRI. However, it remains unclear whether the subtle tissue changes associated with disease or age are properly preserved with DL approaches and whether DL reconstruction results can benefit clinical applications. Here, we provide the first evidence that DL approaches to tissue microstructure reconstruction yield reliable brain tissue microstructure analysis based on clinically feasible dMRI scans. Specifically, we reconstructed tissue microstructure from four different brain dMRI datasets with only 12 diffusion gradients, a clinically feasible protocol, and the neurite orientation dispersion and density imaging (NODDI) and spherical mean technique (SMT) models were considered. With these results we show that disease-related and age-dependent alterations of brain tissue were accurately identified. These findings demonstrate that DL tissue microstructure reconstruction can accurately quantify microstructural alterations in the brain based on clinically feasible dMRI.

Assessment of the Depiction of Superficial White Matter Using Ultra-High-Resolution Diffusion MRI

The superficial white matter (SWM) consists of numerous short-range association fibers connecting adjacent and nearby gyri and plays an important role in brain function, development, aging, and various neurological disorders. Diffusion MRI (dMRI) tractography is an advanced imaging technique that enables in vivo mapping of the SWM. However, detailed imaging of the small, highly-curved fibers of the SWM is a challenge for current clinical and research dMRI acquisitions. This work investigates the efficacy of mapping the SWM using in vivo ultra-high-resolution dMRI data. We compare the SWM mapping performance from two dMRI acquisitions: a high-resolution 0.76-mm isotropic acquisition using the generalized slice-dithered enhanced resolution (gSlider) protocol and a lower resolution 1.25-mm isotropic acquisition obtained from the Human Connectome Project Young Adult (HCP-YA) database. Our results demonstrate significant differences in the cortico-cortical anatomical connectivity that is depicted by these two acquisitions. We perform a detailed assessment of the anatomical plausibility of these results with respect to the nonhuman primate (macaque) tract-tracing literature. We find that the high-resolution gSlider dataset is more successful at depicting a large number of true positive anatomical connections in the SWM. An additional cortical coverage analysis demonstrates significantly higher cortical coverage in the gSlider dataset for SWM streamlines under 40 mm in length. Overall, we conclude that the spatial resolution of the dMRI data is one important factor that can significantly affect the mapping of SWM. Considering the relatively long acquisition time, the application of dMRI tractography for SWM mapping in future work should consider the balance of data acquisition efforts and the efficacy of SWM depiction.

Detailed delineation of the fetal brain in diffusion MRI via multi-task learning

Diffusion-weighted MRI is increasingly used to study the normal and abnormal development of fetal brain inutero. Recent studies have shown that dMRI can offer invaluable insights into the neurodevelopmental processes in the fetal stage. However, because of the low data quality and rapid brain development, reliable analysis of fetal dMRI data requires dedicated computational methods that are currently unavailable. The lack of automated methods for fast, accurate, and reproducible data analysis has seriously limited our ability to tap the potential of fetal brain dMRI for medical and scientific applications. In this work, we developed and validated a unified computational framework to (1) segment the brain tissue into white matter, cortical/subcortical gray matter, and cerebrospinal fluid, (2) segment 31 distinct white matter tracts, and (3) parcellate the brain's cortex and delineate the deep gray nuclei and white matter structures into 96 anatomically meaningful regions. We utilized a set of manual, semi-automatic, and automatic approaches to annotate 97 fetal brains. Using these labels, we developed and validated a multi-task deep learning method to perform the three computations. Our evaluations show that the new method can accurately carry out all three tasks, achieving a mean Dice similarity coefficient of 0.865 on tissue segmentation, 0.825 on white matter tract segmentation, and 0.819 on parcellation. The proposed method can greatly advance the field of fetal neuroimaging as it can lead to substantial improvements in fetal brain tractography, tract-specific analysis, and structural connectivity assessment.

Detailed delineation of the fetal brain in diffusion MRI via multi-task learning

Diffusion-weighted MRI is increasingly used to study the normal and abnormal development of fetal brain inutero. Recent studies have shown that dMRI can offer invaluable insights into the neurodevelopmental processes in the fetal stage. However, because of the low data quality and rapid brain development, reliable analysis of fetal dMRI data requires dedicated computational methods that are currently unavailable. The lack of automated methods for fast, accurate, and reproducible data analysis has seriously limited our ability to tap the potential of fetal brain dMRI for medical and scientific applications. In this work, we developed and validated a unified computational framework to (1) segment the brain tissue into white matter, cortical/subcortical gray matter, and cerebrospinal fluid, (2) segment 31 distinct white matter tracts, and (3) parcellate the brain's cortex and delineate the deep gray nuclei and white matter structures into 96 anatomically meaningful regions. We utilized a set of manual, semi-automatic, and automatic approaches to annotate 97 fetal brains. Using these labels, we developed and validated a multi-task deep learning method to perform the three computations. Our evaluations show that the new method can accurately carry out all three tasks, achieving a mean Dice similarity coefficient of 0.865 on tissue segmentation, 0.825 on white matter tract segmentation, and 0.819 on parcellation. The proposed method can greatly advance the field of fetal neuroimaging as it can lead to substantial improvements in fetal brain tractography, tract-specific analysis, and structural connectivity assessment.

Anatomically constrained tractography of the fetal brain

Diffusion-weighted Magnetic Resonance Imaging (dMRI) is increasingly used to study the fetal brain in utero. An important computation enabled by dMRI is streamline tractography, which has unique applications such as tract-specific analysis of the brain white matter and structural connectivity assessment. However, due to the low fetal dMRI data quality and the challenging nature of tractography, existing methods tend to produce highly inaccurate results. They generate many false streamlines while failing to reconstruct the streamlines that constitute the major white matter tracts. In this paper, we advocate for anatomically constrained tractography based on an accurate segmentation of the fetal brain tissue directly in the dMRI space. We develop a deep learning method to compute the segmentation automatically. Experiments on independent test data show that this method can accurately segment the fetal brain tissue and drastically improve the tractography results. It enables the reconstruction of highly curved tracts such as optic radiations. Importantly, our method infers the tissue segmentation and streamline propagation direction from a diffusion tensor fit to the dMRI data, making it applicable to routine fetal dMRI scans. The proposed method can facilitate the study of fetal brain white matter tracts with dMRI.

Impact of Systolic Blood Viscosity on Deep White Matter Hyperintensities in Patients With Acute Ischemic Stroke

BACKGROUND

Elevated blood viscosity (BV), a critical determinant in blood rheology, is a contributing factor in cerebrovascular diseases. The specific influence of BV on small vessel disease burden remains unexplored. This study aims to examine the relationship between BV and regional white matter hyperintensity (WMH) volume in patients with acute ischemic stroke.

METHODS AND RESULTS

We enrolled a cohort of 302 patients with acute ischemic stroke or transient ischemic attack who were admitted to a hospital within 7 days of symptom onset in this study. We measured whole BV using a scanning capillary-tube viscometer and categorized systolic blood viscosity into 3 groups based on established references. We quantified and normalized WMH volumes using automated localization and segmentation software by NEUROPHET Inc. We performed multivariable logistic regression analysis to assess the correlation between systolic BV and WMH. The mean subject age was 66.7±13.4 years, and 38.7% (n=117) of the participants were female. Among a total of 302 patients, patients with higher deep WMH volume (T3) were typically older and had an atrial fibrillation, strokes of cardioembolic or undetermined cause, elevated levels of C-reactive protein, diastolic blood viscosity and systolic BV. A multivariable adjustment revealed a significant association between high systolic BV and increased deep-WMH volume (odds ratio [OR], 2.636 [95% CI, 1.225-5.673]).

CONCLUSIONS

Elevated systolic BV is more likely to be associated with deep WMH volume in patients with acute ischemic stroke or transient ischemic attack. These findings reveal novel therapeutic strategies focusing on blood rheology to enhance cerebral microcirculation in stroke management.

One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label

Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.

The Mexican dataset of a repetitive transcranial magnetic stimulation clinical trial on cocaine use disorder patients: SUDMEX TMS

Cocaine use disorder (CUD) is a global health problem with severe consequences, leading to behavioral, cognitive, and neurobiological disturbances. While consensus on treatments is still ongoing, repetitive transcranial magnetic stimulation (rTMS) has emerged as a promising approach for medication-resistant disorders, including substance use disorders. In this context, here we present the SUDMEX-TMS, a Mexican dataset from an rTMS clinical trial involving CUD patients. This longitudinal dataset comprises 54 CUD patients (including 8 females) with data collected at five time points: baseline (T0), two weeks (T1), three months (T2), six months (T3) follow-up, and twelve months (T4) follow-up. The clinical rTMS treatment followed a double-blinded randomized clinical trial design (n = 24 sham/30 active) for 2 weeks, followed by an open-label phase. The dataset includes demographic, clinical, and cognitive measures, as well as magnetic resonance imaging (MRI) data collected at all time points, encompassing structural (T1-weighted), functional (resting-state fMRI), and multishell diffusion-weighted (DWI-HARDI) sequences. This dataset offers the opportunity to investigate the impact of rTMS on CUD participants, considering clinical, cognitive, and multimodal MRI metrics in a longitudinal framework.

DDParcel: Deep Learning Anatomical Brain Parcellation From Diffusion MRI

Parcellation of anatomically segregated cortical and subcortical brain regions is required in diffusion MRI (dMRI) analysis for region-specific quantification and better anatomical specificity of tractography. Most current dMRI parcellation approaches compute the parcellation from anatomical MRI (T1- or T2-weighted) data, using tools such as FreeSurfer or CAT12, and then register it to the diffusion space. However, the registration is challenging due to image distortions and low resolution of dMRI data, often resulting in mislabeling in the derived brain parcellation. Furthermore, these approaches are not applicable when anatomical MRI data is unavailable. As an alternative we developed the Deep Diffusion Parcellation (DDParcel), a deep learning method for fast and accurate parcellation of brain anatomical regions directly from dMRI data. The input to DDParcel are dMRI parameter maps and the output are labels for 101 anatomical regions corresponding to the FreeSurfer Desikan-Killiany (DK) parcellation. A multi-level fusion network leverages complementary information in the different input maps, at three network levels: input, intermediate layer, and output. DDParcel learns the registration of diffusion features to anatomical MRI from the high-quality Human Connectome Project data. Then, to predict brain parcellation for a new subject, the DDParcel network no longer requires anatomical MRI data but only the dMRI data. Comparing DDParcel's parcellation with T1w-based parcellation shows higher test-retest reproducibility and a higher regional homogeneity, while requiring much less computational time. Generalizability is demonstrated on a range of populations and dMRI acquisition protocols. Utility of DDParcel's parcellation is demonstrated on tractography analysis for fiber tract identification.

DiMANI: diffusion MRI for anatomical nuclei imaging-Application for the direct visualization of thalamic subnuclei

The thalamus is a centrally located and heterogeneous brain structure that plays a critical role in various sensory, motor, and cognitive processes. However, visualizing the individual subnuclei of the thalamus using conventional MRI techniques is challenging. This difficulty has posed obstacles in targeting specific subnuclei for clinical interventions such as deep brain stimulation (DBS). In this paper, we present DiMANI, a novel method for directly visualizing the thalamic subnuclei using diffusion MRI (dMRI). The DiMANI contrast is computed by averaging, voxelwise, diffusion-weighted volumes enabling the direct distinction of thalamic subnuclei in individuals. We evaluated the reproducibility of DiMANI through multiple approaches. First, we utilized a unique dataset comprising 8 scans of a single participant collected over a 3-year period. Secondly, we quantitatively assessed manual segmentations of thalamic subnuclei for both intra-rater and inter-rater reliability. Thirdly, we qualitatively correlated DiMANI imaging data from several patients with Essential Tremor with the localization of implanted DBS electrodes and clinical observations. Lastly, we demonstrated that DiMANI can provide similar features at 3T and 7T MRI, using varying numbers of diffusion directions. Our results establish that DiMANI is a reproducible and clinically relevant method to directly visualize thalamic subnuclei. This has significant implications for the development of new DBS targets and the optimization of DBS therapy.

Deep learning model for differentiating acute myeloid and lymphoblastic leukemia in peripheral blood cell images via myeloblast and lymphoblast classification

Objective

Acute leukemia (AL) is a life-threatening malignant disease that occurs in the bone marrow and blood, and is classified as either acute myeloid leukemia (AML) or acute lymphoblastic leukemia (ALL). Diagnosing AL warrants testing methods, such as flow cytometry, which require trained professionals, time, and money. We aimed to develop a model that can classify peripheral blood images of 12 cell types, including pathological cells associated with AL, using artificial intelligence.

Methods

We acquired 42,386 single-cell images of peripheral blood slides from 282 patients (82 with AML, 40 with ALL, and 160 with immature granulocytes).

Results

The performance of EfficientNet-V2 (B2) using the original image size exhibited the greatest accuracy (accuracy, 0.8779; precision, 0.7221; recall, 0.7225; and F1 score, 0.7210). The next-best accuracy was achieved by EfficientNet-V1 (B1), with a 256 × 256 pixels image. F1 score was the greatest for EfficientNet-V1 (B1) with the original image size. EfficientNet-V1 (B1) and EfficientNet-V2 (B2) were used to develop an ensemble model, and the accuracy (0.8858) and F1 score (0.7361) were improved. The classification performance of the developed ensemble model for the 12 cell types was good, with an area under the receiver operating characteristic curve above 0.9, and F1 scores for myeloblasts and lymphoblasts of 0.8873 and 0.8006, respectively.

Conclusions

The performance of the developed ensemble model for the 12 cell classifications was satisfactory, particularly for myeloblasts and lymphoblasts. We believe that the application of our model will benefit healthcare settings where the rapid and accurate diagnosis of AL is difficult.

Efficient semi-supervised semantic segmentation of electron microscopy cancer images with sparse annotations

Electron microscopy (EM) enables imaging at a resolution of nanometers and can shed light on how cancer evolves to develop resistance to therapy. Acquiring these images has become a routine task.However, analyzing them is now a bottleneck, as manual structure identification is very time-consuming and can take up to several months for a single sample. Deep learning approaches offer a suitable solution to speed up the analysis. In this work, we present a study of several state-of-the-art deep learning models for the task of segmenting nuclei and nucleoli in volumes from tumor biopsies. We compared previous results obtained with the ResUNet architecture to the more recent UNet++, FracTALResNet, SenFormer, and CEECNet models. In addition, we explored the utilization of unlabeled images through semi-supervised learning with Cross Pseudo Supervision. We have trained and evaluated all of the models on sparse manual labels from three fully annotated in-house datasets that we have made available on demand, demonstrating improvements in terms of 3D Dice score. From the analysis of these results, we drew conclusions on the relative gains of using more complex models, and semi-supervised learning as well as the next steps for the mitigation of the manual segmentation bottleneck.

Efficient semi-supervised semantic segmentation of electron microscopy cancer images with sparse annotations

Electron microscopy (EM) enables imaging at nanometer resolution and can shed light on how cancer evolves to develop resistance to therapy. Acquiring these images has become a routine task; however, analyzing them is now the bottleneck, as manual structure identification is very time-consuming and can take up to several months for a single sample. Deep learning approaches offer a suitable solution to speed up the analysis. In this work, we present a study of several state-of-the-art deep learning models for the task of segmenting nuclei and nucleoli in volumes from tumor biopsies. We compared previous results obtained with the ResUNet architecture to the more recent UNet++, FracTALResNet, SenFormer, and CEECNet models. In addition, we explored the utilization of unlabeled images through semi-supervised learning with Cross Pseudo Supervision. We have trained and evaluated all of the models on sparse manual labels from three fully annotated in-house datasets that we have made available on demand, demonstrating improvements in terms of 3D Dice score. From the analysis of these results, we drew conclusions on the relative gains of using more complex models, semi-supervised learning as well as next steps for the mitigation of the manual segmentation bottleneck.

Microstructural Cortical Gray Matter Changes Preceding Accelerated Volume Changes in Individuals at Clinical High Risk for Psychosis

Recent studies show that accelerated cortical gray matter (GM) volume reduction seen in anatomical MRI can help distinguish between individuals at clinical high risk (CHR) for psychosis who will develop psychosis and those who will not. This reduction is thought to result from an accumulation of microstructural changes, such as decreased spine density and dendritic arborization. Detecting the microstructural sources of these changes before they accumulate is crucial, as volume reduction likely indicates an underlying neurodegenerative process. Our study aimed to detect these microstructural GM alterations using diffusion MRI (dMRI). We tested for baseline and longitudinal group differences in anatomical and dMRI data from 160 individuals at CHR and 96 healthy controls (HC) acquired in a single imaging site. Eight cortical lobes were examined for GM volume and GM microstructure. A novel dMRI measure, interstitial free water (iFW), was used to quantify GM microstructure by eliminating cerebrospinal fluid contribution. Additionally, we assessed whether these measures differentiated the 33 individuals at CHR who developed psychosis (CHR-P) from the 127 individuals at CHR who did not (CHR-NP). At baseline the CHR group had significantly higher iFW than HC in the prefrontal, temporal, parietal, and occipital lobes, while volume was reduced only in the temporal lobe. Neither iFW nor volume differentiated between the CHR-P and CHR-NP groups at baseline. However, in most brain areas, the CHR-P group demonstrated significantly accelerated iFW increase and volume reduction with time than the CHR-NP group. Our results demonstrate that microstructural GM changes in individuals at CHR have a wider extent than volumetric changes and they predate the acceleration of brain changes that occur around psychosis onset. Microstructural GM changes are thus an early pathology at the prodromal stage of psychosis that may be useful for early detection and a better mechanistic understanding of psychosis development.

Validation of deep learning techniques for quality augmentation in diffusion MRI for clinical studies

The objective of this study is to evaluate the efficacy of deep learning (DL) techniques in improving the quality of diffusion MRI (dMRI) data in clinical applications. The study aims to determine whether the use of artificial intelligence (AI) methods in medical images may result in the loss of critical clinical information and/or the appearance of false information. To assess this, the focus was on the angular resolution of dMRI and a clinical trial was conducted on migraine, specifically between episodic and chronic migraine patients. The number of gradient directions had an impact on white matter analysis results, with statistically significant differences between groups being drastically reduced when using 21 gradient directions instead of the original 61. Fourteen teams from different institutions were tasked to use DL to enhance three diffusion metrics (FA, AD and MD) calculated from data acquired with 21 gradient directions and a b-value of 1000 s/mm2. The goal was to produce results that were comparable to those calculated from 61 gradient directions. The results were evaluated using both standard image quality metrics and Tract-Based Spatial Statistics (TBSS) to compare episodic and chronic migraine patients. The study results suggest that while most DL techniques improved the ability to detect statistical differences between groups, they also led to an increase in false positive. The results showed that there was a constant growth rate of false positives linearly proportional to the new true positives, which highlights the risk of generalization of AI-based tasks when assessing diverse clinical cohorts and training using data from a single group. The methods also showed divergent performance when replicating the original distribution of the data and some exhibited significant bias. In conclusion, extreme caution should be exercised when using AI methods for harmonization or synthesis in clinical studies when processing heterogeneous data in clinical studies, as important information may be altered, even when global metrics such as structural similarity or peak signal-to-noise ratio appear to suggest otherwise.

Diffusion MRI data analysis assisted by deep learning synthesized anatomical images (DeepAnat)

Diffusion MRI is a useful neuroimaging tool for non-invasive mapping of human brain microstructure and structural connections. The analysis of diffusion MRI data often requires brain segmentation, including volumetric segmentation and cerebral cortical surfaces, from additional high-resolution T1-weighted (T1w) anatomical MRI data, which may be unacquired, corrupted by subject motion or hardware failure, or cannot be accurately co-registered to the diffusion data that are not corrected for susceptibility-induced geometric distortion. To address these challenges, this study proposes to synthesize high-quality T1w anatomical images directly from diffusion data using convolutional neural networks (CNNs) (entitled "DeepAnat"), including a U-Net and a hybrid generative adversarial network (GAN), and perform brain segmentation on synthesized T1w images or assist the co-registration using synthesized T1w images. The quantitative and systematic evaluations using data of 60 young subjects provided by the Human Connectome Project (HCP) show that the synthesized T1w images and results for brain segmentation and comprehensive diffusion analysis tasks are highly similar to those from native T1w data. The brain segmentation accuracy is slightly higher for the U-Net than the GAN. The efficacy of DeepAnat is further validated on a larger dataset of 300 more elderly subjects provided by the UK Biobank. Moreover, the U-Nets trained and validated on the HCP and UK Biobank data are shown to be highly generalizable to the diffusion data from Massachusetts General Hospital Connectome Diffusion Microstructure Dataset (MGH CDMD) acquired with different hardware systems and imaging protocols and therefore can be used directly without retraining or with fine-tuning for further improved performance. Finally, it is quantitatively demonstrated that the alignment between native T1w images and diffusion images uncorrected for geometric distortion assisted by synthesized T1w images substantially improves upon that by directly co-registering the diffusion and T1w images using the data of 20 subjects from MGH CDMD. In summary, our study demonstrates the benefits and practical feasibility of DeepAnat for assisting various diffusion MRI data analyses and supports its use in neuroscientific applications.

EG-Unet: Edge-Guided cascaded networks for automated frontal brain segmentation in MR images

Accurate segmentation of frontal lobe areas on magnetic resonance imaging (MRI) can assist in diagnosing and managing idiopathic normal-pressure hydrocephalus. However, frontal lobe segmentation is challenging due to the complexity of the degree and shape of damage and the ambiguity of the boundaries of frontal lobe sites. Therefore, to extract the rich edge information and feature representation of the frontal lobe, this paper designs an edge guidance (EG) module to enhance the representation of edge features. Accordingly, an edge-guided cascade network framework (EG-Net) is proposed to segment frontal lobe parts automatically. Two-dimensional MRI slice images are fed into the edge generation and segmentation networks. First, the edge generation network extracts the edge information from the input image. Then, the edge information is sent to the EG module to generate an edge attention map for feature representation enhancement. Meanwhile, multi-scale attentional convolution (MSA) is utilized in the feature coding stage of the segmentation network to obtain feature responses from different perceptual fields in the coding stage and enrich the spatial context information. Besides, the feature fusion module is employed to selectively aggregate the multi-scale features in the coding stage with the edge features output by the EG module. Finally, the two components are fused, and a decoder recovers the spatial information to generate the final prediction results. An extensive quantitative comparison is performed on a publicly available brain MRI dataset (MICCAI 2012) to evaluate the effectiveness of the proposed algorithm. The experimental results indicate that the proposed method achieves an average DICE score of 95.77% compared to some advanced methods, which is 4.96% better than the classical U-Net. The results demonstrate the potential of the proposed EG-Net in improving the accuracy of frontal edge pixel classification through edge guidance.

Value of deep learning models based on ultrasonic dynamic videos for distinguishing thyroid nodules

Objective

This study was designed to distinguish benign and malignant thyroid nodules by using deep learning(DL) models based on ultrasound dynamic videos.

Methods

Ultrasound dynamic videos of 1018 thyroid nodules were retrospectively collected from 657 patients in Zhejiang Cancer Hospital from January 2020 to December 2020 for the tests with 5 DL models.

Results

In the internal test set, the area under the receiver operating characteristic curve (AUROC) was 0.929(95% CI: 0.888,0.970) for the best-performing model LSTM Two radiologists interpreted the dynamic video with AUROC values of 0.760 (95% CI: 0.653, 0.867) and 0.815 (95% CI: 0.778, 0.853). In the external test set, the best-performing DL model had AUROC values of 0.896(95% CI: 0.847,0.945), and two ultrasound radiologist had AUROC values of 0.754 (95% CI: 0.649,0.850) and 0.833 (95% CI: 0.797,0.869).

Conclusion

This study demonstrates that the DL model based on ultrasound dynamic videos performs better than the ultrasound radiologists in distinguishing thyroid nodules.

Deep learning model for automatic segmentation of lungs and pulmonary metastasis in small animal MR images

Lungs are the most frequent site of metastases growth. The amount and size of pulmonary metastases acquired from MRI imaging data are the important criteria to assess the efficacy of new drugs in preclinical models. While efficient solutions both for MR imaging and the downstream automatic segmentation have been proposed for human patients, both MRI lung imaging and segmentation in preclinical animal models remains challenging due to the physiological motion (respiratory and cardiac movements), to the low amount of protons in this organ and to the particular challenge of precise segmentation of metastases. As a consequence post-mortem analysis is currently required to obtain information on metastatic volume. In this work, we have developed a complete methodological pipeline for automated analysis of lungs and metastases in mice, consisting of an MR sequence for image acquisition and a deep learning method for automatic segmentation of both lungs and metastases. On one hand, we optimized an MR sequence for mouse lung imaging with high contrast for high detection sensitivity. On the other hand we developed DeepMeta, a multiclass U-Net 3+ deep learning model to automatically segment the images. To assess if the proposed deep learning pipeline is able to provide an accurate segmentation of both lungs and pulmonary metastases, we have longitudinally imaged mice with fast- and slow-growing metastasis. Fifty-five balb/c mice were injected with two different derivatives of renal carcinoma cells. Mice were imaged with a SG-bSSFP (self-gated balanced steady state free precession) sequence at different time points after the injection of cancer cells. Both lung and metastases segmentations were manually performed by experts. DeepMeta was trained to perform lung and metastases segmentation based on the resulting ground truth annotations. Volumes of lungs and of pulmonary metastases as well as the number of metastases per mouse were measured on a separate test dataset of MR images. Thanks to the SG method, the 3D bSSFP images of lungs were artifact-free, enabling the downstream detection and serial follow-up of metastases. Moreover, both lungs and metastases segmentation was accurately performed by DeepMeta as soon as they reached the volume of ∼ 0.02 m m 3 . Thus we were able to distinguish two groups of mice in terms of number and volume of pulmonary metastases as well as in terms of the slow versus fast patterns of growth of metastases. We have shown that our methodology combining SG-bSSFP with deep learning, enables processing of the whole animal lungs and is thus a viable alternative to histology alone.

DORIS: A diffusion MRI-based 10 tissue class deep learning segmentation algorithm tailored to improve anatomically-constrained tractography

Modern tractography algorithms such as anatomically-constrained tractography (ACT) are based on segmentation maps of white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). These maps are generally estimated from a T1-weighted (T1w) image and then registered in diffusion weighted images (DWI) space. Registration of T1w to diffusion space and partial volume estimation are challenging and rarely voxel-perfect. Diffusion-based segmentation would, thus, potentially allow not to have higher quality anatomical priors injected in the tractography process. On the other hand, even if FA-based tractography is possible without T1 registration, the literature shows that this technique suffers from multiple issues such as holes in the tracking mask and a high proportion of generated broken and anatomically implausible streamlines. Therefore, there is an important need for a tissue segmentation algorithm that works directly in the native diffusion space. We propose DORIS, a DWI-based deep learning segmentation algorithm. DORIS outputs 10 different tissue classes including WM, GM, CSF, ventricles, and 6 other subcortical structures (putamen, pallidum, hippocampus, caudate, amygdala, and thalamus). DORIS was trained and validated on a wide range of subjects, including 1,000 individuals from 22 to 90 years old from clinical and research DWI acquisitions, from 5 public databases. In the absence of a "true" ground truth in diffusion space, DORIS used a silver standard strategy from Freesurfer output registered onto the DWI. This strategy is extensively evaluated and discussed in the current study. Segmentation maps provided by DORIS are quantitatively compared to Freesurfer and FSL-fast and the impacts on tractography are evaluated. Overall, we show that DORIS is fast, accurate, and reproducible and that DORIS-based tractograms produce bundles with a longer mean length and fewer anatomically implausible streamlines.

DBB - A Distorted Brain Benchmark for Automatic Tissue Segmentation in Paediatric Patients

T1-weighted magnetic resonance images provide a comprehensive view of the morphology of the human brain at the macro scale. These images are usually the input of a segmentation process that aims detecting the anatomical structures labeling them according to a predefined set of target tissues. Automated methods for brain tissue segmentation rely on anatomical priors of the human brain structures. This is the reason why their performance is quite accurate on healthy individuals. Nevertheless model-based tools become less accurate in clinical practice, specifically in the cases of severe lesions or highly distorted cerebral anatomy. More recently there are empirical evidences that a data-driven approach can be more robust in presence of alterations of brain structures, even though the learning model is trained on healthy brains. Our contribution is a benchmark to support an open investigation on how the tissue segmentation of distorted brains can be improved by adopting a supervised learning approach. We formulate a precise definition of the task and propose an evaluation metric for a fair and quantitative comparison. The training sample is composed of almost one thousand healthy individuals. Data include both T1-weighted MR images and their labeling of brain tissues. The test sample is a collection of several tens of individuals with severe brain distortions. Data and code are openly published on BrainLife, an open science platform for reproducible neuroscience data analysis.

Segmentation of macular neovascularization and leakage in fluorescein angiography images in neovascular age-related macular degeneration using deep learning

BACKGROUND/OBJECTIVES

We aim to develop an objective fully automated Artificial intelligence (AI) algorithm for MNV lesion size and leakage area segmentation on fluorescein angiography (FA) in patients with neovascular age-related macular degeneration (nAMD).

SUBJECTS/METHODS

Two FA image datasets collected form large prospective multicentre trials consisting of 4710 images from 513 patients and 4558 images from 514 patients were used to develop and evaluate a deep learning-based algorithm to detect CNV lesion size and leakage area automatically. Manual segmentation of was performed by certified FA graders of the Vienna Reading Center. Precision, Recall and F1 score between AI predictions and manual annotations were computed. In addition, two masked retina experts conducted a clinical-applicability evaluation, comparing the quality of AI based and manual segmentations.

RESULTS

For CNV lesion size and leakage area segmentation, we obtained F1 scores of 0.73 and 0.65, respectively. Expert review resulted in a slight preference for the automated segmentations in both datasets. The quality of automated segmentations was slightly more often judged as good compared to manual annotations.

CONCLUSIONS

CNV lesion size and leakage area can be segmented by our automated model at human-level performance, its output being well-accepted during clinical applicability testing. The results provide proof-of-concept that an automated deep learning approach can improve efficacy of objective biomarker analysis in FA images and will be well-suited for clinical application.

Deep Diffusion MRI Registration (DDMReg): A Deep Learning Method for Diffusion MRI Registration

In this paper, we present a deep learning method, DDMReg, for accurate registration between diffusion MRI (dMRI) datasets. In dMRI registration, the goal is to spatially align brain anatomical structures while ensuring that local fiber orientations remain consistent with the underlying white matter fiber tract anatomy. DDMReg is a novel method that uses joint whole-brain and tract-specific information for dMRI registration. Based on the successful VoxelMorph framework for image registration, we propose a novel registration architecture that leverages not only whole brain information but also tract-specific fiber orientation information. DDMReg is an unsupervised method for deformable registration between pairs of dMRI datasets: it does not require nonlinearly pre-registered training data or the corresponding deformation fields as ground truth. We perform comparisons with four state-of-the-art registration methods on multiple independently acquired datasets from different populations (including teenagers, young and elderly adults) and different imaging protocols and scanners. We evaluate the registration performance by assessing the ability to align anatomically corresponding brain structures and ensure fiber spatial agreement between different subjects after registration. Experimental results show that DDMReg obtains significantly improved registration performance compared to the state-of-the-art methods. Importantly, we demonstrate successful generalization of DDMReg to dMRI data from different populations with varying ages and acquired using different acquisition protocols and different scanners.

The Mexican magnetic resonance imaging dataset of patients with cocaine use disorder: SUDMEX CONN

Cocaine use disorder (CUD) is a substance use disorder (SUD) characterized by compulsion to seek, use and abuse of cocaine, with severe health and economic consequences for the patients, their families and society. Due to the lack of successful treatments and high relapse rate, more research is needed to understand this and other SUD. Here, we present the SUDMEX CONN dataset, a Mexican open dataset of 74 CUD patients (9 female) and matched 64 healthy controls (6 female) that includes demographic, cognitive, clinical, and magnetic resonance imaging (MRI) data. MRI data includes: 1) structural (T1-weighted), 2) multishell high-angular resolution diffusion-weighted (DWI-HARDI) and 3) functional (resting state fMRI) sequences. The repository contains unprocessed MRI data available in brain imaging data structure (BIDS) format with corresponding metadata available at the OpenNeuro data sharing platform. Researchers can pursue brain variability between these groups or use a single group for a larger population sample.

Measurement(s)	functional brain measurement • Diffusion Weighted Imaging • Abnormality of brain morphology • Alteration Of Cognitive Function • Clinical Study
Technology Type(s)	Functional Magnetic Resonance Imaging • Diffusion Weighted Imaging • Turbo Field Echo MRI • neuropsychological test • Clinical Evaluation
Factor Type(s)	Cocaine Dependence
Sample Characteristic - Organism	Homo
Sample Characteristic - Location	Mexico City

Multimodal Neuroimaging in Rett Syndrome With MECP2 Mutation

Rett syndrome (RTT) is a rare neurodevelopmental disorder characterized by severe cognitive, social, and physical impairments resulting from de novo mutations in the X-chromosomal methyl-CpG binding protein gene 2 (MECP2). While there is still no cure for RTT, exploring up-to date neurofunctional diagnostic markers, discovering new potential therapeutic targets, and searching for novel drug efficacy evaluation indicators are fundamental. Multiple neuroimaging studies on brain structure and function have been carried out in RTT-linked gene mutation carriers to unravel disease-specific imaging features and explore genotype-phenotype associations. Here, we reviewed the neuroimaging literature on this disorder. MRI morphologic studies have shown global atrophy of gray matter (GM) and white matter (WM) and regional variations in brain maturation. Diffusion tensor imaging (DTI) studies have demonstrated reduced fractional anisotropy (FA) in left peripheral WM areas, left major WM tracts, and cingulum bilaterally, and WM microstructural/network topology changes have been further found to be correlated with behavioral abnormalities in RTT. Cerebral blood perfusion imaging studies using single-photon emission CT (SPECT) or PET have evidenced a decreased global cerebral blood flow (CBF), particularly in prefrontal and temporoparietal areas, while magnetic resonance spectroscopy (MRS) and PET studies have contributed to unraveling metabolic alterations in patients with RTT. The results obtained from the available reports confirm that multimodal neuroimaging can provide new insights into a complex interplay between genes, neurotransmitter pathway abnormalities, disease-related behaviors, and clinical severity. However, common limitations related to the available studies include small sample sizes and hypothesis-based and region-specific approaches. We, therefore, conclude that this field is still in its early development phase and that multimodal/multisequence studies with improved post-processing technologies as well as combined PET–MRI approaches are urgently needed to further explore RTT brain alterations.

Pseudo-Label-Assisted Self-Organizing Maps for Brain Tissue Segmentation in Magnetic Resonance Imaging

Brain tissue segmentation in magnetic resonance imaging volumes is an important image processing step for analyzing the human brain. This paper presents a novel approach named Pseudo-Label Assisted Self-Organizing Map (PLA-SOM) that enhances the result produced by a base segmentation method. Using the output of a base method, PLA-SOM calculates pseudo-labels in order to keep inter-class separation and intra-class compactness in the training phase. For the mapping phase, PLA-SOM uses a novel fuzzy function that combines feature space learned by the SOM's prototypes, topological ordering from the map, and spatial information from a brain atlas. We assessed PLA-SOM performance on synthetic and real MRIs of the brain, obtained from the BrainWeb and the Internet Brain Image Repository datasets. The experimental results showed evidence of segmentation improvement achieved by the proposed method over six different base methods. The best segmentation improvements reported by PLA-SOM on synthetic brain scans are 11%, 6%, and 4% for the tissue classes cerebrospinal fluid, gray matter, and white matter, respectively. On real brain scans, PLA-SOM achieved segmentation enhancements of 15%, 5%, and 12% for cerebrospinal fluid, gray matter, and white matter, respectively.

Comparison of Two-Dimensional- and Three-Dimensional-Based U-Net Architectures for Brain Tissue Classification in One-Dimensional Brain CT

Brain tissue segmentation plays a crucial role in feature extraction, volumetric quantification, and morphometric analysis of brain scans. For the assessment of brain structure and integrity, CT is a non-invasive, cheaper, faster, and more widely available modality than MRI. However, the clinical application of CT is mostly limited to the visual assessment of brain integrity and exclusion of copathologies. We have previously developed two-dimensional (2D) deep learning-based segmentation networks that successfully classified brain tissue in head CT. Recently, deep learning-based MRI segmentation models successfully use patch-based three-dimensional (3D) segmentation networks. In this study, we aimed to develop patch-based 3D segmentation networks for CT brain tissue classification. Furthermore, we aimed to compare the performance of 2D- and 3D-based segmentation networks to perform brain tissue classification in anisotropic CT scans. For this purpose, we developed 2D and 3D U-Net-based deep learning models that were trained and validated on MR-derived segmentations from scans of 744 participants of the Gothenburg H70 Cohort with both CT and T1-weighted MRI scans acquired timely close to each other. Segmentation performance of both 2D and 3D models was evaluated on 234 unseen datasets using measures of distance, spatial similarity, and tissue volume. Single-task slice-wise processed 2D U-Nets performed better than multitask patch-based 3D U-Nets in CT brain tissue classification. These findings provide support to the use of 2D U-Nets to segment brain tissue in one-dimensional (1D) CT. This could increase the application of CT to detect brain abnormalities in clinical settings.

Deep learning from MRI-derived labels enables automatic brain tissue classification on human brain CT

Automatic methods for feature extraction, volumetry, and morphometric analysis in clinical neuroscience typically operate on images obtained with magnetic resonance (MR) imaging equipment. Although CT scans are less expensive to acquire and more widely available than MR scans, their application is currently limited to the visual assessment of brain integrity and the exclusion of co-pathologies. CT has rarely been used for tissue classification because the contrast between grey matter and white matter was considered insufficient. In this study, we propose an automatic method for segmenting grey matter (GM), white matter (WM), cerebrospinal fluid (CSF), and intracranial volume (ICV) from head CT images. A U-Net deep learning model was trained and validated on CT images with MRI-derived segmentation labels. We used data from 744 participants of the Gothenburg H70 Birth Cohort Studies for whom CT and T1-weighted MR images had been acquired on the same day. Our proposed model predicted brain tissue classes accurately from unseen CT images (Dice coefficients of 0.79, 0.82, 0.75, 0.93 and 0.98 for GM, WM, CSF, brain volume and ICV, respectively). To contextualize these results, we generated benchmarks based on established MR-based methods and intentional image degradation. Our findings demonstrate that CT-derived segmentations can be used to delineate and quantify brain tissues, opening new possibilities for the use of CT in clinical practice and research.

Automated multiclass tissue segmentation of clinical brain MRIs with lesions

•

A U-Net incorporating spatial prior information can successfully segment 6 brain tissue types.

•

The U-Net was able to segment gray and white matter in the presence of lesions.

•

The U-Net surpassed the performance of its source algorithm in an external dataset.

•

Segmentations were produced in a hundredth of the time of its predecessor algorithm.

Delineation and quantification of normal and abnormal brain tissues on Magnetic Resonance Images is fundamental to the diagnosis and longitudinal assessment of neurological diseases. Here we sought to develop a convolutional neural network for automated multiclass tissue segmentation of brain MRIs that was robust at typical clinical resolutions and in the presence of a variety of lesions. We trained a 3D U-Net for full brain multiclass tissue segmentation from a prior atlas-based segmentation method on an internal dataset that consisted of 558 clinical T1-weighted brain MRIs (453/52/53; training/validation/test) of patients with one of 50 different diagnostic entities (n = 362) or with a normal brain MRI (n = 196). We then used transfer learning to refine our model on an external dataset that consisted of 7 patients with hand-labeled tissue types. We evaluated the tissue-wise and intra-lesion performance with different loss functions and spatial prior information in the validation set and applied the best performing model to the internal and external test sets. The network achieved an average overall Dice score of 0.87 and volume similarity of 0.97 in the internal test set. Further, the network achieved a median intra-lesion tissue segmentation accuracy of 0.85 inside lesions within white matter and 0.61 inside lesions within gray matter. After transfer learning, the network achieved an average overall Dice score of 0.77 and volume similarity of 0.96 in the external dataset compared to human raters. The network had equivalent or better performance than the original atlas-based method on which it was trained across all metrics and produced segmentations in a hundredth of the time. We anticipate that this pipeline will be a useful tool for clinical decision support and quantitative analysis of clinical brain MRIs in the presence of lesions.