Automated 3D Fetal Brain Segmentation Using an Optimized Deep Learning Approach

A Dempster-Shafer Approach to Trustworthy AI With Application to Fetal Brain MRI Segmentation

Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.

Examining the relationship between fetal cortical thickness, gestational age, and maternal psychological distress

In utero exposure to maternal stress, anxiety, and depression has been associated with reduced cortical thickness (CT), and CT changes, in turn, to adverse neuropsychiatric outcomes. Here, we investigated global and regional (G/RCT) changes associated with fetal exposure to maternal psychological distress in 265 brain MRI studies from 177 healthy fetuses of low-risk pregnant women. GCT was measured from cortical gray matter (CGM) voxels; RCT was estimated from 82 cortical regions. GCT and RCT in 87% of regions strongly correlated with GA. Fetal exposure was most strongly associated with RCT in the parahippocampal region, ventromedial prefrontal cortex, and supramarginal gyrus suggesting that cortical alterations commonly associated with prenatal exposure could emerge in-utero. However, we note that while regional fetal brain involvement conformed to patterns observed in newborns and children exposed to prenatal maternal psychological distress, the reported associations did not survive multiple comparisons correction. This could be because the effects are more subtle in this early developmental window or because majority of the pregnant women in our study did not experience high levels of maternal distress. It is our hope that the current findings will spur future hypothesis-driven studies that include a full spectrum of maternal mental health scores.

Review on deep learning fetal brain segmentation from Magnetic Resonance images

Brain segmentation is often the first and most critical step in quantitative analysis of the brain for many clinical applications, including fetal imaging. Different aspects challenge the segmentation of the fetal brain in magnetic resonance imaging (MRI), such as the non-standard position of the fetus owing to his/her movements during the examination, rapid brain development, and the limited availability of imaging data. In recent years, several segmentation methods have been proposed for automatically partitioning the fetal brain from MR images. These algorithms aim to define regions of interest with different shapes and intensities, encompassing the entire brain, or isolating specific structures. Deep learning techniques, particularly convolutional neural networks (CNNs), have become a state-of-the-art approach in the field because they can provide reliable segmentation results over heterogeneous datasets. Here, we review the deep learning algorithms developed in the field of fetal brain segmentation and categorize them according to their target structures. Finally, we discuss the perceived research gaps in the literature of the fetal domain, suggesting possible future research directions that could impact the management of fetal MR images.

A Clinical Approach to Semiautomated Three-Dimensional Fetal Brain Biometry-Comparing the Strengths and Weaknesses of Two Diagnostic Tools: 5DCNS+TM and SonoCNSTM

(1) Objective: We aimed to evaluate the accuracy and efficacy of AI-assisted biometric measurements of the fetal central nervous system (CNS) by comparing two semiautomatic postprocessing tools. We further aimed to discuss the additional value of semiautomatically generated sagittal and coronal planes of the CNS. (2) Methods: Three-dimensional (3D) volumes were analyzed with two semiautomatic software tools, 5DCNS+™ and SonoCNS™. The application of 5DCNS+™ results in nine planes (axial, coronal and sagittal) displayed in a single template; SonoCNS™ depicts three axial cutting sections. The tools were compared regarding automatic biometric measurement accuracy. (3) Results: A total of 129 fetuses were included for final analysis. Our data indicate that, in terms of the biometric quantification of head circumference (HC), biparietal diameter (BPD), transcerebellar diameter (TCD) and cisterna magna (CM), the accuracy of SonoCNS™ was higher with respect to the manual measurement of an experienced examiner compared to 5DCNS+™, whereas it was the other way around regarding the diameter of the posterior horn of the lateral ventricle (Vp). The inclusion of four orthogonal coronal views in 5DCNS+™ gives valuable information regarding spatial arrangements, particularly of midline structures. (4) Conclusions: Both tools were able to ease assessment of the intracranial anatomy, highlighting the additional value of automated algorithms in clinical use. SonoCNS™ showed a superior accuracy of plane reconstruction and biometry, but volume reconstruction using 5DCNS+™ provided more detailed information, which is needed for an entire neurosonogram as suggested by international guidelines.

Fetal brain tissue annotation and segmentation challenge results

In-utero fetal MRI is emerging as an important tool in the diagnosis and analysis of the developing human brain. Automatic segmentation of the developing fetal brain is a vital step in the quantitative analysis of prenatal neurodevelopment both in the research and clinical context. However, manual segmentation of cerebral structures is time-consuming and prone to error and inter-observer variability. Therefore, we organized the Fetal Tissue Annotation (FeTA) Challenge in 2021 in order to encourage the development of automatic segmentation algorithms on an international level. The challenge utilized FeTA Dataset, an open dataset of fetal brain MRI reconstructions segmented into seven different tissues (external cerebrospinal fluid, gray matter, white matter, ventricles, cerebellum, brainstem, deep gray matter). 20 international teams participated in this challenge, submitting a total of 21 algorithms for evaluation. In this paper, we provide a detailed analysis of the results from both a technical and clinical perspective. All participants relied on deep learning methods, mainly U-Nets, with some variability present in the network architecture, optimization, and image pre- and post-processing. The majority of teams used existing medical imaging deep learning frameworks. The main differences between the submissions were the fine tuning done during training, and the specific pre- and post-processing steps performed. The challenge results showed that almost all submissions performed similarly. Four of the top five teams used ensemble learning methods. However, one team's algorithm performed significantly superior to the other submissions, and consisted of an asymmetrical U-Net network architecture. This paper provides a first of its kind benchmark for future automatic multi-tissue segmentation algorithms for the developing human brain in utero.

A Deep Learning-Based Automatic Segmentation and 3D Visualization Technique for Intracranial Hemorrhage Detection Using Computed Tomography Images

Intracranial hemorrhage (ICH) occurs when blood leaks inside the skull as a result of trauma to the skull or due to medical conditions. ICH usually requires immediate medical and surgical attention because the disease has a high mortality rate, long-term disability potential, and other potentially life-threatening complications. There are a wide range of severity levels, sizes, and morphologies of ICHs, making accurate identification challenging. Hemorrhages that are small are more likely to be missed, particularly in healthcare systems that experience high turnover when it comes to computed tomography (CT) investigations. Although many neuroimaging modalities have been developed, CT remains the standard for diagnosing trauma and hemorrhage (including non-traumatic ones). A CT scan-based diagnosis can provide time-critical, urgent ICH surgery that could save lives because CT scan-based diagnoses can be obtained rapidly. The purpose of this study is to develop a machine-learning algorithm that can detect intracranial hemorrhage based on plain CT images taken from 75 patients. CT images were preprocessed using brain windowing, skull-stripping, and image inversion techniques. Hemorrhage segmentation was performed using multiple pre-trained models on preprocessed CT images. A U-Net model with DenseNet201 pre-trained encoder outperformed other U-Net, U-Net++, and FPN (Feature Pyramid Network) models with the highest Dice similarity coefficient (DSC) and intersection over union (IoU) scores, which were previously used in many other medical applications. We presented a three-dimensional brain model highlighting hemorrhages from ground truth and predicted masks. The volume of hemorrhage was measured volumetrically to determine the size of the hematoma. This study is essential in examining ICH for diagnostic purposes in clinical practice by comparing the predicted 3D model with the ground truth.

Self-Enhanced Mixed Attention Network for Three-Modal Images Few-Shot Semantic Segmentation

As an important computer vision technique, image segmentation has been widely used in various tasks. However, in some extreme cases, the insufficient illumination would result in a great impact on the performance of the model. So more and more fully supervised methods use multi-modal images as their input. The dense annotated large datasets are difficult to obtain, but the few-shot methods still can have satisfactory results with few pixel-annotated samples. Therefore, we propose the Visible-Depth-Thermal (three-modal) images few-shot semantic segmentation method. It utilizes the homogeneous information of three-modal images and the complementary information of different modal images, which can improve the performance of few-shot segmentation tasks. We constructed a novel indoor dataset VDT-2048-5ⁱ for the three-modal images few-shot semantic segmentation task. We also proposed a Self-Enhanced Mixed Attention Network (SEMANet), which consists of a Self-Enhanced module (SE) and a Mixed Attention module (MA). The SE module amplifies the difference between the different kinds of features and strengthens the weak connection for the foreground features. The MA module fuses the three-modal feature to obtain a better feature. Compared with the most advanced methods before, our model improves mIoU by 3.8% and 3.3% in 1-shot and 5-shot settings, respectively, which achieves state-of-the-art performance. In the future, we will solve failure cases by obtaining more discriminative and robust feature representations, and explore achieving high performance with fewer parameters and computational costs.

Review of deep learning and artificial intelligence models in fetal brain magnetic resonance imaging

Central nervous system abnormalities in fetuses are fairly common, happening in 0.1% to 0.2% of live births and in 3% to 6% of stillbirths. So initial detection and categorization of fetal Brain abnormalities are critical. Manually detecting and segmenting fetal brain magnetic resonance imaging (MRI) could be time-consuming, and susceptible to interpreter experience. Artificial intelligence (AI) algorithms and machine learning approaches have a high potential for assisting in the early detection of these problems, improving the diagnosis process and follow-up procedures. The use of AI and machine learning techniques in fetal brain MRI was the subject of this narrative review paper. Using AI, anatomic fetal brain MRI processing has investigated models to predict specific landmarks and segmentation automatically. All gestation age weeks (17-38 wk) and different AI models (mainly Convolutional Neural Network and U-Net) have been used. Some models' accuracy achieved 95% and more. AI could help preprocess and post-process fetal images and reconstruct images. Also, AI can be used for gestational age prediction (with one-week accuracy), fetal brain extraction, fetal brain segmentation, and placenta detection. Some fetal brain linear measurements, such as Cerebral and Bone Biparietal Diameter, have been suggested. Classification of brain pathology was studied using diagonal quadratic discriminates analysis, K-nearest neighbor, random forest, naive Bayes, and radial basis function neural network classifiers. Deep learning methods will become more powerful as more large-scale, labeled datasets become available. Having shared fetal brain MRI datasets is crucial because there aren not many fetal brain pictures available. Also, physicians should be aware of AI's function in fetal brain MRI, particularly neuroradiologists, general radiologists, and perinatologists.

Systematic evaluation of head motion on resting-state functional connectivity MRI in the neonate

Reliability and robustness of resting state functional connectivity MRI (rs-fcMRI) relies, in part, on minimizing the influence of head motion on measured brain signals. The confounding effects of head motion on functional connectivity have been extensively studied in adults, but its impact on newborn brain connectivity remains unexplored. Here, using a large newborn data set consisting of 159 rs-fcMRI scans acquired in the Developing Brain Institute at Children's National Hospital and 416 scans from The Developing Human Connectome Project (dHCP), we systematically investigated associations between head motion and rs-fcMRI. Head motion during the scan significantly affected connectivity at sensory-related networks and default mode networks, and at the whole brain scale; the direction of motion effects varied across the whole brain. Comparing high- versus low-head motion groups suggested that head motion can impact connectivity estimates across the whole brain. Censoring of high-motion volumes using frame-wise displacement significantly reduced the confounding effects of head motion on neonatal rs-fcMRI. Lastly, in the dHCP data set, we demonstrated similar persistent associations between head motion and network connectivity despite implementing a standard denoising strategy. Collectively, our results highlight the importance of using rigorous head motion correction in preprocessing neonatal rs-fcMRI to yield reliable estimates of brain activity.

Development of Gestational Age-Based Fetal Brain and Intracranial Volume Reference Norms Using Deep Learning

BACKGROUND AND PURPOSE

Fetal brain MR imaging interpretations are subjective and require subspecialty expertise. We aimed to develop a deep learning algorithm for automatically measuring intracranial and brain volumes of fetal brain MRIs across gestational ages.

MATERIALS AND METHODS

This retrospective study included 246 patients with singleton pregnancies at 19-38 weeks gestation. A 3D U-Net was trained to segment the intracranial contents of 2D fetal brain MRIs in the axial, coronal, and sagittal planes. An additional 3D U-Net was trained to segment the brain from the output of the first model. Models were tested on MRIs of 10 patients (28 planes) via Dice coefficients and volume comparison with manual reference segmentations. Trained U-Nets were applied to 200 additional MRIs to develop normative reference intracranial and brain volumes across gestational ages and then to 9 pathologic fetal brains.

RESULTS

Fetal intracranial and brain compartments were automatically segmented in a mean of 6.8 (SD, 1.2) seconds with median Dices score of 0.95 and 0.90, respectively (interquartile ranges, 0.91-0.96/0.89-0.91) on the test set. Correlation with manual volume measurements was high (Pearson r = 0.996, P < .001). Normative samples of intracranial and brain volumes across gestational ages were developed. Eight of 9 pathologic fetal intracranial volumes were automatically predicted to be >2 SDs from this age-specific reference mean. There were no effects of fetal sex, maternal diabetes, or maternal age on intracranial or brain volumes across gestational ages.

CONCLUSIONS

Deep learning techniques can quickly and accurately quantify intracranial and brain volumes on clinical fetal brain MRIs and identify abnormal volumes on the basis of a normative reference standard.

Harnessing the Power of Advanced Fetal Neuroimaging to Understand In Utero Footprints for Later Neuropsychiatric Disorders

Adverse intrauterine events may profoundly impact fetal risk for future adult diseases. The mechanisms underlying this increased vulnerability are complex and remain poorly understood. Contemporary advances in fetal magnetic resonance imaging (MRI) have provided clinicians and scientists with unprecedented access to in vivo human fetal brain development to begin to identify emerging endophenotypes of neuropsychiatric disorders such as autism spectrum disorder, attention-deficit/hyperactivity disorder, and schizophrenia. In this review, we discuss salient findings of normal fetal neurodevelopment from studies using advanced, multimodal MRI that have provided unparalleled characterization of in utero prenatal brain morphology, metabolism, microstructure, and functional connectivity. We appraise the clinical utility of these normative data in identifying high-risk fetuses before birth. We highlight available studies that have investigated the predictive validity of advanced prenatal brain MRI findings and long-term neurodevelopmental outcomes. We then discuss how ex utero quantitative MRI findings can inform in utero investigations toward the pursuit of early biomarkers of risk. Lastly, we explore future opportunities to advance our understanding of the prenatal origins of neuropsychiatric disorders using precision fetal imaging.

Magnetic resonance imaging of the monkey fetal brain in utero

Non-human primates (NHPs) are the closest living relatives of the human and play a critical role in investigating the effects of maternal viral infection and consumption of medicines, drugs, and alcohol on fetal development. With the advance of contemporary fast MRI techniques with parallel imaging, fetal MRI is becoming a robust tool increasingly used in clinical practice and preclinical studies to examine congenital abnormalities including placental dysfunction, congenital heart disease (CHD), and brain abnormalities non-invasively. Because NHPs are usually scanned under anesthesia, the motion artifact is reduced substantially, allowing multi-parameter MRI techniques to be used intensively to examine the fetal development in a single scanning session or longitudinal studies. In this paper, the MRI techniques for scanning monkey fetal brains in utero in biomedical research are summarized. Also, a fast imaging protocol including T2-weighted imaging, diffusion MRI, resting-state functional MRI (rsfMRI) to examine rhesus monkey fetal brains in utero on a clinical 3T scanner is introduced.

Effective Approaches to Fetal Brain Segmentation in MRI and Gestational Age Estimation by Utilizing a Multiview Deep Inception Residual Network and Radiomics

To completely comprehend neurodevelopment in healthy and congenitally abnormal fetuses, quantitative analysis of the human fetal brain is essential. This analysis requires the use of automatic multi-tissue fetal brain segmentation techniques. This paper proposes an end-to-end automatic yet effective method for a multi-tissue fetal brain segmentation model called IRMMNET. It includes a inception residual encoder block (EB) and a dense spatial attention (DSAM) block, which facilitate the extraction of multi-scale fetal-brain-tissue-relevant information from multi-view MRI images, enhance the feature reuse, and substantially reduce the number of parameters of the segmentation model. Additionally, we propose three methods for predicting gestational age (GA)-GA prediction by using a 3D autoencoder, GA prediction using radiomics features, and GA prediction using the IRMMNET segmentation model's encoder. Our experiments were performed on a dataset of 80 pathological and non-pathological magnetic resonance fetal brain volume reconstructions across a range of gestational ages (20 to 33 weeks) that were manually segmented into seven different tissue categories. The results showed that the proposed fetal brain segmentation model achieved a Dice score of 0.791±0.18, outperforming the state-of-the-art methods. The radiomics-based GA prediction methods achieved the best results (RMSE: 1.42). We also demonstrated the generalization capabilities of the proposed methods for tasks such as head and neck tumor segmentation and the prediction of patients' survival days.

Semi-automatic segmentation of the fetal brain from magnetic resonance imaging

BACKGROUND

Volumetric measurements of fetal brain maturation in the third trimester of pregnancy are key predictors of developmental outcomes. Improved understanding of fetal brain development trajectories may aid in identifying and clinically managing at-risk fetuses. Currently, fetal brain structures in magnetic resonance images (MRI) are often manually segmented, which requires both time and expertise. To facilitate the targeting and measurement of brain structures in the fetus, we compared the results of five segmentation methods applied to fetal brain MRI data to gold-standard manual tracings.

METHODS

Adult women with singleton pregnancies (n = 21), of whom five were scanned twice, approximately 3 weeks apart, were recruited [26 total datasets, median gestational age (GA) = 34.8, IQR = 30.9-36.6]. T2-weighted single-shot fast spin echo images of the fetal brain were acquired on 1.5T and 3T MRI scanners. Images were first combined into a single 3D anatomical volume. Next, a trained tracer manually segmented the thalamus, cerebellum, and total cerebral volumes. The manual segmentations were compared with five automatic methods of segmentation available within Advanced Normalization Tools (ANTs) and FMRIB's Linear Image Registration Tool (FLIRT) toolboxes. The manual and automatic labels were compared using Dice similarity coefficients (DSCs). The DSC values were compared using Friedman's test for repeated measures.

RESULTS

Comparing cerebellum and thalamus masks against the manually segmented masks, the median DSC values for ANTs and FLIRT were 0.72 [interquartile range (IQR) = 0.6-0.8] and 0.54 (IQR = 0.4-0.6), respectively. A Friedman's test indicated that the ANTs registration methods, primarily nonlinear methods, performed better than FLIRT (p < 0.001).

CONCLUSION

Deformable registration methods provided the most accurate results relative to manual segmentation. Overall, this semi-automatic subcortical segmentation method provides reliable performance to segment subcortical volumes in fetal MR images. This method reduces the costs of manual segmentation, facilitating the measurement of typical and atypical fetal brain development.

An Automated Deep Learning Model for the Cerebellum Segmentation from Fetal Brain Images

Cerebellum measures taken from routinely obtained ultrasound (US) images have been frequently employed to determine gestational age and identify developing central nervous system's anatomical abnormalities. Standardized cerebellar assessments from large-scale clinical datasets are required to investigate correlations between the growing cerebellum and postnatal neurodevelopmental results. These studies could uncover structural abnormalities that could be employed as indicators to forecast neurodevelopmental and growth consequences. To achieve this, higher-throughput, precise, and impartial measures must be used to replace the existing human, semiautomatic, and advanced algorithms, which seem to be time-consuming and inaccurate. In this article, we presented an innovative deep learning (DL) technique for automatic fetal cerebellum segmentation from 2-dimensional (2D) US brain images. We present ReU-Net, a semantic segmentation network tailored to the anatomy of the fetal cerebellum. Moreover, we use U-Net as a foundation models with the incorporation of residual blocks and Wiener filter over the last 2 layers to segregate the cerebellum (c) from the noisy US data. 590 images for training and 150 images for testing were taken; also, we employed a 5-fold cross-assessment method. Our ReU-Net scored 91%, 92%, 25.42, 98%, 92%, and 94% for Dice Score Coefficient (DSC), F1-score, Hausdorff Distance (HD), accuracy, recall, and precision, correspondingly. The suggested method outperforms the other U-Net predicated techniques by a quantitatively significant margin (p 0.001). Our presented approach can be used to allow high bandwidth imaging techniques in medical study fetal US images as well as biometric evaluation on a broader scale in fetal US images.

FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net

An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.