1
|
Fidon L, Aertsen M, Kofler F, Bink A, David AL, Deprest T, Emam D, Guffens F, Jakab A, Kasprian G, Kienast P, Melbourne A, Menze B, Mufti N, Pogledic I, Prayer D, Stuempflen M, Van Elslander E, Ourselin S, Deprest J, Vercauteren T. A Dempster-Shafer Approach to Trustworthy AI With Application to Fetal Brain MRI Segmentation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:3784-3795. [PMID: 38198270 DOI: 10.1109/tpami.2023.3346330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.
Collapse
|
2
|
Ciceri T, Casartelli L, Montano F, Conte S, Squarcina L, Bertoldo A, Agarwal N, Brambilla P, Peruzzo D. Fetal brain MRI atlases and datasets: A review. Neuroimage 2024; 292:120603. [PMID: 38588833 DOI: 10.1016/j.neuroimage.2024.120603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 03/28/2024] [Accepted: 04/05/2024] [Indexed: 04/10/2024] Open
Abstract
Fetal brain development is a complex process involving different stages of growth and organization which are crucial for the development of brain circuits and neural connections. Fetal atlases and labeled datasets are promising tools to investigate prenatal brain development. They support the identification of atypical brain patterns, providing insights into potential early signs of clinical conditions. In a nutshell, prenatal brain imaging and post-processing via modern tools are a cutting-edge field that will significantly contribute to the advancement of our understanding of fetal development. In this work, we first provide terminological clarification for specific terms (i.e., "brain template" and "brain atlas"), highlighting potentially misleading interpretations related to inconsistent use of terms in the literature. We discuss the major structures and neurodevelopmental milestones characterizing fetal brain ontogenesis. Our main contribution is the systematic review of 18 prenatal brain atlases and 3 datasets. We also tangentially focus on clinical, research, and ethical implications of prenatal neuroimaging.
Collapse
Affiliation(s)
- Tommaso Ciceri
- NeuroImaging Lab, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy; Department of Information Engineering, University of Padua, Padua, Italy
| | - Luca Casartelli
- Theoretical and Cognitive Neuroscience Unit, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Florian Montano
- Diagnostic Imaging and Neuroradiology Unit, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Stefania Conte
- Psychology Department, State University of New York at Binghamton, New York, USA
| | - Letizia Squarcina
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | - Alessandra Bertoldo
- Department of Information Engineering, University of Padua, Padua, Italy; Padova Neuroscience Center, University of Padua, Padua, Italy
| | - Nivedita Agarwal
- Diagnostic Imaging and Neuroradiology Unit, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Paolo Brambilla
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy; Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.
| | - Denis Peruzzo
- NeuroImaging Lab, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| |
Collapse
|
3
|
Uus AU, Hall M, Grigorescu I, Avena Zampieri C, Egloff Collado A, Payette K, Matthew J, Kyriakopoulou V, Hajnal JV, Hutter J, Rutherford MA, Deprez M, Story L. Automated body organ segmentation, volumetry and population-averaged atlas for 3D motion-corrected T2-weighted fetal body MRI. Sci Rep 2024; 14:6637. [PMID: 38503833 PMCID: PMC10950851 DOI: 10.1038/s41598-024-57087-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 03/14/2024] [Indexed: 03/21/2024] Open
Abstract
Structural fetal body MRI provides true 3D information required for volumetry of fetal organs. However, current clinical and research practice primarily relies on manual slice-wise segmentation of raw T2-weighted stacks, which is time consuming, subject to inter- and intra-observer bias and affected by motion-corruption. Furthermore, there are no existing standard guidelines defining a universal approach to parcellation of fetal organs. This work produces the first parcellation protocol of the fetal body organs for motion-corrected 3D fetal body MRI. It includes 10 organ ROIs relevant to fetal quantitative volumetry studies. We also introduce the first population-averaged T2w MRI atlas of the fetal body. The protocol was used as a basis for training of a neural network for automated organ segmentation. It showed robust performance for different gestational ages. This solution minimises the need for manual editing and significantly reduces time. The general feasibility of the proposed pipeline was also assessed by analysis of organ growth charts created from automated parcellations of 91 normal control 3T MRI datasets that showed expected increase in volumetry during 22-38 weeks gestational age range.
Collapse
Affiliation(s)
- Alena U Uus
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK.
| | - Megan Hall
- Centre for the Developing Brain, King's College London, London, UK
- Department of Women and Children's Health, King's College London, London, UK
- Fetal Medicine Unit, Guy's and St Thomas' NHS Foundation Trust, London, UK
| | - Irina Grigorescu
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | - Carla Avena Zampieri
- Centre for the Developing Brain, King's College London, London, UK
- Department of Women and Children's Health, King's College London, London, UK
| | | | - Kelly Payette
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | - Jacqueline Matthew
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | | | - Joseph V Hajnal
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | - Jana Hutter
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
- Smart Imaging Lab, Radiological Institute, University Hospital Erlangen, Erlangen, Germany
| | | | - Maria Deprez
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | - Lisa Story
- Centre for the Developing Brain, King's College London, London, UK
- Department of Women and Children's Health, King's College London, London, UK
- Fetal Medicine Unit, Guy's and St Thomas' NHS Foundation Trust, London, UK
| |
Collapse
|
4
|
Matthew J, Uus A, De Souza L, Wright R, Fukami-Gartner A, Priego G, Saija C, Deprez M, Collado AE, Hutter J, Story L, Malamateniou C, Rhode K, Hajnal J, Rutherford MA. Craniofacial phenotyping with fetal MRI: a feasibility study of 3D visualisation, segmentation, surface-rendered and physical models. BMC Med Imaging 2024; 24:52. [PMID: 38429666 PMCID: PMC10905839 DOI: 10.1186/s12880-024-01230-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 02/19/2024] [Indexed: 03/03/2024] Open
Abstract
This study explores the potential of 3D Slice-to-Volume Registration (SVR) motion-corrected fetal MRI for craniofacial assessment, traditionally used only for fetal brain analysis. In addition, we present the first description of an automated pipeline based on 3D Attention UNet trained for 3D fetal MRI craniofacial segmentation, followed by surface refinement. Results of 3D printing of selected models are also presented.Qualitative analysis of multiplanar volumes, based on the SVR output and surface segmentations outputs, were assessed with computer and printed models, using standardised protocols that we developed for evaluating image quality and visibility of diagnostic craniofacial features. A test set of 25, postnatally confirmed, Trisomy 21 fetal cases (24-36 weeks gestational age), revealed that 3D reconstructed T2 SVR images provided 66-100% visibility of relevant craniofacial and head structures in the SVR output, and 20-100% and 60-90% anatomical visibility was seen for the baseline and refined 3D computer surface model outputs respectively. Furthermore, 12 of 25 cases, 48%, of refined surface models demonstrated good or excellent overall quality with a further 9 cases, 36%, demonstrating moderate quality to include facial, scalp and external ears. Additional 3D printing of 12 physical real-size models (20-36 weeks gestational age) revealed good/excellent overall quality in all cases and distinguishable features between healthy control cases and cases with confirmed anomalies, with only minor manual adjustments required before 3D printing.Despite varying image quality and data heterogeneity, 3D T2w SVR reconstructions and models provided sufficient resolution for the subjective characterisation of subtle craniofacial features. We also contributed a publicly accessible online 3D T2w MRI atlas of the fetal head, validated for accurate representation of normal fetal anatomy.Future research will focus on quantitative analysis, optimizing the pipeline, and exploring diagnostic, counselling, and educational applications in fetal craniofacial assessment.
Collapse
Affiliation(s)
- Jacqueline Matthew
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK.
- Guy's and St Thomas' NHS Foundation Trust, London, UK.
| | - Alena Uus
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Leah De Souza
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Robert Wright
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Abi Fukami-Gartner
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Gema Priego
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
- Barking, Havering and Redbridge University Hospitals NHS Trust, London, UK
| | - Carlo Saija
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Maria Deprez
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Alexia Egloff Collado
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
- Guy's and St Thomas' NHS Foundation Trust, London, UK
| | - Jana Hutter
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Lisa Story
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
- Guy's and St Thomas' NHS Foundation Trust, London, UK
| | | | - Kawal Rhode
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Jo Hajnal
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
| | - Mary A Rutherford
- School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Hospital, London, UK
- Guy's and St Thomas' NHS Foundation Trust, London, UK
| |
Collapse
|
5
|
Huang Y, Yang X, Liu L, Zhou H, Chang A, Zhou X, Chen R, Yu J, Chen J, Chen C, Liu S, Chi H, Hu X, Yue K, Li L, Grau V, Fan DP, Dong F, Ni D. Segment anything model for medical images? Med Image Anal 2024; 92:103061. [PMID: 38086235 DOI: 10.1016/j.media.2023.103061] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 09/28/2023] [Accepted: 12/05/2023] [Indexed: 01/12/2024]
Abstract
The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: (1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. (2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. (3) SAM performed better with manual hints, especially box, than the Everything mode. (4) SAM could help human annotation with high labeling quality and less time. (5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. (6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. (7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. (8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. Codes and models are available at: https://github.com/yuhoo0302/Segment-Anything-Model-for-Medical-Images. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM.
Collapse
Affiliation(s)
- Yuhao Huang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Xin Yang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Lian Liu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Han Zhou
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Ao Chang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Xinrui Zhou
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Rusi Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Junxuan Yu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Jiongquan Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Chaoyu Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Sijing Liu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | | | - Xindi Hu
- Shenzhen RayShape Medical Technology Co., Ltd, Shenzhen, China
| | - Kejuan Yue
- Hunan First Normal University, Changsha, China
| | - Lei Li
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Vicente Grau
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Deng-Ping Fan
- Computer Vision Lab (CVL), ETH Zurich, Zurich, Switzerland
| | - Fajin Dong
- Ultrasound Department, the Second Clinical Medical College, Jinan University, China; First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People's Hospital, Shenzhen, China.
| | - Dong Ni
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China.
| |
Collapse
|
6
|
He Y, Ge R, Qi X, Chen Y, Wu J, Coatrieux JL, Yang G, Li S. Learning Better Registration to Learn Better Few-Shot Medical Image Segmentation: Authenticity, Diversity, and Robustness. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:2588-2601. [PMID: 35895657 DOI: 10.1109/tnnls.2022.3190452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In this work, we address the task of few-shot medical image segmentation (MIS) with a novel proposed framework based on the learning registration to learn segmentation (LRLS) paradigm. To cope with the limitations of lack of authenticity, diversity, and robustness in the existing LRLS frameworks, we propose the better registration better segmentation (BRBS) framework with three main contributions that are experimentally shown to have substantial practical merit. First, we improve the authenticity in the registration-based generation program and propose the knowledge consistency constraint strategy that constrains the registration network to learn according to the domain knowledge. It brings the semantic-aligned and topology-preserved registration, thus allowing the generation program to output new data with great space and style authenticity. Second, we deeply studied the diversity of the generation process and propose the space-style sampling program, which introduces the modeling of the transformation path of style and space change between few atlases and numerous unlabeled images into the generation program. Therefore, the sampling on the transformation paths provides much more diverse space and style features to the generated data effectively improving the diversity. Third, we first highlight the robustness in the learning of segmentation in the LRLS paradigm and propose the mix misalignment regularization, which simulates the misalignment distortion and constrains the network to reduce the fitting degree of misaligned regions. Therefore, it builds regularization for these regions improving the robustness of segmentation learning. Without any bells and whistles, our approach achieves a new state-of-the-art performance in few-shot MIS on two challenging tasks that outperform the existing LRLS-based few-shot methods. We believe that this novel and effective framework will provide a powerful few-shot benchmark for the field of medical image and efficiently reduce the costs of medical image research. All of our code will be made publicly available online.
Collapse
|
7
|
Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y, Jose A, Roy R, Merhof D. Advances in medical image analysis with vision Transformers: A comprehensive review. Med Image Anal 2024; 91:103000. [PMID: 37883822 DOI: 10.1016/j.media.2023.103000] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 09/30/2023] [Accepted: 10/11/2023] [Indexed: 10/28/2023]
Abstract
The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.
Collapse
Affiliation(s)
- Reza Azad
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Amirhossein Kazerouni
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Moein Heidari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | | | - Amirali Molaei
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Yiwei Jia
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Abin Jose
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Rijo Roy
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| |
Collapse
|
8
|
Steger C, Moatti C, Payette K, De Silvestro A, Nguyen TD, Coraj S, Yakoub N, Natalucci G, Kottke R, Tuura R, Knirsch W, Jakab A. Characterization of dynamic patterns of human fetal to neonatal brain asymmetry with deformation-based morphometry. Front Neurosci 2023; 17:1252850. [PMID: 38130698 PMCID: PMC10734644 DOI: 10.3389/fnins.2023.1252850] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 11/03/2023] [Indexed: 12/23/2023] Open
Abstract
Introduction Despite established knowledge on the morphological and functional asymmetries in the human brain, the understanding of how brain asymmetry patterns change during late fetal to neonatal life remains incomplete. The goal of this study was to characterize the dynamic patterns of inter-hemispheric brain asymmetry over this critically important developmental stage using longitudinally acquired MRI scans. Methods Super-resolution reconstructed T2-weighted MRI of 20 neurotypically developing participants were used, and for each participant fetal and neonatal MRI was acquired. To quantify brain morphological changes, deformation-based morphometry (DBM) on the longitudinal MRI scans was utilized. Two registration frameworks were evaluated and used in our study: (A) fetal to neonatal image registration and (B) registration through a mid-time template. Developmental changes of cerebral asymmetry were characterized as (A) the inter-hemispheric differences of the Jacobian determinant (JD) of fetal to neonatal morphometry change and the (B) time-dependent change of the JD capturing left-right differences at fetal or neonatal time points. Left-right and fetal-neonatal differences were statistically tested using multivariate linear models, corrected for participants' age and sex and using threshold-free cluster enhancement. Results Fetal to neonatal morphometry changes demonstrated asymmetry in the temporal pole, and left-right asymmetry differences between fetal and neonatal timepoints revealed temporal changes in the temporal pole, likely to go from right dominant in fetal to a bilateral morphology in neonatal timepoint. Furthermore, the analysis revealed right-dominant subcortical gray matter in neonates and three clusters of increased JD values in the left hemisphere from fetal to neonatal timepoints. Discussion While these findings provide evidence that morphological asymmetry gradually emerges during development, discrepancies between registration frameworks require careful considerations when using DBM for longitudinal data of early brain development.
Collapse
Affiliation(s)
- Céline Steger
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
- Pediatric Heart Center, Division of Pediatric Cardiology, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and Swiss Federal Institute of Technology, Zurich, Switzerland
| | - Charles Moatti
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich, Switzerland
| | - Kelly Payette
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Alexandra De Silvestro
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
- Pediatric Heart Center, Division of Pediatric Cardiology, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Thi Dao Nguyen
- Newborn Research, Department of Neonatology, University of Zurich and University Hospital Zurich, Zurich, Switzerland
| | - Seline Coraj
- Larsson-Rosenquist Foundation Center for Neurodevelopment, Growth and Nutrition of the Newborn, Department of Neonatology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Ninib Yakoub
- Larsson-Rosenquist Foundation Center for Neurodevelopment, Growth and Nutrition of the Newborn, Department of Neonatology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Giancarlo Natalucci
- Newborn Research, Department of Neonatology, University of Zurich and University Hospital Zurich, Zurich, Switzerland
- Larsson-Rosenquist Foundation Center for Neurodevelopment, Growth and Nutrition of the Newborn, Department of Neonatology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Raimund Kottke
- Department of Diagnostic Imaging, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Ruth Tuura
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Walter Knirsch
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
- Pediatric Heart Center, Division of Pediatric Cardiology, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Andras Jakab
- Center for MR Research, University Children’s Hospital Zurich, University of Zurich, Zürich, Switzerland
- Children’s Research Center, University Children’s Hospital Zurich, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and Swiss Federal Institute of Technology, Zurich, Switzerland
| |
Collapse
|
9
|
Peng Y, Xu Y, Wang M, Zhang H, Xie J. The nnU-Net based method for automatic segmenting fetal brain tissues. Health Inf Sci Syst 2023; 11:17. [PMID: 36998806 PMCID: PMC10043149 DOI: 10.1007/s13755-023-00220-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Accepted: 03/09/2023] [Indexed: 04/01/2023] Open
Abstract
The magnetic resonance (MR) images of fetuses make it possible for doctors to detect out pathological fetal brains in early stages. Brain tissue segmentation is prerequisite for making brain morphology and volume analyses. nnU-Net is an automatic segmentation method based on deep learning. It can adaptively configure itself, so as to adapt to a specific task via preprocessing, network architecture, training, and post-processing. Therefore, we adapt nnU-Net to segment seven types of fetal brain tissues, including external cerebrospinal fluid, gray matter, white matter, ventricle, cerebellum, deep gray matter, and brainstem. With regard to the characteristics of the FeTA 2021 data, some adjustments are made to the original nnU-Net, so that it can segment seven types of fetal brain tissues precisely as far as possible. The average segmentation results on FeTA 2021 training data demonstrate that our advanced nnU-Net is superior to the peers including SegNet, CoTr, AC U-Net and ResUnet. Its average segmentation results are 0.842, 11.759 and 0.957 in terms of Dice, HD95 and VS criteria. Moreover, the experimental results on FeTA 2021 test data further demonstrate that our advanced nnU-Net has obtained good segmentation performance of 0.774, 14.699 and 0.875 in terms of Dice, HD95 and VS, ranked the third in FeTA 2021 challenge. Our advanced nnU-Net realized the task for segmenting the fetal brain tissues using MR images of different gestational ages, which can help doctors to make correct and seasonable diagnoses.
Collapse
Affiliation(s)
- Ying Peng
- School of Computer Science, Shaanxi Normal University, West Chang’an Avenue, Chang’an District, Xi’an, 710119 Shaanxi People’s Republic of China
| | - Yandi Xu
- School of Medicine, Tulane University, 1430 Tulane Ave, New Orleans, LA 70112 USA
| | - Mingzhao Wang
- School of Computer Science, Shaanxi Normal University, West Chang’an Avenue, Chang’an District, Xi’an, 710119 Shaanxi People’s Republic of China
| | - Huiquan Zhang
- School of Computer Science, Shaanxi Normal University, West Chang’an Avenue, Chang’an District, Xi’an, 710119 Shaanxi People’s Republic of China
| | - Juanying Xie
- School of Computer Science, Shaanxi Normal University, West Chang’an Avenue, Chang’an District, Xi’an, 710119 Shaanxi People’s Republic of China
| |
Collapse
|
10
|
Ciceri T, Squarcina L, Giubergia A, Bertoldo A, Brambilla P, Peruzzo D. Review on deep learning fetal brain segmentation from Magnetic Resonance images. Artif Intell Med 2023; 143:102608. [PMID: 37673558 DOI: 10.1016/j.artmed.2023.102608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 05/31/2023] [Accepted: 06/06/2023] [Indexed: 09/08/2023]
Abstract
Brain segmentation is often the first and most critical step in quantitative analysis of the brain for many clinical applications, including fetal imaging. Different aspects challenge the segmentation of the fetal brain in magnetic resonance imaging (MRI), such as the non-standard position of the fetus owing to his/her movements during the examination, rapid brain development, and the limited availability of imaging data. In recent years, several segmentation methods have been proposed for automatically partitioning the fetal brain from MR images. These algorithms aim to define regions of interest with different shapes and intensities, encompassing the entire brain, or isolating specific structures. Deep learning techniques, particularly convolutional neural networks (CNNs), have become a state-of-the-art approach in the field because they can provide reliable segmentation results over heterogeneous datasets. Here, we review the deep learning algorithms developed in the field of fetal brain segmentation and categorize them according to their target structures. Finally, we discuss the perceived research gaps in the literature of the fetal domain, suggesting possible future research directions that could impact the management of fetal MR images.
Collapse
Affiliation(s)
- Tommaso Ciceri
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy; Department of Information Engineering, University of Padua, Padua, Italy
| | - Letizia Squarcina
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | - Alice Giubergia
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy; Department of Information Engineering, University of Padua, Padua, Italy
| | - Alessandra Bertoldo
- Department of Information Engineering, University of Padua, Padua, Italy; University of Padua, Padova Neuroscience Center, Padua, Italy
| | - Paolo Brambilla
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy; Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.
| | - Denis Peruzzo
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| |
Collapse
|
11
|
Payette K, Li HB, de Dumast P, Licandro R, Ji H, Siddiquee MMR, Xu D, Myronenko A, Liu H, Pei Y, Wang L, Peng Y, Xie J, Zhang H, Dong G, Fu H, Wang G, Rieu Z, Kim D, Kim HG, Karimi D, Gholipour A, Torres HR, Oliveira B, Vilaça JL, Lin Y, Avisdris N, Ben-Zvi O, Bashat DB, Fidon L, Aertsen M, Vercauteren T, Sobotka D, Langs G, Alenyà M, Villanueva MI, Camara O, Fadida BS, Joskowicz L, Weibin L, Yi L, Xuesong L, Mazher M, Qayyum A, Puig D, Kebiri H, Zhang Z, Xu X, Wu D, Liao K, Wu Y, Chen J, Xu Y, Zhao L, Vasung L, Menze B, Cuadra MB, Jakab A. Fetal brain tissue annotation and segmentation challenge results. Med Image Anal 2023; 88:102833. [PMID: 37267773 DOI: 10.1016/j.media.2023.102833] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 03/16/2023] [Accepted: 04/20/2023] [Indexed: 06/04/2023]
Abstract
In-utero fetal MRI is emerging as an important tool in the diagnosis and analysis of the developing human brain. Automatic segmentation of the developing fetal brain is a vital step in the quantitative analysis of prenatal neurodevelopment both in the research and clinical context. However, manual segmentation of cerebral structures is time-consuming and prone to error and inter-observer variability. Therefore, we organized the Fetal Tissue Annotation (FeTA) Challenge in 2021 in order to encourage the development of automatic segmentation algorithms on an international level. The challenge utilized FeTA Dataset, an open dataset of fetal brain MRI reconstructions segmented into seven different tissues (external cerebrospinal fluid, gray matter, white matter, ventricles, cerebellum, brainstem, deep gray matter). 20 international teams participated in this challenge, submitting a total of 21 algorithms for evaluation. In this paper, we provide a detailed analysis of the results from both a technical and clinical perspective. All participants relied on deep learning methods, mainly U-Nets, with some variability present in the network architecture, optimization, and image pre- and post-processing. The majority of teams used existing medical imaging deep learning frameworks. The main differences between the submissions were the fine tuning done during training, and the specific pre- and post-processing steps performed. The challenge results showed that almost all submissions performed similarly. Four of the top five teams used ensemble learning methods. However, one team's algorithm performed significantly superior to the other submissions, and consisted of an asymmetrical U-Net network architecture. This paper provides a first of its kind benchmark for future automatic multi-tissue segmentation algorithms for the developing human brain in utero.
Collapse
Affiliation(s)
- Kelly Payette
- Center for MR Research, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich, Zurich, Switzerland.
| | - Hongwei Bran Li
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland; Department of Informatics, Technical University of Munich, Munich, Germany
| | - Priscille de Dumast
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; CIBM, Center for Biomedical Imaging, Lausanne, Switzerland
| | - Roxane Licandro
- Laboratory for Computational Neuroimaging, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital/Harvard Medical School, Charlestown, MA, United States; Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab (CIR), Medical University of Vienna, Vienna, Austria
| | - Hui Ji
- Center for MR Research, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich, Zurich, Switzerland
| | | | | | | | - Hao Liu
- Shanghai Jiaotong University, China
| | | | | | - Ying Peng
- School of Computer Science, Shaanxi Normal University, Xi'an 710119, China
| | - Juanying Xie
- School of Computer Science, Shaanxi Normal University, Xi'an 710119, China
| | - Huiquan Zhang
- School of Computer Science, Shaanxi Normal University, Xi'an 710119, China
| | - Guiming Dong
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Hao Fu
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Guotai Wang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - ZunHyan Rieu
- Research Institute, NEUROPHET Inc., Seoul 06247, South Korea
| | - Donghyeon Kim
- Research Institute, NEUROPHET Inc., Seoul 06247, South Korea
| | - Hyun Gi Kim
- Department of Radiology, The Catholic University of Korea, Eunpyeong St. Mary's Hospital, Seoul 06247, South Korea
| | - Davood Karimi
- Boston Children's Hospital and Harvard Medical School, Boston, MA, United States
| | - Ali Gholipour
- Boston Children's Hospital and Harvard Medical School, Boston, MA, United States
| | - Helena R Torres
- Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal; Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal; ICVS/3B's - PT Government Associate Laboratory, Braga Guimarães, Portugal
| | - Bruno Oliveira
- Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal; Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal; ICVS/3B's - PT Government Associate Laboratory, Braga Guimarães, Portugal
| | - João L Vilaça
- 2Ai - School of Technology, IPCA, Barcelos, Portugal
| | - Yang Lin
- Department of Computer Science, Hong Kong University of Science and Technology, China
| | - Netanell Avisdris
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel; Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Israel
| | - Ori Ben-Zvi
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Israel; Sagol School of Neuroscience, Tel Aviv University, Israel
| | - Dafna Ben Bashat
- Sagol School of Neuroscience, Tel Aviv University, Israel; Sackler Faculty of Medicine, Tel Aviv University, Israel
| | - Lucas Fidon
- School of Biomedical Engineering & Imaging Sciences, King's College London, London SE1 7EU, United Kingdom
| | - Michael Aertsen
- Department of Radiology, University Hospitals Leuven, Leuven 3000, Belgium
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King's College London, London SE1 7EU, United Kingdom
| | - Daniel Sobotka
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Georg Langs
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Mireia Alenyà
- BCN-MedTech, Department of Information and Communications Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Maria Inmaculada Villanueva
- Department of Information and Communications Technologies, Universitat Pompeu Fabra, Barcelona, Spain; Institut d'Investigacions Biomèdiques August Pi i Sunyer, Barcelona, Spain
| | - Oscar Camara
- BCN-MedTech, Department of Information and Communications Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Bella Specktor Fadida
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Liao Weibin
- School of Computer Science, Beijing Institute of Technology, China
| | - Lv Yi
- School of Computer Science, Beijing Institute of Technology, China
| | - Li Xuesong
- School of Computer Science, Beijing Institute of Technology, China
| | - Moona Mazher
- Department of Computer Engineering and Mathematics, University Rovira i Virgili,Spain
| | | | - Domenec Puig
- Department of Computer Engineering and Mathematics, University Rovira i Virgili,Spain
| | - Hamza Kebiri
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; CIBM, Center for Biomedical Imaging, Lausanne, Switzerland
| | - Zelin Zhang
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Yuquan Campus, Hangzhou, China
| | - Xinyi Xu
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Yuquan Campus, Hangzhou, China
| | - Dan Wu
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Yuquan Campus, Hangzhou, China
| | | | - Yixuan Wu
- Zhejiang University, Hangzhou, China
| | | | - Yunzhi Xu
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Yuquan Campus, Hangzhou, China
| | - Li Zhao
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Yuquan Campus, Hangzhou, China
| | - Lana Vasung
- Division of Newborn Medicine, Department of Pediatrics, Boston Children's Hospital, United States; Department of Pediatrics, Harvard Medical School, United States
| | - Bjoern Menze
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Meritxell Bach Cuadra
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; CIBM, Center for Biomedical Imaging, Lausanne, Switzerland
| | - Andras Jakab
- Center for MR Research, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich, Zurich, Switzerland; University Research Priority Project Adaptive Brain Circuits in Development and Learning (AdaBD), University of Zürich, Zurich, Switzerland
| |
Collapse
|
12
|
Ciceri T, Squarcina L, Pigoni A, Ferro A, Montano F, Bertoldo A, Persico N, Boito S, Triulzi FM, Conte G, Brambilla P, Peruzzo D. Geometric Reliability of Super-Resolution Reconstructed Images from Clinical Fetal MRI in the Second Trimester. Neuroinformatics 2023; 21:549-563. [PMID: 37284977 PMCID: PMC10406722 DOI: 10.1007/s12021-023-09635-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/20/2023] [Indexed: 06/08/2023]
Abstract
Fetal Magnetic Resonance Imaging (MRI) is an important noninvasive diagnostic tool to characterize the central nervous system (CNS) development, significantly contributing to pregnancy management. In clinical practice, fetal MRI of the brain includes the acquisition of fast anatomical sequences over different planes on which several biometric measurements are manually extracted. Recently, modern toolkits use the acquired two-dimensional (2D) images to reconstruct a Super-Resolution (SR) isotropic volume of the brain, enabling three-dimensional (3D) analysis of the fetal CNS.We analyzed 17 fetal MR exams performed in the second trimester, including orthogonal T2-weighted (T2w) Turbo Spin Echo (TSE) and balanced Fast Field Echo (b-FFE) sequences. For each subject and type of sequence, three distinct high-resolution volumes were reconstructed via NiftyMIC, MIALSRTK, and SVRTK toolkits. Fifteen biometric measurements were assessed both on the acquired 2D images and SR reconstructed volumes, and compared using Passing-Bablok regression, Bland-Altman plot analysis, and statistical tests.Results indicate that NiftyMIC and MIALSRTK provide reliable SR reconstructed volumes, suitable for biometric assessments. NiftyMIC also improves the operator intraclass correlation coefficient on the quantitative biometric measures with respect to the acquired 2D images. In addition, TSE sequences lead to more robust fetal brain reconstructions against intensity artifacts compared to b-FFE sequences, despite the latter exhibiting more defined anatomical details.Our findings strengthen the adoption of automatic toolkits for fetal brain reconstructions to perform biometry evaluations of fetal brain development over common clinical MR at an early pregnancy stage.
Collapse
Affiliation(s)
- Tommaso Ciceri
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
- Department of Information Engineering, University of Padua, Padua, Italy
| | - Letizia Squarcina
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | - Alessandro Pigoni
- Social and Affective Neuroscience Group, IMT School for Advanced Studies Lucca, Lucca, Italy
- Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Adele Ferro
- Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Florian Montano
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| | - Alessandra Bertoldo
- Department of Information Engineering, University of Padua, Padua, Italy
- Padova Neuroscience Center, University of Padua, Padua, Italy
| | - Nicola Persico
- Department of Woman, Child and Newborn, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Simona Boito
- Department of Woman, Child and Newborn, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Fabio Maria Triulzi
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
- Department of Services and Preventive Medicine, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Giorgio Conte
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
- Department of Services and Preventive Medicine, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Paolo Brambilla
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy.
- Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.
| | - Denis Peruzzo
- NeuroImaging Laboratory, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Italy
| |
Collapse
|
13
|
Costanzo A, Ertl-Wagner B, Sussman D. AFNet Algorithm for Automatic Amniotic Fluid Segmentation from Fetal MRI. Bioengineering (Basel) 2023; 10:783. [PMID: 37508809 PMCID: PMC10376488 DOI: 10.3390/bioengineering10070783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 06/25/2023] [Accepted: 06/27/2023] [Indexed: 07/30/2023] Open
Abstract
Amniotic Fluid Volume (AFV) is a crucial fetal biomarker when diagnosing specific fetal abnormalities. This study proposes a novel Convolutional Neural Network (CNN) model, AFNet, for segmenting amniotic fluid (AF) to facilitate clinical AFV evaluation. AFNet was trained and tested on a manually segmented and radiologist-validated AF dataset. AFNet outperforms ResUNet++ by using efficient feature mapping in the attention block and transposing convolutions in the decoder. Our experimental results show that AFNet achieved a mean Intersection over Union (mIoU) of 93.38% on our dataset, thereby outperforming other state-of-the-art models. While AFNet achieves performance scores similar to those of the UNet++ model, it does so while utilizing merely less than half the number of parameters. By creating a detailed AF dataset with an improved CNN architecture, we enable the quantification of AFV in clinical practice, which can aid in diagnosing AF disorders during gestation.
Collapse
Affiliation(s)
- Alejo Costanzo
- Department of Electrical, Computer and Biomedical Engineering, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University, Toronto, ON M5B 2K3, Canada
- Institute for Biomedical Engineering, Science and Technology (iBEST), Toronto Metropolitan University and St. Michael's Hospital, Toronto, ON M5B 1T8, Canada
| | - Birgit Ertl-Wagner
- Department of Diagnostic Imaging, The Hospital for Sick Children, Toronto, ON M5G 1X8, Canada
- Department of Medical Imaging, Faculty of Medicine, University of Toronto, Toronto, ON M5T 1W7, Canada
| | - Dafna Sussman
- Department of Electrical, Computer and Biomedical Engineering, Faculty of Engineering and Architectural Sciences, Toronto Metropolitan University, Toronto, ON M5B 2K3, Canada
- Institute for Biomedical Engineering, Science and Technology (iBEST), Toronto Metropolitan University and St. Michael's Hospital, Toronto, ON M5B 1T8, Canada
- Department of Obstetrics and Gynecology, Faculty of Medicine, University of Toronto, Toronto, ON M5G 1E2, Canada
| |
Collapse
|
14
|
Vahedifard F, Adepoju JO, Supanich M, Ai HA, Liu X, Kocak M, Marathu KK, Byrd SE. Review of deep learning and artificial intelligence models in fetal brain magnetic resonance imaging. World J Clin Cases 2023; 11:3725-3735. [PMID: 37383127 PMCID: PMC10294149 DOI: 10.12998/wjcc.v11.i16.3725] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 01/30/2023] [Accepted: 05/06/2023] [Indexed: 06/02/2023] Open
Abstract
Central nervous system abnormalities in fetuses are fairly common, happening in 0.1% to 0.2% of live births and in 3% to 6% of stillbirths. So initial detection and categorization of fetal Brain abnormalities are critical. Manually detecting and segmenting fetal brain magnetic resonance imaging (MRI) could be time-consuming, and susceptible to interpreter experience. Artificial intelligence (AI) algorithms and machine learning approaches have a high potential for assisting in the early detection of these problems, improving the diagnosis process and follow-up procedures. The use of AI and machine learning techniques in fetal brain MRI was the subject of this narrative review paper. Using AI, anatomic fetal brain MRI processing has investigated models to predict specific landmarks and segmentation automatically. All gestation age weeks (17-38 wk) and different AI models (mainly Convolutional Neural Network and U-Net) have been used. Some models' accuracy achieved 95% and more. AI could help preprocess and post-process fetal images and reconstruct images. Also, AI can be used for gestational age prediction (with one-week accuracy), fetal brain extraction, fetal brain segmentation, and placenta detection. Some fetal brain linear measurements, such as Cerebral and Bone Biparietal Diameter, have been suggested. Classification of brain pathology was studied using diagonal quadratic discriminates analysis, K-nearest neighbor, random forest, naive Bayes, and radial basis function neural network classifiers. Deep learning methods will become more powerful as more large-scale, labeled datasets become available. Having shared fetal brain MRI datasets is crucial because there aren not many fetal brain pictures available. Also, physicians should be aware of AI's function in fetal brain MRI, particularly neuroradiologists, general radiologists, and perinatologists.
Collapse
Affiliation(s)
- Farzan Vahedifard
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| | - Jubril O Adepoju
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| | - Mark Supanich
- Division for Diagnostic Medical Physics, Department of Radiology and Nuclear Medicine, Rush University Medical Center, Chicago, IL 606012, United States
| | - Hua Asher Ai
- Division for Diagnostic Medical Physics, Department of Radiology and Nuclear Medicine, Rush University Medical Center, Chicago, IL 606012, United States
| | - Xuchu Liu
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| | - Mehmet Kocak
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| | - Kranthi K Marathu
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| | - Sharon E Byrd
- Department of Diagnostic Radiology and Nuclear Medicine, Rush Medical College, Chicago, IL 606012, United States
| |
Collapse
|
15
|
Haq KT, Roberts A, Berk F, Allen S, Swift LM, Posnack NG. KairoSight-3.0 : A Validated Optical Mapping Software to Characterize Cardiac Electrophysiology, Excitation-Contraction Coupling, and Alternans. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.01.538926. [PMID: 37205349 PMCID: PMC10187248 DOI: 10.1101/2023.05.01.538926] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Background Cardiac optical mapping is an imaging technique that measures fluorescent signals across a cardiac preparation. Dual optical mapping of voltage-sensitive and calcium-sensitive probes allow for simultaneous recordings of cardiac action potentials and intracellular calcium transients with high spatiotemporal resolution. The analysis of these complex optical datasets is both time intensive and technically challenging; as such, we have developed a software package for semi-automated image processing and analysis. Herein, we report an updated version of our software package ( KairoSight-3 . 0 ) with features to enhance characterization of cardiac parameters using optical signals. Methods To test software validity and applicability, we used Langendorff-perfused heart preparations to record transmembrane voltage and intracellular calcium signals from the epicardial surface. Isolated hearts from guinea pigs and rats were loaded with a potentiometric dye (RH237) and/or calcium indicator dye (Rhod-2AM) and fluorescent signals were acquired. We used Python 3.8.5 programming language to develop the KairoSight-3 . 0 software. Cardiac maps were validated with a user-specified manual mapping approach. Results Manual maps of action potential duration (30 or 80% repolarization), calcium transient duration (30 or 80% reuptake), action potential and calcium transient alternans were constituted to validate the accuracy of software-generated maps. Manual and software maps had high accuracy, with >97% of manual and software values falling within 10 ms of each other and >75% within 5 ms for action potential duration and calcium transient duration measurements (n=1000-2000 pixels). Further, our software package includes additional cardiac metric measurement tools to analyze signal-to-noise ratio, conduction velocity, action potential and calcium transient alternans, and action potential-calcium transient coupling time to produce physiologically meaningful optical maps. Conclusions KairoSight-3 . 0 has enhanced capabilities to perform measurements of cardiac electrophysiology, calcium handling, and the excitation-contraction coupling with satisfactory accuracy. Graphical Abstract Demonstrating Experimental and Data Analysis Workflow Created with Biorender.com.
Collapse
|
16
|
Uus AU, Kyriakopoulou V, Makropoulos A, Fukami-Gartner A, Cromb D, Davidson A, Cordero-Grande L, Price AN, Grigorescu I, Williams LZJ, Robinson EC, Lloyd D, Pushparajah K, Story L, Hutter J, Counsell SJ, Edwards AD, Rutherford MA, Hajnal JV, Deprez M. BOUNTI: Brain vOlumetry and aUtomated parcellatioN for 3D feTal MRI. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.18.537347. [PMID: 37131820 PMCID: PMC10153133 DOI: 10.1101/2023.04.18.537347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Fetal MRI is widely used for quantitative brain volumetry studies. However, currently, there is a lack of universally accepted protocols for fetal brain parcellation and segmentation. Published clinical studies tend to use different segmentation approaches that also reportedly require significant amounts of time-consuming manual refinement. In this work, we propose to address this challenge by developing a new robust deep learning-based fetal brain segmentation pipeline for 3D T2w motion corrected brain images. At first, we defined a new refined brain tissue parcellation protocol with 19 regions-of-interest using the new fetal brain MRI atlas from the Developing Human Connectome Project. This protocol design was based on evidence from histological brain atlases, clear visibility of the structures in individual subject 3D T2w images and the clinical relevance to quantitative studies. It was then used as a basis for developing an automated deep learning brain tissue parcellation pipeline trained on 360 fetal MRI datasets with different acquisition parameters using semi-supervised approach with manually refined labels propagated from the atlas. The pipeline demonstrated robust performance for different acquisition protocols and GA ranges. Analysis of tissue volumetry for 390 normal participants (21-38 weeks gestational age range), scanned with three different acquisition protocols, did not reveal significant differences for major structures in the growth charts. Only minor errors were present in < 15% of cases thus significantly reducing the need for manual refinement. In addition, quantitative comparison between 65 fetuses with ventriculomegaly and 60 normal control cases were in agreement with the findings reported in our earlier work based on manual segmentations. These preliminary results support the feasibility of the proposed atlas-based deep learning approach for large-scale volumetric analysis. The created fetal brain volumetry centiles and a docker with the proposed pipeline are publicly available online at https://hub.docker.com/r/fetalsvrtk/segmentation (tag brain_bounti_tissue).
Collapse
Affiliation(s)
- Alena U Uus
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | | | | | | | - Daniel Cromb
- Centre for the Developing Brain, King's College London, London, UK
| | - Alice Davidson
- Centre for the Developing Brain, King's College London, London, UK
| | - Lucilio Cordero-Grande
- Centre for the Developing Brain, King's College London, London, UK
- Biomedical Image Technologies, ETSI Telecomunicacion, Universidad Politécnica de Madrid and CIBER-BBN, ISCII, Madrid, Spain
| | - Anthony N Price
- Centre for the Developing Brain, King's College London, London, UK
| | - Irina Grigorescu
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | - Logan Z J Williams
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | - Emma C Robinson
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | - David Lloyd
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Department of Congenital Heart Disease, Evelina London Children's Hospital, London, UK
| | - Kuberan Pushparajah
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Department of Congenital Heart Disease, Evelina London Children's Hospital, London, UK
| | - Lisa Story
- Centre for the Developing Brain, King's College London, London, UK
| | - Jana Hutter
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | | | - A David Edwards
- Centre for the Developing Brain, King's College London, London, UK
| | | | - Joseph V Hajnal
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | - Maria Deprez
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| |
Collapse
|
17
|
de Vareilles H, Rivière D, Mangin JF, Dubois J. Development of cortical folds in the human brain: An attempt to review biological hypotheses, early neuroimaging investigations and functional correlates. Dev Cogn Neurosci 2023; 61:101249. [PMID: 37141790 DOI: 10.1016/j.dcn.2023.101249] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 03/28/2023] [Accepted: 04/21/2023] [Indexed: 05/06/2023] Open
Abstract
The folding of the human brain mostly takes place in utero, making it challenging to study. After a few pioneer studies looking into it in post-mortem foetal specimen, modern approaches based on neuroimaging have allowed the community to investigate the folding process in vivo, its normal progression, its early disturbances, and its relationship to later functional outcomes. In this review article, we aimed to first give an overview of the current hypotheses on the mechanisms governing cortical folding. After describing the methodological difficulties raised by its study in fetuses, neonates and infants with magnetic resonance imaging (MRI), we reported our current understanding of sulcal pattern emergence in the developing brain. We then highlighted the functional relevance of early sulcal development, through recent insights about hemispheric asymmetries and early factors influencing this dynamic such as prematurity. Finally, we outlined how longitudinal studies have started to relate early folding markers and the child's sensorimotor and cognitive outcome. Through this review, we hope to raise awareness on the potential of studying early sulcal patterns both from a fundamental and clinical perspective, as a window into early neurodevelopment and plasticity in relation to growth in utero and postnatal environment of the child.
Collapse
Affiliation(s)
- H de Vareilles
- Université Paris-Saclay, NeuroSpin-BAOBAB, CEA, CNRS, Gif-sur-Yvette, France.
| | - D Rivière
- Université Paris-Saclay, NeuroSpin-BAOBAB, CEA, CNRS, Gif-sur-Yvette, France
| | - J F Mangin
- Université Paris-Saclay, NeuroSpin-BAOBAB, CEA, CNRS, Gif-sur-Yvette, France
| | - J Dubois
- Université Paris Cité, NeuroDiderot, Inserm, Paris, France; Université Paris-Saclay, NeuroSpin-UNIACT, CEA, Gif-sur-Yvette, France
| |
Collapse
|
18
|
Karimi D, Rollins CK, Velasco-Annis C, Ouaalam A, Gholipour A. Learning to segment fetal brain tissue from noisy annotations. Med Image Anal 2023; 85:102731. [PMID: 36608414 PMCID: PMC9974964 DOI: 10.1016/j.media.2022.102731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Revised: 11/17/2022] [Accepted: 12/23/2022] [Indexed: 01/03/2023]
Abstract
Automatic fetal brain tissue segmentation can enhance the quantitative assessment of brain development at this critical stage. Deep learning methods represent the state of the art in medical image segmentation and have also achieved impressive results in brain segmentation. However, effective training of a deep learning model to perform this task requires a large number of training images to represent the rapid development of the transient fetal brain structures. On the other hand, manual multi-label segmentation of a large number of 3D images is prohibitive. To address this challenge, we segmented 272 training images, covering 19-39 gestational weeks, using an automatic multi-atlas segmentation strategy based on deformable registration and probabilistic atlas fusion, and manually corrected large errors in those segmentations. Since this process generated a large training dataset with noisy segmentations, we developed a novel label smoothing procedure and a loss function to train a deep learning model with smoothed noisy segmentations. Our proposed methods properly account for the uncertainty in tissue boundaries. We evaluated our method on 23 manually-segmented test images of a separate set of fetuses. Results show that our method achieves an average Dice similarity coefficient of 0.893 and 0.916 for the transient structures of younger and older fetuses, respectively. Our method generated results that were significantly more accurate than several state-of-the-art methods including nnU-Net that achieved the closest results to our method. Our trained model can serve as a valuable tool to enhance the accuracy and reproducibility of fetal brain analysis in MRI.
Collapse
Affiliation(s)
- Davood Karimi
- Department of Radiology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
| | - Caitlin K Rollins
- Department of Neurology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | - Clemente Velasco-Annis
- Department of Radiology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | - Abdelhakim Ouaalam
- Department of Radiology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | - Ali Gholipour
- Department of Radiology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
19
|
Wahid KA, Lin D, Sahin O, Cislo M, Nelms BE, He R, Naser MA, Duke S, Sherer MV, Christodouleas JP, Mohamed ASR, Murphy JD, Fuller CD, Gillespie EF. Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites. Sci Data 2023; 10:161. [PMID: 36949088 PMCID: PMC10033824 DOI: 10.1038/s41597-023-02062-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 03/10/2023] [Indexed: 03/24/2023] Open
Abstract
Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.
Collapse
Affiliation(s)
- Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Diana Lin
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Onur Sahin
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael Cislo
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Mohammed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Simon Duke
- Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, UK
| | - Michael V Sherer
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | - John P Christodouleas
- Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA, USA
- Elekta, Atlanta, GA, USA
| | - Abdallah S R Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - James D Murphy
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA.
| | - Erin F Gillespie
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
- Fred Hutchinson Cancer Center, Seattle, WA, USA.
| |
Collapse
|
20
|
Urru A, Nakaki A, Benkarim O, Crovetto F, Segalés L, Comte V, Hahner N, Eixarch E, Gratacos E, Crispi F, Piella G, González Ballester MA. An automatic pipeline for atlas-based fetal and neonatal brain segmentation and analysis. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 230:107334. [PMID: 36682108 DOI: 10.1016/j.cmpb.2023.107334] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 11/29/2022] [Accepted: 01/02/2023] [Indexed: 06/17/2023]
Abstract
BACKGROUND AND OBJECTIVE The automatic segmentation of perinatal brain structures in magnetic resonance imaging (MRI) is of utmost importance for the study of brain growth and related complications. While different methods exist for adult and pediatric MRI data, there is a lack for automatic tools for the analysis of perinatal imaging. METHODS In this work, a new pipeline for fetal and neonatal segmentation has been developed. We also report the creation of two new fetal atlases, and their use within the pipeline for atlas-based segmentation, based on novel registration methods. The pipeline is also able to extract cortical and pial surfaces and compute features, such as curvature, local gyrification index, sulcal depth, and thickness. RESULTS Results show that the introduction of the new templates together with our segmentation strategy leads to accurate results when compared to expert annotations, as well as better performances when compared to a reference pipeline (developing Human Connectome Project (dHCP)), for both early and late-onset fetal brains. CONCLUSIONS These findings show the potential of the presented atlases and the whole pipeline for application in both fetal, neonatal, and longitudinal studies, which could lead to dramatic improvements in the understanding of perinatal brain development.
Collapse
Affiliation(s)
- Andrea Urru
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Ayako Nakaki
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain; Institut d'Investigacions Biomédiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Oualid Benkarim
- McConnell Brain Imaging Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Francesca Crovetto
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain
| | - Laura Segalés
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain
| | - Valentin Comte
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Nadine Hahner
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain; Institut d'Investigacions Biomédiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Elisenda Eixarch
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain; Institut d'Investigacions Biomédiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Centre for Biomedical Research on Rare Diseases (CIBERER), Barcelona, Spain
| | - Eduard Gratacos
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain; Institut d'Investigacions Biomédiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Centre for Biomedical Research on Rare Diseases (CIBERER), Barcelona, Spain
| | - Fàtima Crispi
- BCNatal | Fetal Medicine Research Center (Hospital Clínic and Hospital Sant Joan de Déu), University of Barcelona, Barcelona, Spain; Institut d'Investigacions Biomédiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Centre for Biomedical Research on Rare Diseases (CIBERER), Barcelona, Spain
| | - Gemma Piella
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Miguel A González Ballester
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain; ICREA, Barcelona, Spain.
| |
Collapse
|
21
|
Huang X, Liu Y, Li Y, Qi K, Gao A, Zheng B, Liang D, Long X. Deep Learning-Based Multiclass Brain Tissue Segmentation in Fetal MRIs. SENSORS (BASEL, SWITZERLAND) 2023; 23:655. [PMID: 36679449 PMCID: PMC9862805 DOI: 10.3390/s23020655] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 12/31/2022] [Accepted: 01/03/2023] [Indexed: 06/17/2023]
Abstract
Fetal brain tissue segmentation is essential for quantifying the presence of congenital disorders in the developing fetus. Manual segmentation of fetal brain tissue is cumbersome and time-consuming, so using an automatic segmentation method can greatly simplify the process. In addition, the fetal brain undergoes a variety of changes throughout pregnancy, such as increased brain volume, neuronal migration, and synaptogenesis. In this case, the contrast between tissues, especially between gray matter and white matter, constantly changes throughout pregnancy, increasing the complexity and difficulty of our segmentation. To reduce the burden of manual refinement of segmentation, we proposed a new deep learning-based segmentation method. Our approach utilized a novel attentional structural block, the contextual transformer block (CoT-Block), which was applied in the backbone network model of the encoder-decoder to guide the learning of dynamic attentional matrices and enhance image feature extraction. Additionally, in the last layer of the decoder, we introduced a hybrid dilated convolution module, which can expand the receptive field and retain detailed spatial information, effectively extracting the global contextual information in fetal brain MRI. We quantitatively evaluated our method according to several performance measures: dice, precision, sensitivity, and specificity. In 80 fetal brain MRI scans with gestational ages ranging from 20 to 35 weeks, we obtained an average Dice similarity coefficient (DSC) of 83.79%, an average Volume Similarity (VS) of 84.84%, and an average Hausdorff95 Distance (HD95) of 35.66 mm. We also used several advanced deep learning segmentation models for comparison under equivalent conditions, and the results showed that our method was superior to other methods and exhibited an excellent segmentation performance.
Collapse
Affiliation(s)
- Xiaona Huang
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Department of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yang Liu
- Shenzhen Maternity and Child Healthcare Hospital, Shenzhen 518027, China
| | - Yuhan Li
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Department of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Keying Qi
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Department of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Ang Gao
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Bowen Zheng
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Dong Liang
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Xiaojing Long
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| |
Collapse
|
22
|
Uus AU, van Poppel MPM, Steinweg JK, Grigorescu I, Ramirez Gilliland P, Roberts TA, Egloff Collado A, Rutherford MA, Hajnal JV, Lloyd DFA, Pushparajah K, Deprez M. 3D black blood cardiovascular magnetic resonance atlases of congenital aortic arch anomalies and the normal fetal heart: application to automated multi-label segmentation. J Cardiovasc Magn Reson 2022; 24:71. [PMID: 36517850 PMCID: PMC9753334 DOI: 10.1186/s12968-022-00902-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Accepted: 11/09/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Image-domain motion correction of black-blood contrast T2-weighted fetal cardiovascular magnetic resonance imaging (CMR) using slice-to-volume registration (SVR) provides high-resolution three-dimensional (3D) images of the fetal heart providing excellent 3D visualisation of vascular anomalies [1]. However, 3D segmentation of these datasets, important for both clinical reporting and the application of advanced analysis techniques is currently a time-consuming process requiring manual input with potential for inter-user variability. METHODS In this work, we present novel 3D fetal CMR population-averaged atlases of normal and abnormal fetal cardiovascular anatomy. The atlases are created using motion-corrected 3D reconstructed volumes of 86 third trimester fetuses (gestational age range 29-34 weeks) including: 28 healthy controls, 20 cases with postnatally confirmed neonatal coarctation of the aorta (CoA) and 38 vascular rings (21 right aortic arch (RAA), 17 double aortic arch (DAA)). We used only high image quality datasets with isolated anomalies and without any other deviations in the cardiovascular anatomy.In addition, we implemented and evaluated atlas-guided registration and deep learning (UNETR) methods for automated 3D multi-label segmentation of fetal cardiac vessels. We used images from CoA, RAA and DAA cohorts including: 42 cases for training (14 from each cohort), 3 for validation and 6 for testing. In addition, the potential limitations of the network were investigated on unseen datasets including 3 early gestational age (22 weeks) and 3 low SNR cases. RESULTS We created four atlases representing the average anatomy of the normal fetal heart, postnatally confirmed neonatal CoA, RAA and DAA. Visual inspection was undertaken to verify expected anatomy per subgroup. The results of the multi-label cardiac vessel UNETR segmentation showed 100[Formula: see text] per-vessel detection rate for both normal and abnormal aortic arch anatomy. CONCLUSIONS This work introduces the first set of 3D black-blood T2-weighted CMR atlases of normal and abnormal fetal cardiovascular anatomy including detailed segmentation of the major cardiovascular structures. Additionally, we demonstrated the general feasibility of using deep learning for multi-label vessel segmentation of 3D fetal CMR images.
Collapse
Affiliation(s)
- Alena U Uus
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK.
| | - Milou P M van Poppel
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Department of Congenital Heart Disease, Evelina London Children's Hospital, London, UK
| | - Johannes K Steinweg
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | - Irina Grigorescu
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| | | | - Thomas A Roberts
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Clinical Scientific Computing, Guy's and St Thomas' NHS Foundation Trust, London, UK
| | | | | | - Joseph V Hajnal
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Centre for the Developing Brain, King's College London, London, UK
| | - David F A Lloyd
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Department of Congenital Heart Disease, Evelina London Children's Hospital, London, UK
| | - Kuberan Pushparajah
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
- Department of Congenital Heart Disease, Evelina London Children's Hospital, London, UK
| | - Maria Deprez
- School of Imaging Sciences and Biomedical Engineering, King's College London, London, UK
| |
Collapse
|
23
|
Mazher M, Qayyum A, Puig D, Abdel-Nasser M. Effective Approaches to Fetal Brain Segmentation in MRI and Gestational Age Estimation by Utilizing a Multiview Deep Inception Residual Network and Radiomics. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1708. [PMID: 36554113 PMCID: PMC9778347 DOI: 10.3390/e24121708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 11/16/2022] [Accepted: 11/18/2022] [Indexed: 06/17/2023]
Abstract
To completely comprehend neurodevelopment in healthy and congenitally abnormal fetuses, quantitative analysis of the human fetal brain is essential. This analysis requires the use of automatic multi-tissue fetal brain segmentation techniques. This paper proposes an end-to-end automatic yet effective method for a multi-tissue fetal brain segmentation model called IRMMNET. It includes a inception residual encoder block (EB) and a dense spatial attention (DSAM) block, which facilitate the extraction of multi-scale fetal-brain-tissue-relevant information from multi-view MRI images, enhance the feature reuse, and substantially reduce the number of parameters of the segmentation model. Additionally, we propose three methods for predicting gestational age (GA)-GA prediction by using a 3D autoencoder, GA prediction using radiomics features, and GA prediction using the IRMMNET segmentation model's encoder. Our experiments were performed on a dataset of 80 pathological and non-pathological magnetic resonance fetal brain volume reconstructions across a range of gestational ages (20 to 33 weeks) that were manually segmented into seven different tissue categories. The results showed that the proposed fetal brain segmentation model achieved a Dice score of 0.791±0.18, outperforming the state-of-the-art methods. The radiomics-based GA prediction methods achieved the best results (RMSE: 1.42). We also demonstrated the generalization capabilities of the proposed methods for tasks such as head and neck tumor segmentation and the prediction of patients' survival days.
Collapse
Affiliation(s)
- Moona Mazher
- Departament d’Enginyeria Informatica i Matemátiques, Universitat Rovira i Virgili, 43007 Tarragona, Spain
| | - Abdul Qayyum
- School of Biomedical Engineering and Imaging Sciences, Kings College London, London SE1 9RT, UK
| | - Domenec Puig
- Departament d’Enginyeria Informatica i Matemátiques, Universitat Rovira i Virgili, 43007 Tarragona, Spain
| | - Mohamed Abdel-Nasser
- Electronics and Communication Engineering Section, Electrical Engineering Department, Aswan University, Aswan 81528, Egypt
| |
Collapse
|
24
|
Wang J, Nichols ES, Mueller ME, de Vrijer B, Eagleson R, McKenzie CA, de Ribaupierre S, Duerden EG. Semi-automatic segmentation of the fetal brain from magnetic resonance imaging. Front Neurosci 2022; 16:1027084. [PMID: 36440277 PMCID: PMC9692018 DOI: 10.3389/fnins.2022.1027084] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 10/19/2022] [Indexed: 11/13/2023] Open
Abstract
BACKGROUND Volumetric measurements of fetal brain maturation in the third trimester of pregnancy are key predictors of developmental outcomes. Improved understanding of fetal brain development trajectories may aid in identifying and clinically managing at-risk fetuses. Currently, fetal brain structures in magnetic resonance images (MRI) are often manually segmented, which requires both time and expertise. To facilitate the targeting and measurement of brain structures in the fetus, we compared the results of five segmentation methods applied to fetal brain MRI data to gold-standard manual tracings. METHODS Adult women with singleton pregnancies (n = 21), of whom five were scanned twice, approximately 3 weeks apart, were recruited [26 total datasets, median gestational age (GA) = 34.8, IQR = 30.9-36.6]. T2-weighted single-shot fast spin echo images of the fetal brain were acquired on 1.5T and 3T MRI scanners. Images were first combined into a single 3D anatomical volume. Next, a trained tracer manually segmented the thalamus, cerebellum, and total cerebral volumes. The manual segmentations were compared with five automatic methods of segmentation available within Advanced Normalization Tools (ANTs) and FMRIB's Linear Image Registration Tool (FLIRT) toolboxes. The manual and automatic labels were compared using Dice similarity coefficients (DSCs). The DSC values were compared using Friedman's test for repeated measures. RESULTS Comparing cerebellum and thalamus masks against the manually segmented masks, the median DSC values for ANTs and FLIRT were 0.72 [interquartile range (IQR) = 0.6-0.8] and 0.54 (IQR = 0.4-0.6), respectively. A Friedman's test indicated that the ANTs registration methods, primarily nonlinear methods, performed better than FLIRT (p < 0.001). CONCLUSION Deformable registration methods provided the most accurate results relative to manual segmentation. Overall, this semi-automatic subcortical segmentation method provides reliable performance to segment subcortical volumes in fetal MR images. This method reduces the costs of manual segmentation, facilitating the measurement of typical and atypical fetal brain development.
Collapse
Affiliation(s)
- Jianan Wang
- Biomedical Engineering, Western University, London, ON, Canada
| | - Emily S. Nichols
- Applied Psychology, Faculty of Education, Western University, London, ON, Canada
- Western Institute for Neuroscience, Western University, London, ON, Canada
| | - Megan E. Mueller
- Applied Psychology, Faculty of Education, Western University, London, ON, Canada
| | - Barbra de Vrijer
- Department of Obstetrics and Gynaecology, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Roy Eagleson
- Biomedical Engineering, Western University, London, ON, Canada
- Western Institute for Neuroscience, Western University, London, ON, Canada
- Department of Electrical and Computer Engineering, Western University, London, ON, Canada
| | - Charles A. McKenzie
- Department of Medical Biophysics, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Sandrine de Ribaupierre
- Biomedical Engineering, Western University, London, ON, Canada
- Western Institute for Neuroscience, Western University, London, ON, Canada
- Department of Clinical Neurological Sciences, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
- Department of Anatomy and Cell Biology, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Emma G. Duerden
- Biomedical Engineering, Western University, London, ON, Canada
- Applied Psychology, Faculty of Education, Western University, London, ON, Canada
- Western Institute for Neuroscience, Western University, London, ON, Canada
| |
Collapse
|
25
|
Xu J, Moyer D, Grant PE, Golland P, Iglesias JE, Adalsteinsson E. SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2022; 13436:3-13. [PMID: 37103480 PMCID: PMC10129054 DOI: 10.1007/978-3-031-16446-0_1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/28/2023]
Abstract
Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multiple stacks of MR slices as a sequence. With the attention mechanism, our model automatically detects the relevance between slices and predicts the transformation of one slice using information from other slices. We also estimate the underlying 3D volume to assist slice-to-volume registration and update the volume and transformations alternately to improve accuracy. Results on synthetic data show that our method achieves lower registration error and better reconstruction quality compared with existing state-of-the-art methods. Experiments with real-world MRI data are also performed to demonstrate the ability of the proposed model to improve the quality of 3D reconstruction under severe fetal motion.
Collapse
Affiliation(s)
- Junshen Xu
- Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA
| | - Daniel Moyer
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - P Ellen Grant
- Fetal-Neonatal Neuroimaging and Developmental Science Center, Boston Children's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Polina Golland
- Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - Juan Eugenio Iglesias
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
- Harvard Medical School, Boston, MA, USA
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, UK
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, USA
| | - Elfar Adalsteinsson
- Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA
- Institute for Medical Engineering and Science, MIT, Cambridge, MA, USA
| |
Collapse
|
26
|
Fidon L, Viola E, Mufti N, David AL, Melbourne A, Demaerel P, Ourselin S, Vercauteren T, Deprest J, Aertsen M. A spatio-temporal atlas of the developing fetal brain with spina bifida aperta. OPEN RESEARCH EUROPE 2022; 1:123. [PMID: 37645096 PMCID: PMC10445840 DOI: 10.12688/openreseurope.13914.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/25/2022] [Indexed: 08/31/2023]
Abstract
Background: Spina bifida aperta (SBA) is a birth defect associated with severe anatomical changes in the developing fetal brain. Brain magnetic resonance imaging (MRI) atlases are popular tools for studying neuropathology in the brain anatomy, but previous fetal brain MRI atlases have focused on the normal fetal brain. We aimed to develop a spatio-temporal fetal brain MRI atlas for SBA. Methods: We developed a semi-automatic computational method to compute the first spatio-temporal fetal brain MRI atlas for SBA. We used 90 MRIs of fetuses with SBA with gestational ages ranging from 21 to 35 weeks. Isotropic and motion-free 3D reconstructed MRIs were obtained for all the examinations. We propose a protocol for the annotation of anatomical landmarks in brain 3D MRI of fetuses with SBA with the aim of making spatial alignment of abnormal fetal brain MRIs more robust. In addition, we propose a weighted generalized Procrustes method based on the anatomical landmarks for the initialization of the atlas. The proposed weighted generalized Procrustes can handle temporal regularization and missing annotations. After initialization, the atlas is refined iteratively using non-linear image registration based on the image intensity and the anatomical land-marks. A semi-automatic method is used to obtain a parcellation of our fetal brain atlas into eight tissue types: white matter, ventricular system, cerebellum, extra-axial cerebrospinal fluid, cortical gray matter, deep gray matter, brainstem, and corpus callosum. Results: An intra-rater variability analysis suggests that the seven anatomical land-marks are sufficiently reliable. We find that the proposed atlas outperforms a normal fetal brain atlas for the automatic segmentation of brain 3D MRI of fetuses with SBA. Conclusions: We make publicly available a spatio-temporal fetal brain MRI atlas for SBA, available here: https://doi.org/10.7303/syn25887675. This atlas can support future research on automatic segmentation methods for brain 3D MRI of fetuses with SBA.
Collapse
Affiliation(s)
- Lucas Fidon
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
| | - Elizabeth Viola
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
| | - Nada Mufti
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
- Elizabeth Garrett Anderson Institute for Women’s Health, University College London, London, WC1E 6DB, UK
| | - Anna L. David
- Elizabeth Garrett Anderson Institute for Women’s Health, University College London, London, WC1E 6DB, UK
- Department of Obstetrics and Gynaecology, University Hospitals Leuven, 3000 Leuven, Belgium
| | - Andrew Melbourne
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
| | - Philippe Demaerel
- Department of Radiology, University Hospitals Leuven, 3000 Leuven, Belgium
| | - Sébastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King’s College London, London, SE1 7EU, UK
| | - Jan Deprest
- Elizabeth Garrett Anderson Institute for Women’s Health, University College London, London, WC1E 6DB, UK
- Department of Obstetrics and Gynaecology, University Hospitals Leuven, 3000 Leuven, Belgium
- Department of Radiology, University Hospitals Leuven, 3000 Leuven, Belgium
| | - Michael Aertsen
- Department of Radiology, University Hospitals Leuven, 3000 Leuven, Belgium
| |
Collapse
|
27
|
Uus AU, Egloff Collado A, Roberts TA, Hajnal JV, Rutherford MA, Deprez M. Retrospective motion correction in foetal MRI for clinical applications: existing methods, applications and integration into clinical practice. Br J Radiol 2022:20220071. [PMID: 35834425 DOI: 10.1259/bjr.20220071] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023] Open
Abstract
Foetal MRI is a complementary imaging method to antenatal ultrasound. It provides advanced information for detection and characterisation of foetal brain and body anomalies. Even though modern single shot sequences allow fast acquisition of 2D slices with high in-plane image quality, foetal MRI is intrinsically corrupted by motion. Foetal motion leads to loss of structural continuity and corrupted 3D volumetric information in stacks of slices. Furthermore, the arbitrary and constantly changing position of the foetus requires dynamic readjustment of acquisition planes during scanning.
Collapse
Affiliation(s)
- Alena U Uus
- Department of Biomedical Engineering, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom
| | - Alexia Egloff Collado
- Centre for the Developing Brain, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom
| | - Thomas A Roberts
- Department of Biomedical Engineering, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom.,Clinical Scientific Computing, Guy's and St Thomas' NHS Foundation Trust, London, United Kingdom
| | - Joseph V Hajnal
- Department of Biomedical Engineering, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom.,Centre for the Developing Brain, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom
| | - Mary A Rutherford
- Centre for the Developing Brain, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom
| | - Maria Deprez
- Department of Biomedical Engineering, School Biomedical Engineering and Imaging Sciences, King's College London, St. Thomas' Hospital, London, United Kingdom
| |
Collapse
|
28
|
A Fetal Brain magnetic resonance Acquisition Numerical phantom (FaBiAN). Sci Rep 2022; 12:8682. [PMID: 35606398 PMCID: PMC9127105 DOI: 10.1038/s41598-022-10335-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 04/05/2022] [Indexed: 11/28/2022] Open
Abstract
Accurate characterization of in utero human brain maturation is critical as it involves complex and interconnected structural and functional processes that may influence health later in life. Magnetic resonance imaging is a powerful tool to investigate equivocal neurological patterns during fetal development. However, the number of acquisitions of satisfactory quality available in this cohort of sensitive subjects remains scarce, thus hindering the validation of advanced image processing techniques. Numerical phantoms can mitigate these limitations by providing a controlled environment with a known ground truth. In this work, we present FaBiAN, an open-source Fetal Brain magnetic resonance Acquisition Numerical phantom that simulates clinical T2-weighted fast spin echo sequences of the fetal brain. This unique tool is based on a general, flexible and realistic setup that includes stochastic fetal movements, thus providing images of the fetal brain throughout maturation comparable to clinical acquisitions. We demonstrate its value to evaluate the robustness and optimize the accuracy of an algorithm for super-resolution fetal brain magnetic resonance imaging from simulated motion-corrupted 2D low-resolution series compared to a synthetic high-resolution reference volume. We also show that the images generated can complement clinical datasets to support data-intensive deep learning methods for fetal brain tissue segmentation.
Collapse
|
29
|
Hesse LS, Aliasi M, Moser F, theINTERGROWTH-Twenty First Consortium, Haak MC, Xie W, Jenkinson M, Namburete AIL. Subcortical Segmentation of the Fetal Brain in 3D Ultrasound using Deep Learning. Neuroimage 2022; 254:119117. [PMID: 35331871 DOI: 10.1016/j.neuroimage.2022.119117] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 02/25/2022] [Accepted: 03/17/2022] [Indexed: 12/24/2022] Open
Abstract
The quantification of subcortical volume development from 3D fetal ultrasound can provide important diagnostic information during pregnancy monitoring. However, manual segmentation of subcortical structures in ultrasound volumes is time-consuming and challenging due to low soft tissue contrast, speckle and shadowing artifacts. For this reason, we developed a convolutional neural network (CNN) for the automated segmentation of the choroid plexus (CP), lateral posterior ventricle horns (LPVH), cavum septum pellucidum et vergae (CSPV), and cerebellum (CB) from 3D ultrasound. As ground-truth labels are scarce and expensive to obtain, we applied few-shot learning, in which only a small number of manual annotations (n = 9) are used to train a CNN. We compared training a CNN with only a few individually annotated volumes versus many weakly labelled volumes obtained from atlas-based segmentations. This showed that segmentation performance close to intra-observer variability can be obtained with only a handful of manual annotations. Finally, the trained models were applied to a large number (n = 278) of ultrasound image volumes of a diverse, healthy population, obtaining novel US-specific growth curves of the respective structures during the second trimester of gestation.
Collapse
Affiliation(s)
- Linde S Hesse
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, United Kingdom.
| | - Moska Aliasi
- Department of Obstetrics and Fetal Medicine, Leiden University Medical Center, The Netherlands
| | - Felipe Moser
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, United Kingdom
| | - theINTERGROWTH-Twenty First Consortium
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, United Kingdom; Department of Obstetrics and Fetal Medicine, Leiden University Medical Center, The Netherlands; Visual Geometry Group, Department of Engineering Science, University of Oxford, United Kingdom; Wellcome center for Integrative NeuroImaging, FMRIB, University of Oxford, United Kingdom; Australian Institute for Machine Learning (AIML), Australia; South Australian Health and Medical Research Institute (SAHMRI), Australia
| | - Monique C Haak
- Department of Obstetrics and Fetal Medicine, Leiden University Medical Center, The Netherlands
| | - Weidi Xie
- Visual Geometry Group, Department of Engineering Science, University of Oxford, United Kingdom
| | - Mark Jenkinson
- Wellcome center for Integrative NeuroImaging, FMRIB, University of Oxford, United Kingdom; Australian Institute for Machine Learning (AIML), Australia; South Australian Health and Medical Research Institute (SAHMRI), Australia
| | - Ana I L Namburete
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, United Kingdom
| |
Collapse
|
30
|
Zhao L, Asis-Cruz JD, Feng X, Wu Y, Kapse K, Largent A, Quistorff J, Lopez C, Wu D, Qing K, Meyer C, Limperopoulos C. Automated 3D Fetal Brain Segmentation Using an Optimized Deep Learning Approach. AJNR Am J Neuroradiol 2022; 43:448-454. [PMID: 35177547 PMCID: PMC8910820 DOI: 10.3174/ajnr.a7419] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 12/06/2021] [Indexed: 01/01/2023]
Abstract
BACKGROUND AND PURPOSE MR imaging provides critical information about fetal brain growth and development. Currently, morphologic analysis primarily relies on manual segmentation, which is time-intensive and has limited repeatability. This work aimed to develop a deep learning-based automatic fetal brain segmentation method that provides improved accuracy and robustness compared with atlas-based methods. MATERIALS AND METHODS A total of 106 fetal MR imaging studies were acquired prospectively from fetuses between 23 and 39 weeks of gestation. We trained a deep learning model on the MR imaging scans of 65 healthy fetuses and compared its performance with a 4D atlas-based segmentation method using the Wilcoxon signed-rank test. The trained model was also evaluated on data from 41 fetuses diagnosed with congenital heart disease. RESULTS The proposed method showed high consistency with the manual segmentation, with an average Dice score of 0.897. It also demonstrated significantly improved performance (P < .001) based on the Dice score and 95% Hausdorff distance in all brain regions compared with the atlas-based method. The performance of the proposed method was consistent across gestational ages. The segmentations of the brains of fetuses with high-risk congenital heart disease were also highly consistent with the manual segmentation, though the Dice score was 7% lower than that of healthy fetuses. CONCLUSIONS The proposed deep learning method provides an efficient and reliable approach for fetal brain segmentation, which outperformed segmentation based on a 4D atlas and has been used in clinical and research settings.
Collapse
Affiliation(s)
- L Zhao
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
- Department of Biomedical Engineering (L.Z., D.W.), Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering & Instrument Science, Zhejiang University, China
| | - J D Asis-Cruz
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - X Feng
- Department of Biomedical Engineering (X.F., C.M.), University of Virginia, Charlottesville, Virginia
| | - Y Wu
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - K Kapse
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - A Largent
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - J Quistorff
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - C Lopez
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| | - D Wu
- Department of Biomedical Engineering (L.Z., D.W.), Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering & Instrument Science, Zhejiang University, China
| | - K Qing
- Department of Radiation Oncology (K.Q.), City of Hope National Center, Duarte, California
| | - C Meyer
- Department of Biomedical Engineering (X.F., C.M.), University of Virginia, Charlottesville, Virginia
| | - C Limperopoulos
- From the Department of Diagnostic Imaging and Radiology (L.Z., J.D.A.-C., Y.W., K.K., A.L., J.Q., C. Lopez, C. Limperopoulos), Developing Brain Institute, Children's National, Washington, DC
| |
Collapse
|
31
|
Nenning KH, Langs G. Machine learning in neuroimaging: from research to clinical practice. RADIOLOGIE (HEIDELBERG, GERMANY) 2022; 62:1-10. [PMID: 36044070 PMCID: PMC9732070 DOI: 10.1007/s00117-022-01051-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 07/07/2022] [Indexed: 12/14/2022]
Abstract
Neuroimaging is critical in clinical care and research, enabling us to investigate the brain in health and disease. There is a complex link between the brain's morphological structure, physiological architecture, and the corresponding imaging characteristics. The shape, function, and relationships between various brain areas change during development and throughout life, disease, and recovery. Like few other areas, neuroimaging benefits from advanced analysis techniques to fully exploit imaging data for studying the brain and its function. Recently, machine learning has started to contribute (a) to anatomical measurements, detection, segmentation, and quantification of lesions and disease patterns, (b) to the rapid identification of acute conditions such as stroke, or (c) to the tracking of imaging changes over time. As our ability to image and analyze the brain advances, so does our understanding of its intricate relationships and their role in therapeutic decision-making. Here, we review the current state of the art in using machine learning techniques to exploit neuroimaging data for clinical care and research, providing an overview of clinical applications and their contribution to fundamental computational neuroscience.
Collapse
Affiliation(s)
- Karl-Heinz Nenning
- Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA
- Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria
| | - Georg Langs
- Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria.
| |
Collapse
|