A Deep Attentive Convolutional Neural Network for Automatic Cortical Plate Segmentation in Fetal MRI

Multi-Center Fetal Brain Tissue Annotation (FeTA) Challenge 2022 Results

Segmentation is a critical step in analyzing the developing human fetal brain. There have been vast improvements in automatic segmentation methods in the past several years, and the Fetal Brain Tissue Annotation (FeTA) Challenge 2021 helped to establish an excellent standard of fetal brain segmentation. However, FeTA 2021 was a single center study, limiting real-world clinical applicability and acceptance. The multi-center FeTA Challenge 2022 focused on advancing the generalizability of fetal brain segmentation algorithms for magnetic resonance imaging (MRI). In FeTA 2022, the training dataset contained images and corresponding manually annotated multi-class labels from two imaging centers, and the testing data contained images from these two centers as well as two additional unseen centers. The multi-center data included different MR scanners, imaging parameters, and fetal brain super-resolution algorithms applied. 16 teams participated and 17 algorithms were evaluated. Here, the challenge results are presented, focusing on the generalizability of the submissions. Both in- and out-of-domain, the white matter and ventricles were segmented with the highest accuracy (Top Dice scores: 0.89, 0.87 respectively), while the most challenging structure remains the grey matter (Top Dice score: 0.75) due to anatomical complexity. The top 5 average Dices scores ranged from 0.81-0.82, the top 5 average percentile Hausdorff distance values ranged from 2.3-2.5mm, and the top 5 volumetric similarity scores ranged from 0.90-0.92. The FeTA Challenge 2022 was able to successfully evaluate and advance generalizability of multi-class fetal brain tissue segmentation algorithms for MRI and it continues to benchmark new algorithms.

Anatomically constrained tractography of the fetal brain

Diffusion-weighted Magnetic Resonance Imaging (dMRI) is increasingly used to study the fetal brain in utero. An important computation enabled by dMRI is streamline tractography, which has unique applications such as tract-specific analysis of the brain white matter and structural connectivity assessment. However, due to the low fetal dMRI data quality and the challenging nature of tractography, existing methods tend to produce highly inaccurate results. They generate many false streamlines while failing to reconstruct the streamlines that constitute the major white matter tracts. In this paper, we advocate for anatomically constrained tractography based on an accurate segmentation of the fetal brain tissue directly in the dMRI space. We develop a deep learning method to compute the segmentation automatically. Experiments on independent test data show that this method can accurately segment the fetal brain tissue and drastically improve the tractography results. It enables the reconstruction of highly curved tracts such as optic radiations. Importantly, our method infers the tissue segmentation and streamline propagation direction from a diffusion tensor fit to the dMRI data, making it applicable to routine fetal dMRI scans. The proposed method can facilitate the study of fetal brain white matter tracts with dMRI.

Advances in Fetal Brain Imaging

Over the last 20 years, there have been remarkable developments in fetal brain MR imaging analysis methods. This article delves into the specifics of structural imaging, diffusion imaging, functional MR imaging, and spectroscopy, highlighting the latest advancements in motion correction, fetal brain development atlases, and the challenges and innovations. Furthermore, this article explores the clinical applications of these advanced imaging techniques in comprehending and diagnosing fetal brain development and abnormalities.

A deep learning framework for intracranial aneurysms automatic segmentation and detection on magnetic resonance T1 images

OBJECTIVES

To design a deep learning-based framework for automatic segmentation and detection of intracranial aneurysms (IAs) on magnetic resonance T1 images and test the robustness and performance of framework.

METHODS

A retrospective diagnostic study was conducted based on 159 IAs from 136 patients who underwent the T1 images. Among them, 127 cases were randomly selected for training and validation, and 32 cases were used to assess the accuracy and consistency of our algorithm. We developed and assembled three convolutional neural networks for the segmentation and detection of IAs. The segmentation and detection performance of the model were compared with the ground truth, and various metrics were calculated at the voxel level, IAs level, and patient level to show the performance of our framework.

RESULTS

Our assembled model achieved overall Dice, voxel-level sensitivity, specificity, balanced accuracy, and F1 score of 0.802, 0.874, 0.9998, 0.937, and 0.802, respectively. A coincidence greater than 0.7 between the aneurysms predicted by the model and the ground truth was considered as a true positive. For IAs detection, the sensitivity reached 90.63% with 0.58 false positives per case. The volume of IAs segmented by our model showed a high agreement and consistency with the volume of IAs labeled by experts.

CONCLUSION

The deep learning framework is achievable and robust for IAs segmentation and detection. Our model offers more clinical application opportunities compared to digital subtraction angiography (DSA)-based, CTA-based, and MRA-based methods.

CLINICAL RELEVANCE STATEMENT

Our deep learning framework effectively detects and segments intracranial aneurysms using clinical routine T1 sequences, showing remarkable effectiveness and offering great potential for improving the detection of latent intracranial aneurysms and enabling early identification.

KEY POINTS

•There is no segmentation method based on clinical routine T1 images. Our study shows that the proper deep learning framework can effectively localize the intracranial aneurysms. •The T1-based segmentation and detection method is more universal than other angiography-based detection methods, which can potentially reduce missed diagnoses caused by the absence of angiography images. •The deep learning framework is robust and has the potential to be applied in a clinical setting.

The nnU-Net based method for automatic segmenting fetal brain tissues

The magnetic resonance (MR) images of fetuses make it possible for doctors to detect out pathological fetal brains in early stages. Brain tissue segmentation is prerequisite for making brain morphology and volume analyses. nnU-Net is an automatic segmentation method based on deep learning. It can adaptively configure itself, so as to adapt to a specific task via preprocessing, network architecture, training, and post-processing. Therefore, we adapt nnU-Net to segment seven types of fetal brain tissues, including external cerebrospinal fluid, gray matter, white matter, ventricle, cerebellum, deep gray matter, and brainstem. With regard to the characteristics of the FeTA 2021 data, some adjustments are made to the original nnU-Net, so that it can segment seven types of fetal brain tissues precisely as far as possible. The average segmentation results on FeTA 2021 training data demonstrate that our advanced nnU-Net is superior to the peers including SegNet, CoTr, AC U-Net and ResUnet. Its average segmentation results are 0.842, 11.759 and 0.957 in terms of Dice, HD95 and VS criteria. Moreover, the experimental results on FeTA 2021 test data further demonstrate that our advanced nnU-Net has obtained good segmentation performance of 0.774, 14.699 and 0.875 in terms of Dice, HD95 and VS, ranked the third in FeTA 2021 challenge. Our advanced nnU-Net realized the task for segmenting the fetal brain tissues using MR images of different gestational ages, which can help doctors to make correct and seasonable diagnoses.

Automatic Localization of Key Structures for Subthalamic Nucleus-Deep Brain Stimulation Surgery via Prior-Enhanced Multi-Object Magnetic Resonance Imaging Segmentation

BACKGROUND

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an established and effective neurosurgical treatment for relieving motor symptoms in Parkinson disease. The localization of key brain structures is critical to the success of DBS surgery. However, in clinical practice, this process is heavily dependent on the radiologist's experience.

METHODS

In this study, we propose an automatic localization method of key structures for STN-DBS surgery via prior-enhanced multi-object magnetic resonance imaging segmentation. We use the U-Net architecture for the multi-object segmentation, including STN, red nucleus, brain sulci, gyri, and ventricles. To address the challenge that only half of the brain sulci and gyri locate in the upper area, potentially causing interference in the lower area, we perform region of interest detection and ensemble joint processing to enhance the segmentation performance of brain sulci and gyri.

RESULTS

We evaluate the segmentation accuracy by comparing our method with other state-of-the-art machine learning segmentation methods. The experimental results show that our approach outperforms state-of-the-art methods in terms of segmentation performance. Moreover, our method provides effective visualization of key brain structures from a clinical application perspective and can reduce the segmentation time compared with manual delineation.

CONCLUSIONS

Our proposed method uses deep learning to achieve accurate segmentation of the key structures more quickly than and with comparable accuracy to human manual segmentation. Our method has the potential to improve the efficiency of surgical planning for STN-DBS.

Review on deep learning fetal brain segmentation from Magnetic Resonance images

Brain segmentation is often the first and most critical step in quantitative analysis of the brain for many clinical applications, including fetal imaging. Different aspects challenge the segmentation of the fetal brain in magnetic resonance imaging (MRI), such as the non-standard position of the fetus owing to his/her movements during the examination, rapid brain development, and the limited availability of imaging data. In recent years, several segmentation methods have been proposed for automatically partitioning the fetal brain from MR images. These algorithms aim to define regions of interest with different shapes and intensities, encompassing the entire brain, or isolating specific structures. Deep learning techniques, particularly convolutional neural networks (CNNs), have become a state-of-the-art approach in the field because they can provide reliable segmentation results over heterogeneous datasets. Here, we review the deep learning algorithms developed in the field of fetal brain segmentation and categorize them according to their target structures. Finally, we discuss the perceived research gaps in the literature of the fetal domain, suggesting possible future research directions that could impact the management of fetal MR images.

Efficient Multi-Organ Segmentation From 3D Abdominal CT Images With Lightweight Network and Knowledge Distillation

Accurate segmentation of multiple abdominal organs from Computed Tomography (CT) images plays an important role in computer-aided diagnosis, treatment planning and follow-up. Currently, 3D Convolution Neural Networks (CNN) have achieved promising performance for automatic medical image segmentation tasks. However, most existing 3D CNNs have a large set of parameters and huge floating point operations (FLOPs), and 3D CT volumes have a large size, leading to high computational cost, which limits their clinical application. To tackle this issue, we propose a novel framework based on lightweight network and Knowledge Distillation (KD) for delineating multiple organs from 3D CT volumes. We first propose a novel lightweight medical image segmentation network named LCOV-Net for reducing the model size and then introduce two knowledge distillation modules (i.e., Class-Affinity KD and Multi-Scale KD) to effectively distill the knowledge from a heavy-weight teacher model to improve LCOV-Net's segmentation accuracy. Experiments on two public abdominal CT datasets for multiple organ segmentation showed that: 1) Our LCOV-Net outperformed existing lightweight 3D segmentation models in both computational cost and accuracy; 2) The proposed KD strategy effectively improved the performance of the lightweight network, and it outperformed existing KD methods; 3) Combining the proposed LCOV-Net and KD strategy, our framework achieved better performance than the state-of-the-art 3D nnU-Net with only one-fifth parameters. The code is available at https://github.com/HiLab-git/LCOVNet-and-KD.

Transformers in medical imaging: A survey

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as de facto operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, restoration, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging.

Retrospective motion correction in foetal MRI for clinical applications: existing methods, applications and integration into clinical practice

Foetal MRI is a complementary imaging method to antenatal ultrasound. It provides advanced information for detection and characterisation of foetal brain and body anomalies. Even though modern single shot sequences allow fast acquisition of 2D slices with high in-plane image quality, foetal MRI is intrinsically corrupted by motion. Foetal motion leads to loss of structural continuity and corrupted 3D volumetric information in stacks of slices. Furthermore, the arbitrary and constantly changing position of the foetus requires dynamic readjustment of acquisition planes during scanning.

Artificial intelligence applied to fetal MRI: A scoping review of current research

Artificial intelligence (AI) is defined as the development of computer systems to perform tasks normally requiring human intelligence. A subset of AI, known as machine learning (ML), takes this further by drawing inferences from patterns in data to 'learn' and 'adapt' without explicit instructions meaning that computer systems can 'evolve' and hopefully improve without necessarily requiring external human influences. The potential for this novel technology has resulted in great interest from the medical community regarding how it can be applied in healthcare. Within radiology, the focus has mostly been for applications in oncological imaging, although new roles in other subspecialty fields are slowly emerging.In this scoping review, we performed a literature search of the current state-of-the-art and emerging trends for the use of artificial intelligence as applied to fetal magnetic resonance imaging (MRI). Our search yielded several publications covering AI tools for anatomical organ segmentation, improved imaging sequences and aiding in diagnostic applications such as automated biometric fetal measurements and the detection of congenital and acquired abnormalities. We highlight our own perceived gaps in this literature and suggest future avenues for further research. It is our hope that the information presented highlights the varied ways and potential that novel digital technology could make an impact to future clinical practice with regards to fetal MRI.

BOUNTI: Brain vOlumetry and aUtomated parcellatioN for 3D feTal MRI

Fetal MRI is widely used for quantitative brain volumetry studies. However, currently, there is a lack of universally accepted protocols for fetal brain parcellation and segmentation. Published clinical studies tend to use different segmentation approaches that also reportedly require significant amounts of time-consuming manual refinement. In this work, we propose to address this challenge by developing a new robust deep learning-based fetal brain segmentation pipeline for 3D T2w motion corrected brain images. At first, we defined a new refined brain tissue parcellation protocol with 19 regions-of-interest using the new fetal brain MRI atlas from the Developing Human Connectome Project. This protocol design was based on evidence from histological brain atlases, clear visibility of the structures in individual subject 3D T2w images and the clinical relevance to quantitative studies. It was then used as a basis for developing an automated deep learning brain tissue parcellation pipeline trained on 360 fetal MRI datasets with different acquisition parameters using semi-supervised approach with manually refined labels propagated from the atlas. The pipeline demonstrated robust performance for different acquisition protocols and GA ranges. Analysis of tissue volumetry for 390 normal participants (21-38 weeks gestational age range), scanned with three different acquisition protocols, did not reveal significant differences for major structures in the growth charts. Only minor errors were present in < 15% of cases thus significantly reducing the need for manual refinement. In addition, quantitative comparison between 65 fetuses with ventriculomegaly and 60 normal control cases were in agreement with the findings reported in our earlier work based on manual segmentations. These preliminary results support the feasibility of the proposed atlas-based deep learning approach for large-scale volumetric analysis. The created fetal brain volumetry centiles and a docker with the proposed pipeline are publicly available online at https://hub.docker.com/r/fetalsvrtk/segmentation (tag brain_bounti_tissue).

Demystifying evidential Dempster Shafer-based CNN architecture for fetal plane detection from 2D ultrasound images leveraging fuzzy-contrast enhancement and explainable AI

Ultrasound imaging is a valuable tool for assessing the development of the fetal during pregnancy. However, interpreting ultrasound images manually can be time-consuming and subject to variability. Automated image categorization using machine learning algorithms can streamline the interpretation process by identifying stages of fetal development present in ultrasound images. In particular, deep learning architectures have shown promise in medical image analysis, enabling accurate automated diagnosis. The objective of this research is to identify fetal planes from ultrasound images with higher precision. To achieve this, we trained several convolutional neural network (CNN) architectures on a dataset of 12400 images. Our study focuses on the impact of enhanced image quality by adopting Histogram Equalization and Fuzzy Logic-based contrast enhancement on fetal plane detection using the Evidential Dempster-Shafer Based CNN Architecture, PReLU-Net, SqueezeNET, and Swin Transformer. The results of each classifier were noteworthy, with PreLUNet achieving an accuracy of 91.03%, SqueezeNET reaching 91.03% accuracy, Swin Transformer reaching an accuracy of 88.90%, and the Evidential classifier achieving an accuracy of 83.54%. We evaluated the results in terms of both training and testing accuracies. Additionally, we used LIME and GradCam to examine the decision-making process of the classifiers, providing explainability for their outputs. Our findings demonstrate the potential for automated image categorization in large-scale retrospective assessments of fetal development using ultrasound imaging.

Learning to segment fetal brain tissue from noisy annotations

Automatic fetal brain tissue segmentation can enhance the quantitative assessment of brain development at this critical stage. Deep learning methods represent the state of the art in medical image segmentation and have also achieved impressive results in brain segmentation. However, effective training of a deep learning model to perform this task requires a large number of training images to represent the rapid development of the transient fetal brain structures. On the other hand, manual multi-label segmentation of a large number of 3D images is prohibitive. To address this challenge, we segmented 272 training images, covering 19-39 gestational weeks, using an automatic multi-atlas segmentation strategy based on deformable registration and probabilistic atlas fusion, and manually corrected large errors in those segmentations. Since this process generated a large training dataset with noisy segmentations, we developed a novel label smoothing procedure and a loss function to train a deep learning model with smoothed noisy segmentations. Our proposed methods properly account for the uncertainty in tissue boundaries. We evaluated our method on 23 manually-segmented test images of a separate set of fetuses. Results show that our method achieves an average Dice similarity coefficient of 0.893 and 0.916 for the transient structures of younger and older fetuses, respectively. Our method generated results that were significantly more accurate than several state-of-the-art methods including nnU-Net that achieved the closest results to our method. Our trained model can serve as a valuable tool to enhance the accuracy and reproducibility of fetal brain analysis in MRI.

Applications of Deep Learning to Neurodevelopment in Pediatric Imaging: Achievements and Challenges

Deep learning has achieved remarkable progress, particularly in neuroimaging analysis. Deep learning applications have also been extended from adult to pediatric medical images, and thus, this paper aims to present a systematic review of this recent research. We first introduce the commonly used deep learning methods and architectures in neuroimaging, such as convolutional neural networks, auto-encoders, and generative adversarial networks. A non-exhaustive list of commonly used publicly available pediatric neuroimaging datasets and repositories are included, followed by a categorical review of recent works in pediatric MRI-based deep learning studies in the past five years. These works are categorized into recognizing neurodevelopmental disorders, identifying brain and tissue structures, estimating brain age/maturity, predicting neurodevelopment outcomes, and optimizing MRI brain imaging and analysis. Finally, we also discuss the recent achievements and challenges on these applications of deep learning to pediatric neuroimaging.

Deep Learning-Based Multiclass Brain Tissue Segmentation in Fetal MRIs

Fetal brain tissue segmentation is essential for quantifying the presence of congenital disorders in the developing fetus. Manual segmentation of fetal brain tissue is cumbersome and time-consuming, so using an automatic segmentation method can greatly simplify the process. In addition, the fetal brain undergoes a variety of changes throughout pregnancy, such as increased brain volume, neuronal migration, and synaptogenesis. In this case, the contrast between tissues, especially between gray matter and white matter, constantly changes throughout pregnancy, increasing the complexity and difficulty of our segmentation. To reduce the burden of manual refinement of segmentation, we proposed a new deep learning-based segmentation method. Our approach utilized a novel attentional structural block, the contextual transformer block (CoT-Block), which was applied in the backbone network model of the encoder-decoder to guide the learning of dynamic attentional matrices and enhance image feature extraction. Additionally, in the last layer of the decoder, we introduced a hybrid dilated convolution module, which can expand the receptive field and retain detailed spatial information, effectively extracting the global contextual information in fetal brain MRI. We quantitatively evaluated our method according to several performance measures: dice, precision, sensitivity, and specificity. In 80 fetal brain MRI scans with gestational ages ranging from 20 to 35 weeks, we obtained an average Dice similarity coefficient (DSC) of 83.79%, an average Volume Similarity (VS) of 84.84%, and an average Hausdorff95 Distance (HD95) of 35.66 mm. We also used several advanced deep learning segmentation models for comparison under equivalent conditions, and the results showed that our method was superior to other methods and exhibited an excellent segmentation performance.

ESA-UNet for assisted diagnosis of cardiac magnetic resonance image based on the semantic segmentation of the heart

Background

Cardiovascular diseases have become the number one disease affecting human health in today's society. In the diagnosis of cardiac diseases, magnetic resonance image (MRI) technology is the most widely used one. However, in clinical diagnosis, the analysis of MRI relies on manual work, which is laborious and time-consuming, and also easily influenced by the subjective experience of doctors.

Methods

In this article, we propose an artificial intelligence-aided diagnosis system for cardiac MRI with image segmentation as the main component to assist in the diagnosis of cardiovascular diseases. We first performed adequate pre-processing of MRI. The pre-processing steps include the detection of regions of interest of cardiac MRI data, as well as data normalization and data enhancement, and then we input the images after data pre-processing into the deep learning network module of ESA-Unet for the identification of the aorta in order to obtain preliminary segmentation results, and finally, the boundaries of the segmentation results are further optimized using conditional random fields. For ROI detection, we first use standard deviation filters for filtering to find regions in the heart cycle image sequence where pixel intensity varies strongly with time and then use Canny edge detection and Hough transform techniques to find the region of interest containing the heart. The ESA-Unet proposed in this article, moreover, is jointly designed with a self-attentive mechanism and multi-scale jump connection based on convolutional networks.

Results

The experimental dataset used in this article is from the Department of CT/MRI at the Second Affiliated Hospital of Fujian Medical University. Experiments compare other convolution-based methods, such as UNet, FCN, FPN, and PSPNet, and the results show that our model achieves the best results on Acc, Pr, ReCall, DSC, and IoU metrics. After comparative analysis, the experimental results show that the ESA-UNet network segmentation model designed in this article has higher accuracy, intuitiveness, and more application value than traditional image segmentation algorithms.

Conclusion

With the continuous application of nuclear magnetic resonance technology in clinical diagnosis, the method in this article is expected to become a tool that can effectively improve the efficiency of doctors' diagnoses.

AGMR-Net: Attention-guided multiscale recovery framework for stroke segmentation

Automatic and accurate lesion segmentation is critical to the clinical estimation of the lesion status of stroke diseases and appropriate diagnostic systems. Although existing methods have achieved remarkable results, their further adoption is hindered by: (1) intraclass inconsistency, i.e., large variability between different areas of the lesion; and (2) interclass indistinction, in which normal brain tissue resembles the lesion in appearance. To meet these challenges in stroke segmentation, we propose a novel method, namely attention-guided multiscale recovery framework (AGMR-Net) in this paper. Firstly, a coarse-grained patch attention (CPA) module in the encoding is adopted to obtain a patch-based coarse-grained attention map in a multistage, explicitly supervised way, enabling target spatial context saliency representation with a patch-based weighting technique that eliminates the effect of intraclass inconsistency. Secondly, to obtain more detailed boundary partitioning to meet the challenge of interclass indistinction, a newly designed cross-dimensional feature fusion (CFF) module is used to capture global contextual information to further guide the selective aggregation of 2D and 3D features, which can compensate for the lack of boundary learning capability of 2D convolution. Lastly, in the decoding stage, an innovative designed multiscale deconvolution upsampling (MDU) is used for enhanced recovery of target spatial and boundary information. AGMR-Net is evaluated on the open-source dataset Anatomical Tracings of Lesions After Stroke, achieving the highest Dice similarity coefficient of 0.594, Hausdorff distance of 27.005 mm, and average symmetry surface distance of 7.137 mm, which demonstrates that our proposed method outperforms state-of-the-art methods and has great potential for stroke diagnosis.

Deep collaborative network with alpha matte for precise knee tissue segmentation from MRI

BACKGROUND AND OBJECTIVE

Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), being state of the art, often challenged by the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results.

METHODS

This paper presents a deep learning-based automatic segmentation framework for precise knee tissue segmentation. A novel deep collaborative method is proposed, which consists of an encoder-decoder-based segmentation network in combination with a low rank tensor-reconstructed segmentation network. Low rank reconstruction in MRI tensor sub-blocks is introduced to exploit the morphological variations in knee tissues. To model the tissue boundary regions and effectively utilize the superimposed regions, trimap generation is proposed for defining high, medium and low confidence regions from the multipath CNNs. The secondary path with low rank reconstructed input mitigates the conditions in which the primary segmentation network can potentially fail and overlook the boundary regions. The outcome of the segmentation is solved as an alpha matting problem by blending the trimap with the source input.

RESULTS

Experiments on Osteoarthritis Initiative (OAI) datasets with all the 6 musculoskeletal tissues provide an overall segmentation dice score of 0.8925, where Femoral and Tibial part of cartilage achieving an average dice exceeding 0.9. The volumetric metrics also indicate the superior performances in all tissue compartments.

CONCLUSIONS

Experiments on Osteoarthritis Initiative (OAI) datasets and a self-prepared scan validate the effectiveness of the proposed method. Inclusion of extra prediction scale allowed the model to distinguish and segment the tissue boundary accurately. We specifically demonstrate the application of the proposed method in a cartilage segmentation-based thickness map for diagnosis purposes.

A Fetal Brain magnetic resonance Acquisition Numerical phantom (FaBiAN)

Accurate characterization of in utero human brain maturation is critical as it involves complex and interconnected structural and functional processes that may influence health later in life. Magnetic resonance imaging is a powerful tool to investigate equivocal neurological patterns during fetal development. However, the number of acquisitions of satisfactory quality available in this cohort of sensitive subjects remains scarce, thus hindering the validation of advanced image processing techniques. Numerical phantoms can mitigate these limitations by providing a controlled environment with a known ground truth. In this work, we present FaBiAN, an open-source Fetal Brain magnetic resonance Acquisition Numerical phantom that simulates clinical T2-weighted fast spin echo sequences of the fetal brain. This unique tool is based on a general, flexible and realistic setup that includes stochastic fetal movements, thus providing images of the fetal brain throughout maturation comparable to clinical acquisitions. We demonstrate its value to evaluate the robustness and optimize the accuracy of an algorithm for super-resolution fetal brain magnetic resonance imaging from simulated motion-corrupted 2D low-resolution series compared to a synthetic high-resolution reference volume. We also show that the images generated can complement clinical datasets to support data-intensive deep learning methods for fetal brain tissue segmentation.

Attention-modulated multi-branch convolutional neural networks for neonatal brain tissue segmentation

Accurate measurement of brain structures is essential for the evaluation of neonatal brain growth and development. The conventional methods use manual segmentation to measure brain tissues, which is very time-consuming and inefficient. Recent deep learning achieves excellent performance in computer vision, but it is still unsatisfactory for segmenting magnetic resonance images of neonatal brains because they are immature with unique attributes. In this paper, we propose a novel attention-modulated multi-branch convolutional neural network for neonatal brain tissue segmentation. The proposed network is built on the encoder-decoder framework by introducing both multi-scale convolutions in the encoding path and multi-branch attention modules in the decoding path. Multi-scale convolutions with different kernels are used to extract rich semantic features across large receptive fields in the encoding path. Multi-branch attention modules are used to capture abundant contextual information in the decoding path for segmenting brain tissues by fusing both local features and their corresponding global dependencies. Spatial attention connections between the encoding and decoding paths are designed to increase feature propagation for both avoiding information loss during downsampling and accelerating model training convergence. The proposed network was implemented in comparison with baseline methods on three neonatal brain datasets. Our network achieves the average Dice similarity coefficients/the average Hausdorff distances of 0.9116/8.1289, 0.9367/9.8212 and 0.8931/8.1612 on the customized dCBP2021 dataset, 0.8786/11.7863, 0.8965/13.4296 and 0.8539/10.462 on the public NBAtlas dataset, as well as 0.9253/7.7968, 0.9448/9.5472 and 0.9132/7.5877 on the public dHCP2017 dataset in partitioning the brain into gray matter, white matter and cerebrospinal fluid, respectively. The experimental results show that the proposed method achieves competitive state-of-the-art performance in neonatal brain tissue segmentation. The code and pre-trained models are available at https://github.com/zhangyongqin/AMCNN.

Medical Image Segmentation Using Transformer Networks

Deep learning models represent the state of the art in medical image segmentation. Most of these models are fully-convolutional networks (FCNs), namely each layer processes the output of the preceding layer with convolution operations. The convolution operation enjoys several important properties such as sparse interactions, parameter sharing, and translation equivariance. Because of these properties, FCNs possess a strong and useful inductive bias for image modeling and analysis. However, they also have certain important shortcomings, such as performing a fixed and pre-determined operation on a test image regardless of its content and difficulty in modeling long-range interactions. In this work we show that a different deep neural network architecture, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve more accurate segmentations than FCNs. Our proposed model is based directly on the transformer network architecture. Given a 3D image block, our network divides it into non-overlapping 3D patches and computes a 1D embedding for each patch. The network predicts the segmentation map for the block based on the self-attention between these patch embeddings. Furthermore, in order to address the common problem of scarcity of labeled medical images, we propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that the proposed model can achieve segmentation accuracies that are better than several state of the art FCN architectures on two datasets. Our proposed network can be trained using only tens of labeled images. Moreover, with the proposed pre-training strategies, our network outperforms FCNs when labeled training data is small.

Weakly Supervised Framework for Cancer Region Detection of Hepatocellular Carcinoma in Whole-Slide Pathologic Images Based on Multiscale Attention Convolutional Neural Network

Visual inspection of hepatocellular carcinoma cancer regions by experienced pathologists in whole-slide images (WSIs) is a challenging, labor-intensive, and time-consuming task because of the large scale and high resolution of WSIs. Therefore, a weakly supervised framework based on a multiscale attention convolutional neural network (MSAN-CNN) was introduced into this process. Herein, patch-based images with image-level normal/tumor annotation (rather than images with pixel-level annotation) were fed into a classification neural network. To further improve the performances of cancer region detection, multiscale attention was introduced into the classification neural network. A total of 100 cases were obtained from The Cancer Genome Atlas and divided into 70 training and 30 testing data sets that were fed into the MSAN-CNN framework. The experimental results showed that this framework significantly outperforms the single-scale detection method according to the area under the curve and accuracy, sensitivity, and specificity metrics. When compared with the diagnoses made by three pathologists, MSAN-CNN performed better than a junior- and an intermediate-level pathologist, and slightly worse than a senior pathologist. Furthermore, MSAN-CNN provided a very fast detection time compared with the pathologists. Therefore, a weakly supervised framework based on MSAN-CNN has great potential to assist pathologists in the fast and accurate detection of cancer regions of hepatocellular carcinoma on WSIs.

Can Deep Learning Hit a Moving Target? A Scoping Review of Its Role to Study Neurological Disorders in Children

Neurological disorders dramatically impact patients of any age population, their families, and societies. Pediatrics are among vulnerable age populations who differently experience the devastating consequences of neurological conditions, such as attention-deficit hyperactivity disorders (ADHD), autism spectrum disorders (ASD), cerebral palsy, concussion, and epilepsy. System-level understanding of these neurological disorders, particularly from the brain networks' dynamic perspective, has led to the significant trend of recent scientific investigations. While a dramatic maturation in the network science application domain is evident, leading to a better understanding of neurological disorders, such rapid utilization for studying pediatric neurological disorders falls behind that of the adult population. Aside from the specific technological needs and constraints in studying neurological disorders in children, the concept of development introduces uncertainty and further complexity topping the existing neurologically driven processes caused by disorders. To unravel these complexities, indebted to the availability of high-dimensional data and computing capabilities, approaches based on machine learning have rapidly emerged a new trend to understand pathways better, accurately diagnose, and better manage the disorders. Deep learning has recently gained an ever-increasing role in the era of health and medical investigations. Thanks to its relatively more minor dependency on feature exploration and engineering, deep learning may overcome the challenges mentioned earlier in studying neurological disorders in children. The current scoping review aims to explore challenges concerning pediatric brain development studies under the constraints of neurological disorders and offer an insight into the potential role of deep learning methodology on such a task with varying and uncertain nature. Along with pinpointing recent advancements, possible research directions are highlighted where deep learning approaches can assist in computationally targeting neurological disorder-related processes and translating them into windows of opportunities for interventions in diagnosis, treatment, and management of neurological disorders in children.