ψ-Net: Stacking Densely Convolutional LSTMs for Sub-Cortical Brain Structure Segmentation

Contrastive Registration for Unsupervised Medical Image Segmentation

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

A Parkinson's disease-related nuclei segmentation network based on CNN-Transformer interleaved encoder with feature fusion

Automatic segmentation of Parkinson's disease (PD) related deep gray matter (DGM) nuclei based on brain magnetic resonance imaging (MRI) is significant in assisting the diagnosis of PD. However, due to the degenerative-induced changes in appearance, low tissue contrast, and tiny DGM nuclei size in elders' brain MRI images, many existing segmentation models are limited in the application. To address these challenges, this paper proposes a PD-related DGM nuclei segmentation network to provide precise prior knowledge for aiding diagnosis PD. The encoder of network is designed as an alternating encoding structure where the convolutional neural network (CNN) captures spatial and depth texture features, while the Transformer complements global position information between DGM nuclei. Moreover, we propose a cascaded channel-spatial-wise block to fuse features extracted by the CNN and Transformer, thereby achieving more precise DGM nuclei segmentation. The decoder incorporates a symmetrical boundary attention module, leveraging the symmetrical structures of bilateral nuclei regions by constructing signed distance maps for symmetric differences, which optimizes segmentation boundaries. Furthermore, we employ a dynamic adaptive region of interests weighted Dice loss to enhance sensitivity towards smaller structures, thereby improving segmentation accuracy. In qualitative analysis, our method achieved optimal average values for PD-related DGM nuclei (DSC: 0.854, IOU: 0.750, HD95: 1.691 mm, ASD: 0.195 mm). Experiments conducted on multi-center clinical datasets and public datasets demonstrate the good generalizability of the proposed method. Furthermore, a volumetric analysis of segmentation results reveals significant differences between HCs and PDs. Our method holds promise for assisting clinicians in the rapid and accurate diagnosis of PD, offering a practical method for the imaging analysis of neurodegenerative diseases.

STF-Net: sparsification transformer coding guided network for subcortical brain structure segmentation

Subcortical brain structure segmentation plays an important role in the diagnosis of neuroimaging and has become the basis of computer-aided diagnosis. Due to the blurred boundaries and complex shapes of subcortical brain structures, labeling these structures by hand becomes a time-consuming and subjective task, greatly limiting their potential for clinical applications. Thus, this paper proposes the sparsification transformer (STF) module for accurate brain structure segmentation. The self-attention mechanism is used to establish global dependencies to efficiently extract the global information of the feature map with low computational complexity. Also, the shallow network is used to compensate for low-level detail information through the localization of convolutional operations to promote the representation capability of the network. In addition, a hybrid residual dilated convolution (HRDC) module is introduced at the bottom layer of the network to extend the receptive field and extract multi-scale contextual information. Meanwhile, the octave convolution edge feature extraction (OCT) module is applied at the skip connections of the network to pay more attention to the edge features of brain structures. The proposed network is trained with a hybrid loss function. The experimental evaluation on two public datasets: IBSR and MALC, shows outstanding performance in terms of objective and subjective quality.

A brain subcortical segmentation tool based on anatomy attentional fusion network for developing macaques

Magnetic Resonance Imaging (MRI) plays a pivotal role in the accurate measurement of brain subcortical structures in macaques, which is crucial for unraveling the complexities of brain structure and function, thereby enhancing our understanding of neurodegenerative diseases and brain development. However, due to significant differences in brain size, structure, and imaging characteristics between humans and macaques, computational tools developed for human neuroimaging studies often encounter obstacles when applied to macaques. In this context, we propose an Anatomy Attentional Fusion Network (AAF-Net), which integrates multimodal MRI data with anatomical constraints in a multi-scale framework to address the challenges posed by the dynamic development, regional heterogeneity, and age-related size variations of the juvenile macaque brain, thus achieving precise subcortical segmentation. Specifically, we generate a Signed Distance Map (SDM) based on the initial rough segmentation of the subcortical region by a network as an anatomical constraint, providing comprehensive information on positions, structures, and morphology. Then we construct AAF-Net to fully fuse the SDM anatomical constraints and multimodal images for refined segmentation. To thoroughly evaluate the performance of our proposed tool, over 700 macaque MRIs from 19 datasets were used in this study. Specifically, we employed two manually labeled longitudinal macaque datasets to develop the tool and complete four-fold cross-validations. Furthermore, we incorporated various external datasets to demonstrate the proposed tool's generalization capabilities and promise in brain development research. We have made this tool available as an open-source resource at https://github.com/TaoZhong11/Macaque_subcortical_segmentation for direct application.

GNN-based structural information to improve DNN-based basal ganglia segmentation in children following early brain lesion

Analyzing the basal ganglia following an early brain lesion is crucial due to their noteworthy role in sensory-motor functions. However, the segmentation of these subcortical structures on MRI is challenging in children and is further complicated by the presence of a lesion. Although current deep neural networks (DNN) perform well in segmenting subcortical brain structures in healthy brains, they lack robustness when faced with lesion variability, leading to structural inconsistencies. Given the established spatial organization of the basal ganglia, we propose enhancing the DNN-based segmentation through post-processing with a graph neural network (GNN). The GNN conducts node classification on graphs encoding both class probabilities and spatial information regarding the regions segmented by the DNN. In this study, we focus on neonatal arterial ischemic stroke (NAIS) in children. The approach is evaluated on both healthy children and children after NAIS using three DNN backbones: U-Net, UNETr, and MSGSE-Net. The results show an improvement in segmentation performance, with an increase in the median Dice score by up to 4% and a reduction in the median Hausdorff distance (HD) by up to 93% for healthy children (from 36.45 to 2.57) and up to 91% for children suffering from NAIS (from 40.64 to 3.50). The performance of the method is compared with atlas-based methods. Severe cases of neonatal stroke result in a decline in performance in the injured hemisphere, without negatively affecting the segmentation of the contra-injured hemisphere. Furthermore, the approach demonstrates resilience to small training datasets, a widespread challenge in the medical field, particularly in pediatrics and for rare pathologies.

SCL-Net: Structured Collaborative Learning for PET/CT Based Tumor Segmentation

Collaborative learning methods for medical image segmentation are often variants of UNet, where the constructions of classifiers depend on each other and their outputs are supervised independently. However, they cannot explicitly ensure that optimizing auxiliary classifier heads leads to improved segmentation of target classifier. To resolve this problem, we propose a structured collaborative learning (SCL) method, which consists of a context-aware structured classifier population generation (CA-SCPG) module, where the feature propagation of the target classifier path is directly enhanced by the outputs of auxiliary classifiers via a light-weighted high-level context-aware dense connection (HLCA-DC) mechanism, and a knowledge-aware structured classifier population supervision (KA-SCPS) module, where the auxiliary classifiers are properly supervised under the guidance of target classifier's segmentations. Specifically, SCL is proposed based on a recurrent-dense-siamese decoder (RDS-Decoder), which consists of multiple siamese-decoder paths. CA-SCPG enhances the feature propagation of the decoder paths by HLCA-DC, which densely reuses previous decoder paths' output predictions to belong to the target classes as inputs to the latter decoder paths. KA-SCPS supervises the classifier heads simultaneously with KA-SCPS loss, which consists of a generalized weighted cross-entropy loss for deep class-imbalanced learning and a novel knowledge-aware Dice loss (KA-DL). KA-DL is a weighted Dice loss broadcasting knowledges learnt by the target classifier to other classifier heads, harmonizing the learning process of the classifier population. Experiments are performed based on PET/CT volumes with malignant melanoma, lymphoma, or lung cancer. Experimental results demonstrate the superiority of our SCL, when compared to the state-of-the-art methods and baselines.

Learning to segment subcortical structures from noisy annotations with a novel uncertainty-reliability aware learning framework

Accurate segmentation of subcortical structures is an important task in quantitative brain image analysis. Convolutional neural networks (CNNs) have achieved remarkable results in medical image segmentation. However, due to the difficulty of acquiring high-quality annotations of brain subcortical structures, learning segmentation networks using noisy annotations is an inevitable topic. A common practice is to select images or pixels with reliable annotations for training, which usually may not make full use of the information from the training samples, thus affecting the performance of the learned segmentation model. To address the above problem, in this work, we propose a novel robust learning method and denote it as uncertainty-reliability awareness learning (URAL), which can make sufficient use of all training pixels. At each training iteration, the proposed method first selects training pixels with reliable annotations from the set of pixels with uncertain network prediction, by utilizing a small clean validation set following a meta-learning paradigm. Meanwhile, we propose the online prototypical soft label correction (PSLC) method to estimate the pseudo-labels of label-unreliable pixels. Then, the segmentation loss of label-reliable pixels and the semi-supervised segmentation loss of label-unreliable pixels are used to calibrate the total segmentation loss. Finally, we propose a category-wise contrastive regularization to learn compact feature representations of all uncertain training pixels. Comprehensive experiments are performed on two publicly available brain MRI datasets. The proposed method achieves the best Dice scores and MHD values on both datasets compared to several recent state-of-the-art methods under all label noise settings. Our code is available at https://github.com/neulxlx/URAL.

EA-Net: Edge-aware network for brain structure segmentation via decoupled high and low frequency features

Automatic brain structure segmentation in Magnetic Resonance Image (MRI) plays an important role in the diagnosis of various neuropsychiatric diseases. However, most existing methods yield unsatisfactory results due to blurred boundaries and complex structures. Improving the segmentation ability requires the model to be explicit about the spatial localization and shape appearance of targets, which correspond to the low-frequency content features and the high-frequency edge features, respectively. Therefore, in this paper, to extract rich edge and content feature representations, we focus on the composition of the feature and utilize a frequency decoupling (FD) block to separate the low-frequency and high-frequency parts of the feature. Further, a novel edge-aware network (EA-Net) is proposed for jointly learning to segment brain structures and detect object edges. First, an encoder-decoder sub-network is utilized to obtain multi-level information from the input MRI, which is then sent to the FD block to complete the frequency separation. Further, we use different mechanisms to optimize both the low-frequency and high-frequency features. Finally, these two parts are fused to generate the final prediction. In particular, we extract the content mask and the edge mask from the optimization feature with different supervisions, which forces the network to learn the boundary features of the object. Extensive experiments are performed on two public brain MRI T1 scan datasets (the IBSR dataset and the MALC dataset) to evaluate the effectiveness of the proposed algorithm. The experiments show that the EA-Net achieves the best performance compared with the state-of-the-art methods, and improves the segmentation DSC score by 1.31% at most compared with the U-Net model and its variants. Moreover, we evaluate the EA-Net under different noise disturbances, and the results demonstrate the robustness and superiority of our method under low-quality noisy MRI. Code is available at https://github.com/huqian999/EA-Net.

ID-Seg: an infant deep learning-based segmentation framework to improve limbic structure estimates

Infant brain magnetic resonance imaging (MRI) is a promising approach for studying early neurodevelopment. However, segmenting small regions such as limbic structures is challenging due to their low inter-regional contrast and high curvature. MRI studies of the adult brain have successfully applied deep learning techniques to segment limbic structures, and similar deep learning models are being leveraged for infant studies. However, these deep learning-based infant MRI segmentation models have generally been derived from small datasets, and may suffer from generalization problems. Moreover, the accuracy of segmentations derived from these deep learning models relative to more standard Expectation-Maximization approaches has not been characterized. To address these challenges, we leveraged a large, public infant MRI dataset (n = 473) and the transfer-learning technique to first pre-train a deep convolutional neural network model on two limbic structures: amygdala and hippocampus. Then we used a leave-one-out cross-validation strategy to fine-tune the pre-trained model and evaluated it separately on two independent datasets with manual labels. We term this new approach the Infant Deep learning SEGmentation Framework (ID-Seg). ID-Seg performed well on both datasets with a mean dice similarity score (DSC) of 0.87, a mean intra-class correlation (ICC) of 0.93, and a mean average surface distance (ASD) of 0.31 mm. Compared to the Developmental Human Connectome pipeline (dHCP) pipeline, ID-Seg significantly improved segmentation accuracy. In a third infant MRI dataset (n = 50), we used ID-Seg and dHCP separately to estimate amygdala and hippocampus volumes and shapes. The estimates derived from ID-seg, relative to those from the dHCP, showed stronger associations with behavioral problems assessed in these infants at age 2. In sum, ID-Seg consistently performed well on two different datasets with an 0.87 DSC, however, multi-site testing and extension for brain regions beyond the amygdala and hippocampus are still needed.

MDAN: Mirror Difference Aware Network for Brain Stroke Lesion Segmentation

Brain stroke lesion segmentation is of great importance for stroke rehabilitation neuroimaging analysis. Due to the large variance of stroke lesion shapes and similarities of tissue intensity distribution, it remains a challenging task. To help detect abnormalities, the anatomical symmetries of brain magnetic resonance (MR) images have been widely used as visual cues for clinical practices. However, most methods do not fully utilize structural symmetry information in brain images segmentation. This paper presents a novel mirror difference aware network (MDAN) for stroke lesion segmentation in an encoder-decoder architecture, aiming at holistically exploiting the symmetries of image features. Specifically, a differential feature augmentation (DFA) module is developed in the encoding path to highlight the semantically pathological asymmetries of the features in abnormalities. In the DFA module, a Siamese contrastive supervised loss is designed to enhance discriminative features, and a mirror position-based difference augmentation (MDA) module is used to further magnify the discrepancy information. Moreover, mirror feature fusion (MFF) modules are applied to fuse and transfer the information both of the original input and the horizontally flipped features to the decoding path. Extensive experiments on the Anatomical Tracings of Lesions After Stroke (ATLAS) dataset show the proposed MDAN outperforms the state-of-the-art methods.

Transductive Transfer Learning for Domain Adaptation in Brain Magnetic Resonance Image Segmentation

Segmentation of brain images from Magnetic Resonance Images (MRI) is an indispensable step in clinical practice. Morphological changes of sub-cortical brain structures and quantification of brain lesions are considered biomarkers of neurological and neurodegenerative disorders and used for diagnosis, treatment planning, and monitoring disease progression. In recent years, deep learning methods showed an outstanding performance in medical image segmentation. However, these methods suffer from generalisability problem due to inter-centre and inter-scanner variabilities of the MRI images. The main objective of the study is to develop an automated deep learning segmentation approach that is accurate and robust to the variabilities in scanner and acquisition protocols. In this paper, we propose a transductive transfer learning approach for domain adaptation to reduce the domain-shift effect in brain MRI segmentation. The transductive scenario assumes that there are sets of images from two different domains: (1) source-images with manually annotated labels; and (2) target-images without expert annotations. Then, the network is jointly optimised integrating both source and target images into the transductive training process to segment the regions of interest and to minimise the domain-shift effect. We proposed to use a histogram loss in the feature level to carry out the latter optimisation problem. In order to demonstrate the benefit of the proposed approach, the method has been tested in two different brain MRI image segmentation problems using multi-centre and multi-scanner databases for: (1) sub-cortical brain structure segmentation; and (2) white matter hyperintensities segmentation. The experiments showed that the segmentation performance of a pre-trained model could be significantly improved by up to 10%. For the first segmentation problem it was possible to achieve a maximum improvement from 0.680 to 0.799 in average Dice Similarity Coefficient (DSC) metric and for the second problem the average DSC improved from 0.504 to 0.602. Moreover, the improvements after domain adaptation were on par or showed better performance compared to the commonly used traditional unsupervised segmentation methods (FIRST and LST), also achieving faster execution time. Taking this into account, this work presents one more step toward the practical implementation of deep learning algorithms into the clinical routine.

MSU-Net: Multi-Scale U-Net for 2D Medical Image Segmentation

Aiming at the limitation of the convolution kernel with a fixed receptive field and unknown prior to optimal network width in U-Net, multi-scale U-Net (MSU-Net) is proposed by us for medical image segmentation. First, multiple convolution sequence is used to extract more semantic features from the images. Second, the convolution kernel with different receptive fields is used to make features more diverse. The problem of unknown network width is alleviated by efficient integration of convolution kernel with different receptive fields. In addition, the multi-scale block is extended to other variants of the original U-Net to verify its universality. Five different medical image segmentation datasets are used to evaluate MSU-Net. A variety of imaging modalities are included in these datasets, such as electron microscopy, dermoscope, ultrasound, etc. Intersection over Union (IoU) of MSU-Net on each dataset are 0.771, 0.867, 0.708, 0.900, and 0.702, respectively. Experimental results show that MSU-Net achieves the best performance on different datasets. Our implementation is available at https://github.com/CN-zdy/MSU_Net.

A Deep Spatial Context Guided Framework for Infant Brain Subcortical Segmentation

Accurate subcortical segmentation of infant brain magnetic resonance (MR) images is crucial for studying early subcortical structural growth patterns and related diseases diagnosis. However, dynamic intensity changes, low tissue contrast, and small subcortical size of infant brain MR images make subcortical segmentation a challenging task. In this paper, we propose a spatial context guided, coarse-to-fine deep convolutional neural network (CNN) based framework for accurate infant subcortical segmentation. At the coarse stage, we propose a signed distance map (SDM) learning UNet (SDM-UNet) to predict SDMs from the original multi-modal images, including T1w, T2w, and T1w/T2w images. By doing this, the spatial context information, including the relative position information across different structures and the shape information of the segmented structures contained in the ground-truth SDMs, is used for supervising the SDM-UNet to remedy the bad influence from the low tissue contrast in infant brain MR images and generate high-quality SDMs. To improve the robustness to outliers, a Correntropy based loss is introduced in SDM-UNet to penalize the difference between the ground-truth SDMs and predicted SDMs in training. At the fine stage, the predicted SDMs, which contains spatial context information of subcortical structures, are combined with the multi-modal images, and then fed into a multi-source and multi-path UNet (M2-UNet) for delivering refined segmentation. We validate our method on an infant brain MR image dataset with 24 scans by evaluating the Dice ratio between our segmentation and the manual delineation. Compared to four state-of-the-art methods, our method consistently achieves better performances in both qualitative and quantitative evaluations.