Deep label fusion: A generalizable hybrid multi-atlas and deep convolutional neural network for medical image segmentation

Bi-Fusion of Structure and Deformation at Multi-Scale for Joint Segmentation and Registration

Medical image segmentation and registration are two fundamental and highly related tasks. However, current works focus on the mutual promotion between the two at the loss function level, ignoring the feature information generated by the encoder-decoder network during the task-specific feature mapping process and the potential inter-task feature relationship. This paper proposes a unified multi-task joint learning framework based on bi-fusion of structure and deformation at multi-scale, called BFM-Net, which simultaneously achieves the segmentation results and deformation field in a single-step estimation. BFM-Net consists of a segmentation subnetwork (SegNet), a registration subnetwork (RegNet), and the multi-task connection module (MTC). The MTC module is used to transfer the latent feature representation between segmentation and registration at multi-scale and link different tasks at the network architecture level, including the spatial attention fusion module (SAF), the multi-scale spatial attention fusion module (MSAF) and the velocity field fusion module (VFF). Extensive experiments on MR, CT and ultrasound images demonstrate the effectiveness of our approach. The MTC module can increase the Dice scores of segmentation and registration by 3.2%, 1.6%, 2.2%, and 6.2%, 4.5%, 3.0%, respectively. Compared with six state-of-the-art algorithms for segmentation and registration, BFM-Net can achieve superior performance in various modal images, fully demonstrating its effectiveness and generalization.

Automatic segmentation of medial temporal lobe subregions in multi-scanner, multi-modality MRI of variable quality

Background

Volumetry of subregions in the medial temporal lobe (MTL) computed from automatic segmentation in MRI can track neurodegeneration in Alzheimer's disease. However, image quality may vary in MRI. Poor quality MR images can lead to unreliable segmentation of MTL subregions. Considering that different MRI contrast mechanisms and field strengths (jointly referred to as "modalities" here) offer distinct advantages in imaging different parts of the MTL, we developed a muti-modality segmentation model using both 7 tesla (7T) and 3 tesla (3T) structural MRI to obtain robust segmentation in poor-quality images.

Method

MRI modalities including 3T T1-weighted, 3T T2-weighted, 7T T1-weighted and 7T T2-weighted (7T-T2w) of 197 participants were collected from a longitudinal aging study at the Penn Alzheimer's Disease Research Center. Among them, 7T-T2w was used as the primary modality, and all other modalities were rigidly registered to the 7T-T2w. A model derived from nnU-Net took these registered modalities as input and outputted subregion segmentation in 7T-T2w space. 7T-T2w images most of which had high quality from 25 selected training participants were manually segmented to train the multi-modality model. Modality augmentation, which randomly replaced certain modalities with Gaussian noise, was applied during training to guide the model to extract information from all modalities. To compare our proposed model with a baseline single-modality model in the full dataset with mixed high/poor image quality, we evaluated the ability of derived volume/thickness measures to discriminate Amyloid+ mild cognitive impairment (A+MCI) and Amyloid- cognitively unimpaired (A-CU) groups, as well as the stability of these measurements in longitudinal data.

Results

The multi-modality model delivered good performance regardless of 7T-T2w quality, while the single-modality model under-segmented subregions in poor-quality images. The multi-modality model generally demonstrated stronger discrimination of A+MCI versus A-CU. Intra-class correlation and Bland-Altman plots demonstrate that the multi-modality model had higher longitudinal segmentation consistency in all subregions while the single-modality model had low consistency in poor-quality images.

Conclusion

The multi-modality MRI segmentation model provides an improved biomarker for neurodegeneration in the MTL that is robust to image quality. It also provides a framework for other studies which may benefit from multimodal imaging.

Medial temporal lobe atrophy patterns in early- versus late-onset amnestic Alzheimer's disease

Background

The medial temporal lobe (MTL) is hypothesized to be relatively spared in early-onset Alzheimer's disease (EOAD). Yet, detailed examination of MTL subfield volumes and drivers of atrophy in amnestic EOAD is lacking.

Methods

BioFINDER-2 participants with memory impairment, abnormal amyloid-β status and tau-PET were included. Forty-one EOAD individuals aged ≤65 years and, as comparison, late-onset AD (LOAD, ≥70 years, n=154) and Aβ-negative cognitively unimpaired controls were included. MTL subregions and biomarkers of (co-)pathologies were measured.

Results

AD groups showed smaller MTL subregions compared to controls. Atrophy patterns were similar across AD groups, although LOAD showed thinner entorhinal cortices compared to EOAD. EOAD showed lower WMH compared to LOAD. No differences in MTL tau-PET or transactive response DNA binding protein 43-proxy positivity was found.

Conclusions

We found in vivo evidence for MTL atrophy in amnestic EOAD and overall similar levels to LOAD of MTL tau pathology and co-pathologies.

Automatic segmentation of dura for quantitative analysis of lumbar stenosis: A deep learning study with 518 CT myelograms

BACKGROUND

The diagnosis of lumbar spinal stenosis (LSS) can be challenging because radicular pain is not often present in the culprit-level localization. Accurate segmentation and quantitative analysis of the lumbar dura on radiographic images are key to the accurate differential diagnosis of LSS. The aim of this study is to develop an automatic dura-contouring tool for radiographic quantification on computed tomography myelogram (CTM) for patients with LSS.

METHODS

A total of 518 CTM cases with or without lumbar stenosis were included in this study. A deep learning (DL) segmentation algorithm 3-dimensional (3D) U-Net was deployed. A total of 210 labeled cases were used to develop the dura-contouring tool, with the ratio of the training, independent testing, and external validation datasets being 150:30:30. The Dice score (DCS) was the primary measure to evaluate the segmentation performance of the 3D U-Net, which was subsequently developed as the dura-contouring tool to segment another unlabeled 308 CTM cases with LSS. Automatic masks of 446 slices on the stenotic levels were then meticulously reviewed and revised by human experts, and the cross-sectional area (CSA) of the dura was compared.

RESULTS

The mean DCS of the 3D U-Net were 0.905 ± 0.080, 0.933 ± 0.018, and 0.928 ± 0.034 in the five-fold cross-validation, the independent testing, and the external validation datasets, respectively. The segmentation performance of the dura-contouring tool was also comparable to that of the second observer (the human expert). With the dura-contouring tool, only 59.0% (263/446) of the automatic masks of the stenotic slices needed to be revised. In the revised cases, there were no significant differences in the dura CSA between automatic masks and corresponding revised masks (p = 0.652). Additionally, a strong correlation of dura CSA was found between the automatic masks and corresponding revised masks (r = 0.805).

CONCLUSIONS

A dura-contouring tool was developed that could automatically segment the dural sac on CTM, and it demonstrated high accuracy and generalization ability. Additionally, the dura-contouring tool has the potential to be applied in patients with LSS because it facilitates the quantification of the dural CSA on stenotic slices.

A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping

Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e.g., brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.

Age-related and amyloid-beta-independent tau deposition and its downstream effects

Amyloid-β (Aβ) is hypothesized to facilitate the spread of tau pathology beyond the medial temporal lobe. However, there is evidence that, independently of Aβ, age-related tau pathology might be present outside of the medial temporal lobe. We therefore aimed to study age-related Aβ-independent tau deposition outside the medial temporal lobe in two large cohorts and to investigate potential downstream effects of this on cognition and structural measures. We included 545 cognitively unimpaired adults (40-92 years) from the BioFINDER-2 study (in vivo) and 639 (64-108 years) from the Rush Alzheimer's Disease Center cohorts (ex vivo). 18F-RO948- and 18F-flutemetamol-PET standardized uptake value ratios were calculated for regional tau and global/regional Aβ in vivo. Immunohistochemistry was used to estimate Aβ load and tangle density ex vivo. In vivo medial temporal lobe volumes (subiculum, cornu ammonis 1) and cortical thickness (entorhinal cortex, Brodmann area 35) were obtained using Automated Segmentation for Hippocampal Subfields packages. Thickness of early and late neocortical Alzheimer's disease regions was determined using FreeSurfer. Global cognition and episodic memory were estimated to quantify cognitive functioning. In vivo age-related tau deposition was observed in the medial temporal lobe and in frontal and parietal cortical regions, which was statistically significant when adjusting for Aβ. This was also observed in individuals with low Aβ load. Tau deposition was negatively associated with cortical volumes and thickness in temporal and parietal regions independently of Aβ. The associations between age and cortical volume or thickness were partially mediated via tau in regions with early Alzheimer's disease pathology, i.e. early tau and/or Aβ pathology (subiculum/Brodmann area 35/precuneus/posterior cingulate). Finally, the associations between age and cognition were partially mediated via tau in Brodmann area 35, even when including Aβ-PET as covariate. Results were validated in the ex vivo cohort showing age-related and Aβ-independent increases in tau aggregates in and outside the medial temporal lobe. Ex vivo age-cognition associations were mediated by medial and inferior temporal tau tangle density, while correcting for Aβ density. Taken together, our study provides support for primary age-related tauopathy even outside the medial temporal lobe in vivo and ex vivo, with downstream effects on structure and cognition. These results have implications for our understanding of the spreading of tau outside the medial temporal lobe, also in the context of Alzheimer's disease. Moreover, this study suggests the potential utility of tau-targeting treatments in primary age-related tauopathy, likely already in preclinical stages in individuals with low Aβ pathology.

An automated, geometry-based method for hippocampal shape and thickness analysis

The hippocampus is one of the most studied neuroanatomical structures due to its involvement in attention, learning, and memory as well as its atrophy in ageing, neurological, and psychiatric diseases. Hippocampal shape changes, however, are complex and cannot be fully characterized by a single summary metric such as hippocampal volume as determined from MR images. In this work, we propose an automated, geometry-based approach for the unfolding, point-wise correspondence, and local analysis of hippocampal shape features such as thickness and curvature. Starting from an automated segmentation of hippocampal subfields, we create a 3D tetrahedral mesh model as well as a 3D intrinsic coordinate system of the hippocampal body. From this coordinate system, we derive local curvature and thickness estimates as well as a 2D sheet for hippocampal unfolding. We evaluate the performance of our algorithm with a series of experiments to quantify neurodegenerative changes in Mild Cognitive Impairment and Alzheimer's disease dementia. We find that hippocampal thickness estimates detect known differences between clinical groups and can determine the location of these effects on the hippocampal sheet. Further, thickness estimates improve classification of clinical groups and cognitively unimpaired controls when added as an additional predictor. Comparable results are obtained with different datasets and segmentation algorithms. Taken together, we replicate canonical findings on hippocampal volume/shape changes in dementia, extend them by gaining insight into their spatial localization on the hippocampal sheet, and provide additional, complementary information beyond traditional measures. We provide a new set of sensitive processing and analysis tools for the analysis of hippocampal geometry that allows comparisons across studies without relying on image registration or requiring manual intervention.

Brain reserve contributes to distinguishing preclinical Alzheimer's stages 1 and 2

BACKGROUND

In preclinical Alzheimer's disease, it is unclear why some individuals with amyloid pathologic change are asymptomatic (stage 1), whereas others experience subjective cognitive decline (SCD, stage 2). Here, we examined the association of stage 1 vs. stage 2 with structural brain reserve in memory-related brain regions.

METHODS

We tested whether the volumes of hippocampal subfields and parahippocampal regions were larger in individuals at stage 1 compared to asymptomatic amyloid-negative older adults (healthy controls, HCs). We also tested whether individuals with stage 2 would show the opposite pattern, namely smaller brain volumes than in amyloid-negative individuals with SCD. Participants with cerebrospinal fluid (CSF) biomarker data and bilateral volumetric MRI data from the observational, multi-centric DZNE-Longitudinal Cognitive Impairment and Dementia Study (DELCODE) study were included. The sample comprised 95 amyloid-negative and 26 amyloid-positive asymptomatic participants as well as 104 amyloid-negative and 47 amyloid-positive individuals with SCD. Volumes were based on high-resolution T2-weighted images and automatic segmentation with manual correction according to a recently established high-resolution segmentation protocol.

RESULTS

In asymptomatic individuals, brain volumes of hippocampal subfields and of the parahippocampal cortex were numerically larger in stage 1 compared to HCs, whereas the opposite was the case in individuals with SCD. MANOVAs with volumes as dependent data and age, sex, years of education, and DELCODE site as covariates showed a significant interaction between diagnosis (asymptomatic versus SCD) and amyloid status (Aß42/40 negative versus positive) for hippocampal subfields. Post hoc paired comparisons taking into account the same covariates showed that dentate gyrus and CA1 volumes in SCD were significantly smaller in amyloid-positive than negative individuals. In contrast, CA1 volumes were significantly (p = 0.014) larger in stage 1 compared with HCs.

CONCLUSIONS

These data indicate that HCs and stages 1 and 2 do not correspond to linear brain volume reduction. Instead, stage 1 is associated with larger than expected volumes of hippocampal subfields in the face of amyloid pathology. This indicates a brain reserve mechanism in stage 1 that enables individuals with amyloid pathologic change to be cognitively normal and asymptomatic without subjective cognitive decline.

A Non-Conventional Review on Multi-Modality-Based Medical Image Fusion

Today, medical images play a crucial role in obtaining relevant medical information for clinical purposes. However, the quality of medical images must be analyzed and improved. Various factors affect the quality of medical images at the time of medical image reconstruction. To obtain the most clinically relevant information, multi-modality-based image fusion is beneficial. Nevertheless, numerous multi-modality-based image fusion techniques are present in the literature. Each method has its assumptions, merits, and barriers. This paper critically analyses some sizable non-conventional work within multi-modality-based image fusion. Often, researchers seek help in apprehending multi-modality-based image fusion and choosing an appropriate multi-modality-based image fusion approach; this is unique to their cause. Hence, this paper briefly introduces multi-modality-based image fusion and non-conventional methods of multi-modality-based image fusion. This paper also signifies the merits and downsides of multi-modality-based image fusion.