Dual Adversarial Attention Mechanism for Unsupervised Domain Adaptive Medical Image Segmentation

Analysis of Disease-Induced Changes in Human Locomotor Patterns Through the Co-Joint Synergistic Attention Algorithm

OBJECTIVE

Aiming to quantify and analyze disease-induced alterations in human movement, we explored the co-joint synergy patterns in locomotion through a vision-based co-joint synergistic attention algorithm.

METHODS

We recruited 30 participants (including 15 post-stroke patients and 15 healthy individuals) and extracted their 3D visual motor data for the joint feature coupling by a serial attention module. And we designed a dual-stream classification module for preclassification based on the spatio-temporal characteristics of the data. Then we extracted the important co-joint synergy patterns by a looping mask module and the co-joint synergy variability score.

RESULTS

Through the co-joint synergistic attention algorithm, we found significant differences in joint synergy patterns between post-stroke patients and healthy individuals during upper and lower limb tasks. Furthermore, we obtained quantitative results on the effect of specific diseases on co-joint synergy patterns among healthy individuals and patients. The validity of the result was verified by comparing with the commonly used Non-negative Matrix Factorization (NMF) and the Muscle Synergy Fractionation (MSF) methods.

CONCLUSION

Specific diseases can cause changes in human movement patterns, and by the co-joint synergistic attention algorithm we can analyze the alterations in joint synergies and also quantify the importance of different synergy groups.

SIGNIFICANCE

This research proposes a new approach for identifying specific co-joint synergy patterns arising from disease-altered biomechanics, which provides a new targeted protocol for the rehabilitation process.

Multi-scale dual attention embedded U-shaped network for accurate segmentation of coronary vessels in digital subtraction angiography

BACKGROUND

Most attention-based networks fall short in effectively integrating spatial and channel-wise information across different scales, which results in suboptimal performance for segmenting coronary vessels in x-ray digital subtraction angiography (DSA) images. This limitation becomes particularly evident when attempting to identify tiny sub-branches.

PURPOSE

To address this limitation, a multi-scale dual attention embedded network (named MDA-Net) is proposed to consolidate contextual spatial and channel information across contiguous levels and scales.

METHODS

MDA-Net employs five cascaded double-convolution blocks within its encoder to adeptly extract multi-scale features. It incorporates skip connections that facilitate the retention of low-level feature details throughout the decoding phase, thereby enhancing the reconstruction of detailed image information. Furthermore, MDA modules, which take in features from neighboring scales and hierarchical levels, are tasked with discerning subtle distinctions between foreground elements, such as coronary vessels of diverse morphologies and dimensions, and the complex background, which includes structures like catheters or other tissues with analogous intensities. To sharpen the segmentation accuracy, the network utilizes a composite loss function that integrates intersection over union (IoU) loss with binary cross-entropy loss, ensuring the precision of the segmentation outcomes and maintaining an equilibrium between positive and negative classifications.

RESULTS

Experimental results demonstrate that MDA-Net not only performs more robustly and effectively on DSA images under various image conditions, but also achieves significant advantages over state-of-the-art methods, achieving the optimal scores in terms of IoU, Dice, accuracy, and Hausdorff distance 95%.

CONCLUSIONS

MDA-Net has high robustness for coronary vessels segmentation, providing an active strategy for early diagnosis of cardiovascular diseases. The code is publicly available at https://github.com/30410B/MDA-Net.git.

pFedBCC: Personalizing Federated multi-target domain adaptive segmentation via Bi-pole Collaborative Calibration

BACKGROUND AND OBJECTIVE

Multi-target domain adaptation (MTDA) is a well-established technology for unsupervised segmentation. It can significantly reduce the workload of large-scale data annotations, but assumes that each domain data can be freely accessed. However, data privacy limit its deployment in real-world medical scenes. Aiming at this problem, federated learning (FL) commits a paradigm to handle private cross-institution data.

METHODS

This paper makes the first attempt to apply FedMTDA to medical image segmentation by proposing a personalized Federated Bi-pole Collaborative Calibration (pFedBCC) framework, which leverages unannotated private client data and a public source-domain model to learn a global model at the central server for unsupervised multi-type immunohistochemically (IHC) image segmentation. Concretely, pFedBCC tackles two significant challenges in FedMTDA including client-side prediction drift and server-side aggregation drift via Semantic-affinity-driven Personalized Label Calibration (SPLC) and Source-knowledge-oriented Consistent Gradient Calibration (SCGC). To alleviate local prediction drift, SPLC personalizes a cross-domain graph reasoning module for each client, which achieves semantic-affinity alignment between high-level source- and target-domain features to produce pseudo labels that are semantically consistent with source-domain labels to guide client training. To further alleviate global aggregation drift, SCGC develops a new conflict-gradient clipping scheme, which takes the source-domain gradient as a guidance to ensure that all clients update with similar gradient directions and magnitudes, thereby improving the generalization of the global model.

RESULTS

pFedBCC is evaluated on private and public IHC benchmarks, including the proposed MT-IHC dataset, and the panCK, BCData, DLBC-Morph and LYON19 datasets. Overall, pFedBCC achieves the best performance of 88.8% PA on MT-IHC, as well as 88.4% PA on the LYON19 dataset, respectively.

CONCLUSIONS

The proposed pFedBCC performs better than all comparison methods. The ablation study also confirms the contribution of SPLC and SCGC for unsupervised multi-type IHC image segmentation. This paper constructs a MT-IHC dataset containing more than 19,000 IHC images of 10 types (CgA, CK, Syn, CD, Ki67, P40, P53, EMA, TdT and BCL). Extensive experiments on the MT-IHC and public IHC datasets confirm that pFedBCC outperforms existing FL and DA methods.

Style mixup enhanced disentanglement learning for unsupervised domain adaptation in medical image segmentation

Unsupervised domain adaptation (UDA) has shown impressive performance by improving the generalizability of the model to tackle the domain shift problem for cross-modality medical segmentation. However, most of the existing UDA approaches depend on high-quality image translation with diversity constraints to explicitly augment the potential data diversity, which is hard to ensure semantic consistency and capture domain-invariant representation. In this paper, free of image translation and diversity constraints, we propose a novel Style Mixup Enhanced Disentanglement Learning (SMEDL) for UDA medical image segmentation to further improve domain generalization and enhance domain-invariant learning ability. Firstly, our method adopts disentangled style mixup to implicitly generate style-mixed domains with diverse styles in the feature space through a convex combination of disentangled style factors, which can effectively improve the model generalization. Meanwhile, we further introduce pixel-wise consistency regularization to ensure the effectiveness of style-mixed domains and provide domain consistency guidance. Secondly, we introduce dual-level domain-invariant learning, including intra-domain contrastive learning and inter-domain adversarial learning to mine the underlying domain-invariant representation under both intra- and inter-domain variations. We have conducted comprehensive experiments to evaluate our method on two public cardiac datasets and one brain dataset. Experimental results demonstrate that our proposed method achieves superior performance compared to the state-of-the-art methods for UDA medical image segmentation.

Multi-scale anatomical regularization for domain-adaptive segmentation of pelvic CBCT images

BACKGROUND

Cone beam computed tomography (CBCT) image segmentation is crucial in prostate cancer radiotherapy, enabling precise delineation of the prostate gland for accurate treatment planning and delivery. However, the poor quality of CBCT images poses challenges in clinical practice, making annotation difficult due to factors such as image noise, low contrast, and organ deformation.

PURPOSE

The objective of this study is to create a segmentation model for the label-free target domain (CBCT), leveraging valuable insights derived from the label-rich source domain (CT). This goal is achieved by addressing the domain gap across diverse domains through the implementation of a cross-modality medical image segmentation framework.

METHODS

Our approach introduces a multi-scale domain adaptive segmentation method, performing domain adaptation simultaneously at both the image and feature levels. The primary innovation lies in a novel multi-scale anatomical regularization approach, which (i) aligns the target domain feature space with the source domain feature space at multiple spatial scales simultaneously, and (ii) exchanges information across different scales to fuse knowledge from multi-scale perspectives.

RESULTS

Quantitative and qualitative experiments were conducted on pelvic CBCT segmentation tasks. The training dataset comprises 40 unpaired CBCT-CT images with only CT images annotated. The validation and testing datasets consist of 5 and 10 CT images, respectively, all with annotations. The experimental results demonstrate the superior performance of our method compared to other state-of-the-art cross-modality medical image segmentation methods. The Dice similarity coefficients (DSC) for CBCT image segmentation results is74.6 ± 9.3 $74.6 \pm 9.3$ %, and the average symmetric surface distance (ASSD) is3.9 ± 1.8 mm $3.9\pm 1.8\;\mathrm{mm}$ . Statistical analysis confirms the statistical significance of the improvements achieved by our method.

CONCLUSIONS

Our method exhibits superiority in pelvic CBCT image segmentation compared to its counterparts.

BCL-Former: Localized Transformer Fusion with Balanced Constraint for polyp image segmentation

Polyp segmentation remains challenging for two reasons: (a) the size and shape of colon polyps are variable and diverse; (b) the distinction between polyps and mucosa is not obvious. To solve the above two challenging problems and enhance the generalization ability of segmentation method, we propose the Localized Transformer Fusion with Balanced Constraint (BCL-Former) for Polyp Segmentation. In BCL-Former, the Strip Local Enhancement module (SLE module) is proposed to capture the enhanced local features. The Progressive Feature Fusion module (PFF module) is presented to make the feature aggregation smoother and eliminate the difference between high-level and low-level features. Moreover, the Tversky-based Appropriate Constrained Loss (TacLoss) is proposed to achieve the balance and constraint between True Positives and False Negatives, improving the ability to generalize across datasets. Extensive experiments are conducted on four benchmark datasets. Results show that our proposed method achieves state-of-the-art performance in both segmentation precision and generalization ability. Also, the proposed method is 5%-8% faster than the benchmark method in training and inference. The code is available at: https://github.com/sjc-lbj/BCL-Former.

MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion

Ultrasound image segmentation is a challenging task due to the complexity of lesion types, fuzzy boundaries, and low-contrast images along with the presence of noises and artifacts. To address these issues, we propose an end-to-end multi-scale feature extraction and fusion network (MEF-UNet) for the automatic segmentation of ultrasound images. Specifically, we first design a selective feature extraction encoder, including detail extraction stage and structure extraction stage, to precisely capture the edge details and overall shape features of the lesions. In order to enhance the representation capacity of contextual information, we develop a context information storage module in the skip-connection section, responsible for integrating information from adjacent two-layer feature maps. In addition, we design a multi-scale feature fusion module in the decoder section to merge feature maps with different scales. Experimental results indicate that our MEF-UNet can significantly improve the segmentation results in both quantitative analysis and visual effects.

Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives

Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.

Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview

The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.

Unsupervised Domain Adaptation for Medical Image Segmentation Using Transformer With Meta Attention

Image segmentation is essential to medical image analysis as it provides the labeled regions of interest for the subsequent diagnosis and treatment. However, fully-supervised segmentation methods require high-quality annotations produced by experts, which is laborious and expensive. In addition, when performing segmentation on another unlabeled image modality, the segmentation performance will be adversely affected due to the domain shift. Unsupervised domain adaptation (UDA) is an effective way to tackle these problems, but the performance of the existing methods is still desired to improve. Also, despite the effectiveness of recent Transformer-based methods in medical image segmentation, the adaptability of Transformers is rarely investigated. In this paper, we present a novel UDA framework using a Transformer for building a cross-modality segmentation method with the advantages of learning long-range dependencies and transferring attentive information. To fully utilize the attention learned by the Transformer in UDA, we propose Meta Attention (MA) and use it to perform a fully attention-based alignment scheme, which can learn the hierarchical consistencies of attention and transfer more discriminative information between two modalities. We have conducted extensive experiments on cross-modality segmentation using three datasets, including a whole heart segmentation dataset (MMWHS), an abdominal organ segmentation dataset, and a brain tumor segmentation dataset. The promising results show that our method can significantly improve performance compared with the state-of-the-art UDA methods.

Unsupervised deep consistency learning adaptation network for cardiac cross-modality structural segmentation

Deep neural networks have recently been succeessful in the field of medical image segmentation; however, they are typically subject to performance degradation problems when well-trained models are tested in another new domain with different data distributions. Given that annotated cross-domain images may inaccessible, unsupervised domain adaptation methods that transfer learnable information from annotated source domains to unannotated target domains with different distributions have attracted substantial attention. Many methods leverage image-level or pixel-level translation networks to align domain-invariant information and mitigate domain shift issues. However, These methods rarely perform well when there is a large domain gap. A new unsupervised deep consistency learning adaptation network, which adopts input space consistency learning and output space consistency learning to realize unsupervised domain adaptation and cardiac structural segmentation, is introduced in this paper The framework mainly includes a domain translation path and a cross-modality segmentation path. In domain translation path, a symmetric alignment generator network with attention to cross-modality features and anatomy is introduced to align bidirectional domain features. In the segmentation path, entropy map minimization, output probability map minimization and segmentation prediction minimization are leveraged to align the output space features. The model conducts supervised learning to extract source domain features and conducts unsupervised deep consistency learning to extract target domain features. Through experimental testing on two challenging cross-modality segmentation tasks, our method has robust performance compared to that of previous methods. Furthermore, ablation experiments are conducted to confirm the effectiveness of our framework.