Domain Adaptation by Class Centroid Matching and Local Manifold Self-Learning

Heterogeneous Domain Adaptation With Generalized Similarity and Dissimilarity Regularization

Heterogeneous domain adaptation (HDA) aims to address the transfer learning problems where the source domain and target domain are represented by heterogeneous features. The existing HDA methods based on matrix factorization have been proven to learn transferable features effectively. However, these methods only preserve the original neighbor structure of samples in each domain and do not use the label information to explore the similarity and separability between samples. This would not eliminate the cross-domain bias of samples and may mix cross-domain samples of different classes in the common subspace, misleading the discriminative feature learning of target samples. To tackle the aforementioned problems, we propose a novel matrix factorization-based HDA method called HDA with generalized similarity and dissimilarity regularization (HGSDR). Specifically, we propose a similarity regularizer by establishing the cross-domain Laplacian graph with label information to explore the similarity between cross-domain samples from the identical class. And we propose a dissimilarity regularizer based on the inner product strategy to expand the separability of cross-domain labeled samples from different classes. For unlabeled target samples, we keep their neighbor relationship to preserve the similarity and separability between them in the original space. Hence, the generalized similarity and dissimilarity regularization is built by integrating the above regularizers to facilitate cross-domain samples to form discriminative class distributions. HGSDR can more efficiently match the distributions of the two domains both from the global and sample viewpoints, thereby learning discriminative features for target samples. Extensive experiments on the benchmark datasets demonstrate the superiority of the proposed method against several state-of-the-art methods.

Enhancing Autism Spectrum Disorder identification in multi-site MRI imaging: A multi-head cross-attention and multi-context approach for addressing variability in un-harmonized data

Multi-site MRI imaging poses a significant challenge due to the potential variations in images across different scanners at different sites. This variability can introduce ambiguity in further image analysis. Consequently, the image analysis techniques become site-dependent and scanner-dependent, implying that adjustments in the analysis methodologies are necessary for each scanner configuration. Further, implementing real-time modifications becomes intricate, particularly when incorporating a new type of scanner, as it requires adapting the analysis methods accordingly. Taking into account the aforementioned challenge, we have considered its implications for an Autism spectrum disorder (ASD) application. Our objective is to minimize the impact of site and scanner variability in the analysis, aiming to develop a model that remains effective across different scanners and sites. This entails devising a methodology that allows the same model to function seamlessly across multiple scanner configurations and sites. ASD, a behavioral disorder affecting child development, requires early detection. Clinical observation is time-consuming, prompting the use of fMRI with machine/deep learning for expedited diagnosis. Previous methods leverage fMRI's functional connectivity but often rely on less generalized feature extractors and classifiers. Hence, there is significant room for improvement in the generalizability of detection methods across multi-site data, which is acquired from multiple scanners with different settings. In this study, we propose a Cross-Combination Multi-Scale Multi-Context Framework (CCMSMCF) capable of performing neuroimaging-based diagnostic classification of mental disorders for a multi-site dataset. Thus, this framework attains a degree of internal data harmonization, rendering it to some extent site and scanner-agnostic. Our proposed network, CCMSMCF, is constructed by integrating two sub-modules: the Multi-Head Attention Cross-Scale Module (MHACSM) and the Residual Multi-Context Module (RMCN). We also employ multiple loss functions in a novel manner for training the model, which includes Binary Cross Entropy, Dice loss, and Embedding Coupling loss. The model is validated on the Autism Brain Imaging Data Exchange I (ABIDE-I) dataset, which includes data from multiple scanners across different sites, and achieves promising results.

Unsupervised domain adaptation with weak source domain labels via bidirectional subdomain alignment

Unsupervised domain adaptation (UDA) enables knowledge transfer from a labeled source domain to an unlabeled target domain. However, UDA performance often relies heavily on the accuracy of source domain labels, which are frequently noisy or missing in real applications. To address unreliable source labels, we propose a novel framework for extracting robust, discriminative features via iterative pseudo-labeling, queue-based clustering, and bidirectional subdomain alignment (BSA). The proposed framework begins by generating pseudo-labels for unlabeled source data and constructing codebooks via iterative clustering to obtain label-independent class centroids. Then, the proposed framework performs two main tasks: rectifying features from both domains using BSA to match subdomain distributions and enhance features; and employing a two-stage adversarial process for global feature alignment. The feature rectification is done before feature enhancement, while the global alignment is done after feature enhancement. To optimize our framework, we formulate BSA and adversarial learning as maximizing a log-likelihood function, which is implemented via the Expectation-Maximization algorithm. The proposed framework shows significant improvements compared to state-of-the-art methods on Office-31, Office-Home, and VisDA-2017 datasets, achieving average accuracies of 91.5%, 76.6%, and 87.4%, respectively. Compared to existing methods, the proposed method shows consistent superiority in unsupervised domain adaptation tasks with both fully and weakly labeled source domains.

TFRS: A task-level feature rectification and separation method for few-shot video action recognition

Few-shot video action recognition (FS-VAR) is a challenging task that requires models to have significant expressive power in order to identify previously unseen classes using only a few labeled examples. However, due to the limited number of support samples, the model's performance is highly sensitive to the distribution of the sampled data. The representativeness of the support data is insufficient to cover the entire class, and the support features may contain shared information that confuses the classifier, leading to biased classification. In response to this difficulty, we present a task-level feature rectification and separation (TFRS) method that effectively resolves the sample bias issue. Our main idea is to leverage prior information from base classes to rectify the support samples while removing the commonality of task-level features. This enhances the distinguishability and separability of features in space. Furthermore, TFRS offers a straightforward yet versatile solution that can be seamlessly integrated into various established FS-VAR frameworks. Our design yields significant performance enhancements across various existing works by implementing TFRS, resulting in competitive outcomes on datasets such as UCF101, Kinetics, SSv2, and HMDB51.

Domain Adaptation With Self-Supervised Learning and Feature Clustering for Intelligent Fault Diagnosis

Domain adaptation indeed promotes the progress of intelligent fault diagnosis in industrial scenarios. The abundant labeled samples are not necessary. The identical distribution between the training and testing datasets is not any more the prerequisite for intelligent fault diagnosis working. However, two issues arise subsequently: Feature learning in domain adaptation framework tends to be biased to the source domain, and unreliable pseudolabeling seriously impacts on the conditional domain adaptation. In this article, a new domain adaptation approach with self-supervised learning and feature clustering (DASSL-FC) is proposed, trying to alleviate the issues by unbiased feature learning and pseudolabels updating strategy. Taking different transformation methods as pretext, the transformed data and its pretext train a neural network in an SSL way. As to pseudolabeling, clusters are taken as the auxiliary information to correct the network predicted labels in terms of the "strong cluster" rule. Then, the updated pseudolabels and their confidence are enforced to further estimate the conditional distribution discrepancy and its confidence weight. To verify the effectiveness of the proposed method, the experiments are implemented on intraplatform and interplatforms for simulating the practical scenarios.

Domain Adaptation via Low Rank and Class Discriminative Representation for Autism Spectrum Disorder Identification: A Multi-Site fMRI Study

To construct a more effective model with good generalization performance for inter-site autism spectrum disorder (ASD) diagnosis, domain adaptation based ASD diagnostic models are proposed to alleviate the inter-site heterogeneity. However, most existing methods only reduce the marginal distribution difference without considering class discriminative information, and are difficult to achieve satisfactory results. In this paper, we propose a low rank and class discriminative representation (LRCDR) based multi-source unsupervised domain adaptation method to reduce the marginal and conditional distribution differences synchronously for improving ASD identification. Specifically, LRCDR adopts low rank representation to alleviate the marginal distribution difference between domains by aligning the global structure of the projected multi-site data. To reduce the conditional distribution difference of data from all sites, LRCDR learns the class discriminative representation of data from multiple source domains and the target domain to enhance the intra-class compactness and inter-class separability of the projected data. For inter-site prediction on all ABIDE I data (1102 subjects from 17 sites), LRCDR obtains the mean accuracy of 73.1%, superior to the results of the compared state-of-the-art domain adaptation methods and multi-site ASD identification methods. In addition, we locate some meaningful biomarkers: Most of the top important biomarkers are inter-network resting-state functional connectivities (RSFCs). The proposed LRCDR method can effectively improve the identification of ASD, and has great potential as a clinical diagnostic tool.

Cross-Domain Pattern Classification With Distribution Adaptation Based on Evidence Theory

In pattern classification, there may not exist labeled patterns in the target domain to train a classifier. Domain adaptation (DA) techniques can transfer the knowledge from the source domain with massive labeled patterns to the target domain for learning a classification model. In practice, some objects in the target domain are easily classified by this classification model, and these objects usually can provide more or less useful information for classifying the other objects in the target domain. So a new method called distribution adaptation based on evidence theory (DAET) is proposed to improve the classification accuracy by combining the complementary information derived from both the source and target domains. In DAET, the objects that are easy to classify are first selected as easy-target objects, and the other objects are regarded as hard-target objects. For each hard-target object, we can obtain one classification result with the assistance of massive labeled patterns in the source domain, and another classification result can be acquired based on the easy-target objects with confidently predicted (pseudo) labels. However, the weights of these classification results may vary because the reliabilities of the used information sources are different. The weights are estimated by mean difference reflecting the information source quality. Then, we discount the classification results with the corresponding weights under the framework of the evidence theory, which is expert at dealing with uncertain information. These discounted classification results are combined by an evidential combination rule for making the final class decision. The effectiveness of DAET for cross-domain pattern classification is evaluated with respect to some advanced DA methods, and the experiment results show DAET can significantly improve the classification accuracy.

Guided Discrimination and Correlation Subspace Learning for Domain Adaptation

As a branch of transfer learning, domain adaptation leverages useful knowledge from a source domain to a target domain for solving target tasks. Most of the existing domain adaptation methods focus on how to diminish the conditional distribution shift and learn invariant features between different domains. However, two important factors are overlooked by most existing methods: 1) the transferred features should be not only domain invariant but also discriminative and correlated, and 2) negative transfer should be avoided as much as possible for the target tasks. To fully consider these factors in domain adaptation, we propose a guided discrimination and correlation subspace learning (GDCSL) method for cross-domain image classification. GDCSL considers the domain-invariant, category-discriminative, and correlation learning of data. Specifically, GDCSL introduces the discriminative information associated with the source and target data by minimizing the intraclass scatter and maximizing the interclass distance. By designing a new correlation term, GDCSL extracts the most correlated features from the source and target domains for image classification. The global structure of the data can be preserved in GDCSL because the target samples are represented by the source samples. To avoid negative transfer issues, we use a sample reweighting method to detect target samples with different confidence levels. A semi-supervised extension of GDCSL (Semi-GDCSL) is also proposed, and a novel label selection scheme is introduced to ensure the correction of the target pseudo-labels. Comprehensive and extensive experiments are conducted on several cross-domain data benchmarks. The experimental results verify the effectiveness of the proposed methods over state-of-the-art domain adaptation methods.

Probability-Based Graph Embedding Cross-Domain and Class Discriminative Feature Learning for Domain Adaptation

Feature-based domain adaptation methods project samples from different domains into the same feature space and try to align the distribution of two domains to learn an effective transferable model. The vital problem is how to find a proper way to reduce the domain shift and improve the discriminability of features. To address the above issues, we propose a unified Probability-based Graph embedding Cross-domain and class Discriminative feature learning framework for unsupervised domain adaptation (PGCD). Specifically, we propose novel graph embedding structures to be the class discriminative transfer feature learning item and cross-domain alignment item, which can make the same-category samples compact in each domain, and fully align the local and global geometric structure across domains. Besides, two theoretical analyses are given to prove the interpretability of the proposed graph structures, which can further describe the relationships between samples to samples in single-domain and cross-domain transfer feature learning scenarios. Moreover, we adopt novel weight strategies via probability information to generate robust centroids in each proposed item to enhance the accuracy of transfer feature learning and reduce the error accumulation. Compared with the advanced approaches by comprehensive experiments, the promising performance on the benchmark datasets verify the effectiveness of the proposed model.

Cluster Alignment With Target Knowledge Mining for Unsupervised Domain Adaptation Semantic Segmentation

Unsupervised domain adaptation (UDA) carries out knowledge transfer from the labeled source domain to the unlabeled target domain. Existing feature alignment methods in UDA semantic segmentation achieve this goal by aligning the feature distribution between domains. However, these feature alignment methods ignore the domain-specific knowledge of the target domain. In consequence, 1) the correlation among pixels of the target domain is not explored; and 2) the classifier is not explicitly designed for the target domain distribution. To conquer these obstacles, we propose a novel cluster alignment framework, which mines the domain-specific knowledge when performing the alignment. Specifically, we design a multi-prototype clustering strategy to make the pixel features within the same class tightly distributed for the target domain. Subsequently, a contrastive strategy is developed to align the distributions between domains, with the clustered structure maintained. After that, a novel affinity-based normalized cut loss is devised to learn task-specific decision boundaries. Our method enhances the model's adaptability in the target domain, and can be used as a pre-adaptation for self-training to boost its performance. Sufficient experiments prove the effectiveness of our method against existing state-of-the-art methods on representative UDA benchmarks.

Distribution matching and structure preservation for domain adaptation

Cross-domain classification refers to completing the corresponding classification task in a target domain which lacks label information, by exploring useful knowledge in a related source domain but with different data distribution. Domain adaptation can deal with such cross-domain classification, by reducing divergence of domains and transferring the relevant knowledge from the source to the target. To mine the discriminant information of the source domain samples and the geometric structure information of domains, and thus improve domain adaptation performance, this paper proposes a novel method involving distribution matching and structure preservation for domain adaptation (DMSP). First, it aligns the subspaces of the source domain and target domain on the Grassmann manifold; and learns the non-distorted embedded feature representations of the two domains. Second, in this embedded feature space, the empirical structure risk minimization method with distribution adaptation regularization and intra-domain graph regularization is used to learn an adaptive classifier, further adapting the source and target domains. Finally, we perform extensive experiments on widely used cross-domain classification datasets to validate the superiority of DMSP. The average classification accuracy of DMSP on these datasets is the highest compared with several state-of-the-art domain adaptation methods.

Emotion recognition based on multi-modal physiological signals and transfer learning

In emotion recognition based on physiological signals, collecting enough labeled data of a single subject for training is time-consuming and expensive. The physiological signals’ individual differences and the inherent noise will significantly affect emotion recognition accuracy. To overcome the difference in subject physiological signals, we propose a joint probability domain adaptation with the bi-projection matrix algorithm (JPDA-BPM). The bi-projection matrix method fully considers the source and target domain’s different feature distributions. It can better project the source and target domains into the feature space, thereby increasing the algorithm’s performance. We propose a substructure-based joint probability domain adaptation algorithm (SSJPDA) to overcome physiological signals’ noise effect. This method can avoid the shortcomings that the domain level matching is too rough and the sample level matching is susceptible to noise. In order to verify the effectiveness of the proposed transfer learning algorithm in emotion recognition based on physiological signals, we verified it on the database for emotion analysis using physiological signals (DEAP dataset). The experimental results show that the average recognition accuracy of the proposed SSJPDA-BPM algorithm in the multimodal fusion physiological data from the DEAP dataset is 63.6 and 64.4% in valence and arousal, respectively. Compared with joint probability domain adaptation (JPDA), the performance of valence and arousal recognition accuracy increased by 17.6 and 13.4%, respectively.

Coupled Heterogeneous Tucker Decomposition: A Feature Extraction Method for Multisource Fusion and Domain Adaptation Using Multisource Heterogeneous Remote Sensing Data

To excavate adequately the rich information contained in multisource remote sensing data, feature extraction as basic yet important research has two typical applications: one of which is to extract complementary information of multisource data to improve classification; and the other is to extract shared information across sources for domain adaptation. However, typical feature extraction methods require the input represented as vectors or homogeneous tensors and fail to process multisource data represented as heterogeneous tensors. Therefore, the coupled heterogeneous Tucker decomposition (C-HTD) containing two sub-methods, namely coupled factor matrix-based HTD (CFM-HTD) and coupled core tensor-based HTD (CCT-HTD), is proposed to establish a unified feature extraction framework for multisource fusion and domain adaptation. To handle multisource heterogeneous tensors, multiple Tucker models were constructed to extract features of different sources separately. To cope with the supervised and semi-supervised cases, the class-indicator factor matrix was built to enhance the separability of features using known labels and learned labels. To mine the complementarity of paired multisource samples, coupling constraint was imposed on multiple factor matrices to form CFM-HTD to extract multisource information jointly. To extract domain-adapted features, coupling constraint was imposed on multiple core tensors to form CCT-HTD to encourage data from different sources to have the same class centroid. In addition, to reduce the impact of interference samples on domain adaptation, an adaptive sample-weighting matrix was designed to autonomously remove outliers. Using multiresolution multiangle optical and MSTAR datasets, experimental results show that the C-HTD outperforms typical multisource fusion and domain adaptation methods.

Joint Feature-Space and Sample-Space Based Heterogeneous Feature Transfer Method for Object Recognition Using Remote Sensing Images with Different Spatial Resolutions

To improve the classification results of high-resolution remote sensing images (RSIs), it is necessary to use feature transfer methods to mine the relevant information between high-resolution RSIs and low-resolution RSIs to train the classifiers together. Most of the existing feature transfer methods can only handle homogeneous data (i.e., data with the same dimension) and are susceptible to the quality of the RSIs, while RSIs with different resolutions present different feature dimensions and samples obtained from illumination conditions. To obtain effective classification results, unlike existing methods that focus only on the projection transformation in feature space, a joint feature-space and sample-space heterogeneous feature transfer (JFSSS-HFT) method is proposed to simultaneously process heterogeneous multi-resolution images in feature space using projection matrices of different dimensions and reduce the impact of outliers by adaptive weight factors in the sample space simultaneously to reduce the occurrence of negative transfer. Moreover, the maximum interclass variance term is embedded to improve the discriminant ability of the transferred features. To solve the optimization problem of JFSSS-HFT, the alternating-direction method of multipliers (ADMM) is introduced to alternatively optimize the parameters of JFSSS-HFT. Using different types of ship patches and airplane patches with different resolutions, the experimental results show that the proposed JFSSS-HFT obtains better classification results than the typical feature transferred methods.