Double Stage Transfer Learning for Brain-Computer Interfaces

In the application of brain-computer interfaces (BCIs), electroencephalogram (EEG) signals are difficult to collect in large quantities due to the non-stationary nature and long calibration time required. Transfer learning (TL), which transfers knowledge learned from existing subjects to new subjects, can be applied to solve this problem. Some existing EEG-based TL algorithms cannot achieve good results because they only extract partial features. To achieve effective transfer, a double-stage transfer learning (DSTL) algorithm which applied transfer learning to both preprocessing stage and feature extraction stage of typical BCIs was proposed. First, Euclidean alignment (EA) was used to align EEG trials from different subjects. Second, aligned EEG trials in the source domain were reweighted by the distance between the covariance matrix of each trial in the source domain and the mean covariance matrix of the target domain. Lastly, after extracting spatial features with common spatial patterns (CSP), transfer component analysis (TCA) was adopted to reduce the differences between different domains further. Experiments on two public datasets in two transfer paradigms (multi-source to single-target (MTS) and single-source to single-target (STS)) verified the effectiveness of the proposed method. The proposed DSTL achieved better classification accuracy on two datasets: 84.64% and 77.16% in MTS, 73.38% and 68.58% in STS, which shows that DSTL performs better than other state-of-the-art methods. The proposed DSTL can reduce the difference between the source domain and the target domain, providing a new method for EEG data classification without training dataset.

Cross-Subject Emotion Recognition Based on Domain Similarity of EEG Signal Transfer Learning

For solving the problem of the inevitable decline in the accuracy of cross-subject emotion recognition via Electroencephalograph (EEG) signal transfer learning due to the negative transfer of data in the source domain, this paper offers a new method to dynamically select the data suitable for transfer learning and eliminate the data that may lead to negative transfer. The method which is called cross-subject source domain selection (CSDS) consists of the next three parts. 1) First, a Frank-copula model is established according to Copula function theory to study the correlation between the source domain and the target domain, which is described by the Kendall correlation coefficient. 2) The calculation method for the Maximum Mean Discrepancy is improved to determine the distance between classes in a single source. After normalization, the Kendall correlation coefficient is superimposed, and the threshold is set to identify the source-domain data most suitable for transfer learning. 3) In the process of transfer learning, on the basis of Manifold Embedded Distribution Alignment, the Local Tangent Space Alignment method is used to provide a low-dimensional linear estimation of the local geometry of nonlinear manifolds, which maintains the local characteristics of the sample data after dimensionality reduction. Experimental results show that compared with the traditional methods, the CSDS increases the accuracy of emotion classification by approximately 2.8% and reduces the runtime by approximately 65%.

Domain Adaptation via Low Rank and Class Discriminative Representation for Autism Spectrum Disorder Identification: A Multi-Site fMRI Study

To construct a more effective model with good generalization performance for inter-site autism spectrum disorder (ASD) diagnosis, domain adaptation based ASD diagnostic models are proposed to alleviate the inter-site heterogeneity. However, most existing methods only reduce the marginal distribution difference without considering class discriminative information, and are difficult to achieve satisfactory results. In this paper, we propose a low rank and class discriminative representation (LRCDR) based multi-source unsupervised domain adaptation method to reduce the marginal and conditional distribution differences synchronously for improving ASD identification. Specifically, LRCDR adopts low rank representation to alleviate the marginal distribution difference between domains by aligning the global structure of the projected multi-site data. To reduce the conditional distribution difference of data from all sites, LRCDR learns the class discriminative representation of data from multiple source domains and the target domain to enhance the intra-class compactness and inter-class separability of the projected data. For inter-site prediction on all ABIDE I data (1102 subjects from 17 sites), LRCDR obtains the mean accuracy of 73.1%, superior to the results of the compared state-of-the-art domain adaptation methods and multi-site ASD identification methods. In addition, we locate some meaningful biomarkers: Most of the top important biomarkers are inter-network resting-state functional connectivities (RSFCs). The proposed LRCDR method can effectively improve the identification of ASD, and has great potential as a clinical diagnostic tool.

Heterogeneous Domain Adaptation With Adversarial Neural Representation Learning: Experiments on E-Commerce and Cybersecurity

Learning predictive models in new domains with scarce training data is a growing challenge in modern supervised learning scenarios. This incentivizes developing domain adaptation methods that leverage the knowledge in known domains (source) and adapt to new domains (target) with a different probability distribution. This becomes more challenging when the source and target domains are in heterogeneous feature spaces, known as heterogeneous domain adaptation (HDA). While most HDA methods utilize mathematical optimization to map source and target data to a common space, they suffer from low transferability. Neural representations have proven to be more transferable; however, they are mainly designed for homogeneous environments. Drawing on the theory of domain adaptation, we propose a novel framework, Heterogeneous Adversarial Neural Domain Adaptation (HANDA), to effectively maximize the transferability in heterogeneous environments. HANDA conducts feature and distribution alignment in a unified neural network architecture and achieves domain invariance through adversarial kernel learning. Three experiments were conducted to evaluate the performance against the state-of-the-art HDA methods on major image and text e-commerce benchmarks. HANDA shows statistically significant improvement in predictive performance. The practical utility of HANDA was shown in real-world dark web online markets. HANDA is an important step towards successful domain adaptation in e-commerce applications.

From Big to Small: Adaptive Learning to Partial-Set Domains

Domain adaptation targets at knowledge acquisition and dissemination from a labeled source domain to an unlabeled target domain under distribution shift. Still, the common requirement of identical class space shared across domains hinders applications of domain adaptation to partial-set domains. Recent advances show that deep pre-trained models of large scale endow rich knowledge to tackle diverse downstream tasks of small scale. Thus, there is a strong incentive to adapt models from large-scale domains to small-scale domains. This paper introduces Partial Domain Adaptation (PDA), a learning paradigm that relaxes the identical class space assumption to that the source class space subsumes the target class space. First, we present a theoretical analysis of partial domain adaptation, which uncovers the importance of estimating the transferable probability of each class and each instance across domains. Then, we propose Selective Adversarial Network (SAN and SAN++) with a bi-level selection strategy and an adversarial adaptation mechanism. The bi-level selection strategy up-weighs each class and each instance simultaneously for source supervised training, target self-training, and source-target adversarial adaptation through the transferable probability estimated alternately by the model. Experiments on standard partial-set datasets and more challenging tasks with superclasses show that SAN++ outperforms several domain adaptation methods.

Unsupervised cross-domain functional MRI adaptation for automated major depressive disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) data have been widely used for automated diagnosis of brain disorders such as major depressive disorder (MDD) to assist in timely intervention. Multi-site fMRI data have been increasingly employed to augment sample size and improve statistical power for investigating MDD. However, previous studies usually suffer from significant inter-site heterogeneity caused for instance by differences in scanners and/or scanning protocols. To address this issue, we develop a novel discrepancy-based unsupervised cross-domain fMRI adaptation framework (called UFA-Net) for automated MDD identification. The proposed UFA-Net is designed to model spatio-temporal fMRI patterns of labeled source and unlabeled target samples via an attention-guided graph convolution module, and also leverage a maximum mean discrepancy constrained module for unsupervised cross-site feature alignment between two domains. To the best of our knowledge, this is one of the first attempts to explore unsupervised rs-fMRI adaptation for cross-site MDD identification. Extensive evaluation on 681 subjects from two imaging sites shows that the proposed method outperforms several state-of-the-art methods. Our method helps localize disease-associated functional connectivity abnormalities and is therefore well interpretable and can facilitate fMRI-based analysis of MDD in clinical practice.

Transporting a Prediction Model for Use in a New Target Population

We considered methods for transporting a prediction model for use in a new target population, both when outcome and covariate data for model development are available from a source population that has a different covariate distribution compared with the target population and when covariate data (but not outcome data) are available from the target population. We discuss how to tailor the prediction model to account for differences in the data distribution between the source population and the target population. We also discuss how to assess the model's performance (e.g., by estimating the mean squared prediction error) in the target population. We provide identifiability results for measures of model performance in the target population for a potentially misspecified prediction model under a sampling design where the source and the target population samples are obtained separately. We introduce the concept of prediction error modifiers that can be used to reason about tailoring measures of model performance to the target population. We illustrate the methods in simulated data and apply them to transport a prediction model for lung cancer diagnosis from the National Lung Screening Trial to the nationally representative target population of trial-eligible individuals in the National Health and Nutrition Examination Survey.

Cluster Embedding Joint-Probability-Discrepancy Transfer for Cross-Subject Seizure Detection

Transfer learning (TL) has been applied in seizure detection to deal with differences between different subjects or tasks. In this paper, we consider cross-subject seizure detection that does not rely on patient history records, that is, acquiring knowledge from other subjects through TL to improve seizure detection performance. We propose a novel domain adaptation method, named the Cluster Embedding Joint-Probability-Discrepancy Transfer (CEJT), for data distribution structure learning. Specifically, 1) The joint probability distribution discrepancy is minimized to reduce the distribution shift in the source and target domains, and strengthen the discriminative knowledge of classes. 2) A clustering is performed on the target domain, and the class centroids of sources is used as the clustering prototype of the target domain to enhance data structure. It is worth noting that the manifold regularization is used to improve the quality of clustering prototypes. In addition, a correlation-alignment-based source selection metric (SSC) is designed for most favorable subject selection, reducing the computational cost as well as avoiding some negative transfer. Experiments on 15 patients with focal epilepsy from the Children's Hospital, Zhejiang University School of Medicine (CHZU) database shown that CEJT outperforms several state-of-the-art approaches, and can promote the application of seizure detection.

Cross-Domain Pattern Classification With Distribution Adaptation Based on Evidence Theory

In pattern classification, there may not exist labeled patterns in the target domain to train a classifier. Domain adaptation (DA) techniques can transfer the knowledge from the source domain with massive labeled patterns to the target domain for learning a classification model. In practice, some objects in the target domain are easily classified by this classification model, and these objects usually can provide more or less useful information for classifying the other objects in the target domain. So a new method called distribution adaptation based on evidence theory (DAET) is proposed to improve the classification accuracy by combining the complementary information derived from both the source and target domains. In DAET, the objects that are easy to classify are first selected as easy-target objects, and the other objects are regarded as hard-target objects. For each hard-target object, we can obtain one classification result with the assistance of massive labeled patterns in the source domain, and another classification result can be acquired based on the easy-target objects with confidently predicted (pseudo) labels. However, the weights of these classification results may vary because the reliabilities of the used information sources are different. The weights are estimated by mean difference reflecting the information source quality. Then, we discount the classification results with the corresponding weights under the framework of the evidence theory, which is expert at dealing with uncertain information. These discounted classification results are combined by an evidential combination rule for making the final class decision. The effectiveness of DAET for cross-domain pattern classification is evaluated with respect to some advanced DA methods, and the experiment results show DAET can significantly improve the classification accuracy.

HSELL-Net: A Heterogeneous Sample Enhancement Network With Lifelong Learning Under Industrial Small Samples

Small sample size leads to low accuracy and poor generalization of industrial fault diagnosis modeling. Domain adaptation (DA) attempts to enhance small samples by transferring samples in other similar domains, but it has limited application in industrial fault diagnosis, since the differences in working conditions lead to large variations of fault samples. To address the above issues, this article proposes a heterogeneous sample enhancement network with lifelong learning (HSELL-Net). First, a heterogeneous DA subnet (HDA-subnet) is presented, in which the designed heterogeneous supporting domain ensures dimension alignment and the designed distribution jointly matching improves the performance of distribution matching; thus, fault samples from other working conditions can be employed to reliably enhance small samples. Second, a lifelong learning subnet (LL-subnet) is designed, in which the proposed Admixup and shared knowledge repository enable incremental samples to further enhance small samples without retraining the network. The two subnets are mutually embedded and reinforced to enhance the number and types of small samples; thus, the accuracy and generalization of fault diagnosis under industrial small samples are improved. Finally, benchmark simulated experiments and real-world application experiments are conducted to evaluate the proposed method. Experimental results show the HSELL-Net outperforms the existing works under industrial small samples.

Advancing Imbalanced Domain Adaptation: Cluster-Level Discrepancy Minimization With a Comprehensive Benchmark

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

DAGAM: a domain adversarial graph attention model for subject-independent EEG-based emotion recognition

Objective.Due to individual differences in electroencephalogram (EEG) signals, the learning model built by the subject-dependent technique from one person's data would be inaccurate when applied to another person for emotion recognition. Thus, the subject-dependent approach for emotion recognition may result in poor generalization performance when compared to the subject-independent approach. However, existing studies have attempted but have not fully utilized EEG's topology, nor have they solved the problem caused by the difference in data distribution between the source and target domains.Approach.To eliminate individual differences in EEG signals, this paper proposes the domain adversarial graph attention model, a novel EEG-based emotion recognition model. The basic idea is to generate a graph using biological topology to model multichannel EEG signals. Graph theory can topologically describe and analyze EEG channel relationships and mutual dependencies. Then, unlike other graph convolutional networks, self-attention pooling is used to benefit from the extraction of salient EEG features from the graph, effectively improving performance. Finally, following graph pooling, the domain adversarial model based on the graph is used to identify and handle EEG variation across subjects, achieving good generalizability efficiently.Main Results.We conduct extensive evaluations on two benchmark data sets (SEED and SEED IV) and obtain cutting-edge results in subject-independent emotion recognition. Our model boosts the SEED accuracy to 92.59% (4.06% improvement) with the lowest standard deviation (STD) of 3.21% (2.46% decrements) and SEED IV accuracy to 80.74% (6.90% improvement) with the lowest STD of 4.14% (3.88% decrements), respectively. The computational complexity is drastically reduced in comparison to similar efforts (33 times lower).Significance.We have developed a model that significantly reduces the computation time while maintaining accuracy, making EEG-based emotion decoding more practical and generalizable.

Using transfer learning and dimensionality reduction techniques to improve generalisability of machine-learning predictions of mosquito ages from mid-infrared spectra

BACKGROUND

Old mosquitoes are more likely to transmit malaria than young ones. Therefore, accurate prediction of mosquito population age can drastically improve the evaluation of mosquito-targeted interventions. However, standard methods for age-grading mosquitoes are laborious and costly. We have shown that Mid-infrared spectroscopy (MIRS) can be used to detect age-specific patterns in mosquito cuticles and thus can be used to train age-grading machine learning models. However, these models tend to transfer poorly across populations. Here, we investigate whether applying dimensionality reduction and transfer learning to MIRS data can improve the transferability of MIRS-based predictions for mosquito ages.

METHODS

We reared adults of the malaria vector Anopheles arabiensis in two insectaries. The heads and thoraces of female mosquitoes were scanned using an attenuated total reflection-Fourier transform infrared spectrometer, which were grouped into two different age classes. The dimensionality of the spectra data was reduced using unsupervised principal component analysis or t-distributed stochastic neighbour embedding, and then used to train deep learning and standard machine learning classifiers. Transfer learning was also evaluated to improve transferability of the models when predicting mosquito age classes from new populations.

RESULTS

Model accuracies for predicting the age of mosquitoes from the same population as the training samples reached 99% for deep learning and 92% for standard machine learning. However, these models did not generalise to a different population, achieving only 46% and 48% accuracy for deep learning and standard machine learning, respectively. Dimensionality reduction did not improve model generalizability but reduced computational time. Transfer learning by updating pre-trained models with 2% of mosquitoes from the alternate population improved performance to ~ 98% accuracy for predicting mosquito age classes in the alternative population.

CONCLUSION

Combining dimensionality reduction and transfer learning can reduce computational costs and improve the transferability of both deep learning and standard machine learning models for predicting the age of mosquitoes. Future studies should investigate the optimal quantities and diversity of training data necessary for transfer learning and the implications for broader generalisability to unseen datasets.

Acoustic Emission and Artificial Intelligence Procedure for Crack Source Localization

The acoustic emission (AE) technique is one of the most widely used in the field of structural monitoring. Its popularity mainly stems from the fact that it belongs to the category of non-destructive techniques (NDT) and allows the passive monitoring of structures. The technique employs piezoelectric sensors to measure the elastic ultrasonic wave that propagates in the material as a result of the crack formation's abrupt release of energy. The recorded signal can be investigated to obtain information about the source crack, its position, and its typology (Mode I, Mode II). Over the years, many techniques have been developed for the localization, characterization, and quantification of damage from the study of acoustic emission. The onset time of the signal is an essential information item to be derived from waveform analysis. This information combined with the use of the triangulation technique allows for the identification of the crack location. In the literature, it is possible to find many methods to identify, with increasing accuracy, the onset time of the P-wave. Indeed, the precision of the onset time detection affects the accuracy of identifying the location of the crack. In this paper, two techniques for the definition of the onset time of acoustic emission signals are presented. The first method is based on the Akaike Information Criterion (AIC) while the second one relies on the use of artificial intelligence (AI). A recurrent convolutional neural network (R-CNN) designed for sound event detection (SED) is trained on three different datasets composed of seismic signals and acoustic emission signals to be tested on a real-world acoustic emission dataset. The new method allows taking advantage of the similarities between acoustic emissions, seismic signals, and sound signals, enhancing the accuracy in determining the onset time.

Rethinking Maximum Mean Discrepancy for Visual Domain Adaptation

Existing domain adaptation approaches often try to reduce distribution difference between source and target domains and respect domain-specific discriminative structures by some distribution [e.g., maximum mean discrepancy (MMD)] and discriminative distances (e.g., intra-class and inter-class distances). However, they usually consider these losses together and trade off their relative importance by estimating parameters empirically. It is still under insufficient exploration so far to deeply study their relationships to each other so that we cannot manipulate them correctly and the model's performance degrades. To this end, this article theoretically proves two essential facts: 1) minimizing MMD equals to jointly minimizing their data variance with some implicit weights but, respectively, maximizing the source and target intra-class distances so that feature discriminability degrades and 2) the relationship between intra-class and inter-class distances is as one falls and another rises. Based on this, we propose a novel discriminative MMD with two parallel strategies to correctly restrain the degradation of feature discriminability or the expansion of intra-class distance; specifically: 1) we directly impose a tradeoff parameter on the intra-class distance that is implicit in the MMD according to 1) and 2) we reformulate the inter-class distance with special weights that are analogical to those implicit ones in the MMD and maximizing it can also lead to the intra-class distance falling according to 2). Notably, we do not consider the two strategies in one model due to 2). The experiments on several benchmark datasets not only prove the validity of our revealed theoretical results but also demonstrate that the proposed approach could perform better than some compared state-of-art methods substantially. Our preliminary MATLAB code will be available at https://github.com/WWLoveTransfer/.

Semi-Supervised Heterogeneous Domain Adaptation: Theory and Algorithms

Semi-supervised heterogeneous domain adaptation (SsHeDA) aims to train a classifier for the target domain, in which only unlabeled and a small number of labeled data are available. This is done by leveraging knowledge acquired from a heterogeneous source domain. From algorithmic perspectives, several methods have been proposed to solve the SsHeDA problem; yet there is still no theoretical foundation to explain the nature of the SsHeDA problem or to guide new and better solutions. Motivated by compatibility condition in semi-supervised probably approximately correct (PAC) theory, we explain the SsHeDA problem by proving its generalization error - that is, why labeled heterogeneous source data and unlabeled target data help to reduce the target risk. Guided by our theory, we devise two algorithms as proof of concept. One, kernel heterogeneous domain alignment (KHDA), is a kernel-based algorithm; the other, joint mean embedding alignment (JMEA), is a neural network-based algorithm. When a dataset is small, KHDA's training time is less than JMEA's. When a dataset is large, JMEA is more accurate in the target domain. Comprehensive experiments with image/text classification tasks show KHDA to be the most accurate among all non-neural network baselines, and JMEA to be the most accurate among all baselines.

Guided Discrimination and Correlation Subspace Learning for Domain Adaptation

As a branch of transfer learning, domain adaptation leverages useful knowledge from a source domain to a target domain for solving target tasks. Most of the existing domain adaptation methods focus on how to diminish the conditional distribution shift and learn invariant features between different domains. However, two important factors are overlooked by most existing methods: 1) the transferred features should be not only domain invariant but also discriminative and correlated, and 2) negative transfer should be avoided as much as possible for the target tasks. To fully consider these factors in domain adaptation, we propose a guided discrimination and correlation subspace learning (GDCSL) method for cross-domain image classification. GDCSL considers the domain-invariant, category-discriminative, and correlation learning of data. Specifically, GDCSL introduces the discriminative information associated with the source and target data by minimizing the intraclass scatter and maximizing the interclass distance. By designing a new correlation term, GDCSL extracts the most correlated features from the source and target domains for image classification. The global structure of the data can be preserved in GDCSL because the target samples are represented by the source samples. To avoid negative transfer issues, we use a sample reweighting method to detect target samples with different confidence levels. A semi-supervised extension of GDCSL (Semi-GDCSL) is also proposed, and a novel label selection scheme is introduced to ensure the correction of the target pseudo-labels. Comprehensive and extensive experiments are conducted on several cross-domain data benchmarks. The experimental results verify the effectiveness of the proposed methods over state-of-the-art domain adaptation methods.

Subject-independent EEG emotion recognition with hybrid spatio-temporal GRU-Conv architecture

Recently, various deep learning frameworks have shown excellent performance in decoding electroencephalogram (EEG) signals, especially in human emotion recognition. However, most of them just focus on temporal features and ignore the features based on spatial dimensions. Traditional gated recurrent unit (GRU) model performs well in processing time series data, and convolutional neural network (CNN) can obtain spatial characteristics from input data. Therefore, this paper introduces a hybrid GRU and CNN deep learning framework named GRU-Conv to fully leverage the advantages of both. Nevertheless, contrary to most previous GRU architectures, we retain the output information of all GRU units. So, the GRU-Conv model could extract crucial spatio-temporal features from EEG data. And more especially, the proposed model acquires the multi-dimensional features of multi-units after temporal processing in GRU and then uses CNN to extract spatial information from the temporal features. In this way, the EEG signals with different characteristics could be classified more accurately. Finally, the subject-independent experiment shows that our model has good performance on SEED and DEAP databases. The average accuracy of the former is 87.04%. The mean accuracy of the latter is 70.07% for arousal and 67.36% for valence.

Discriminative Fisher Embedding Dictionary Transfer Learning for Object Recognition

In transfer learning model, the source domain samples and target domain samples usually share the same class labels but have different distributions. In general, the existing transfer learning algorithms ignore the interclass differences and intraclass similarities across domains. To address these problems, this article proposes a transfer learning algorithm based on discriminative Fisher embedding and adaptive maximum mean discrepancy (AMMD) constraints, called discriminative Fisher embedding dictionary transfer learning (DFEDTL). First, combining the label information of source domain and part of target domain, we construct the discriminative Fisher embedding model to preserve the interclass differences and intraclass similarities of training samples in transfer learning. Second, an AMMD model is constructed using atoms and profiles, which can adaptively minimize the distribution differences between source domain and target domain. The proposed method has three advantages: 1) using the Fisher criterion, we construct the discriminative Fisher embedding model between source domain samples and target domain samples, which encourages the samples from the same class to have similar coding coefficients; 2) instead of using the training samples to design the maximum mean discrepancy (MMD), we construct the AMMD model based on the relationship between the dictionary atoms and profiles; thus, the source domain samples can be adaptive to the target domain samples; and 3) the dictionary learning is based on the combination of source and target samples which can avoid the classification error caused by the difference among samples and reduce the tedious and expensive data annotation. A large number of experiments on five public image classification datasets show that the proposed method obtains better classification performance than some state-of-the-art dictionary and transfer learning methods. The code has been available at https://github.com/shilinrui/DFEDTL.

Semi-supervised bidirectional alignment for Remote Sensing cross-domain scene classification

Remote sensing (RS) image scene classification has obtained increasing attention for its broad application prospects. Conventional fully-supervised approaches usually require a large amount of manually-labeled data. As more and more RS images becoming available, how to make full use of these unlabeled data is becoming an urgent topic. Semi-supervised learning, which uses a few labeled data to guide the self-training of numerous unlabeled data, is an intuitive strategy. However, it is hard to apply it to cross-dataset (i.e., cross-domain) scene classification due to the significant domain shift among different datasets. To this end, semi-supervised domain adaptation (SSDA), which can reduce the domain shift and further transfer knowledge from a fully-labeled RS scene dataset (source domain) to a limited-labeled RS scene dataset (target domain), would be a feasible solution. In this paper, we propose an SSDA method termed bidirectional sample-class alignment (BSCA) for RS cross-domain scene classification. BSCA consists of two alignment strategies, unsupervised alignment (UA) and supervised alignment (SA), both of which can contribute to decreasing domain shift. UA concentrates on reducing the distance of maximum mean discrepancy across domains, with no demand for class labels. In contrast, SA aims to achieve the distribution alignment both from source samples to the associate target class centers and from target samples to the associate source class centers, with awareness of their classes. To validate the effectiveness of the proposed method, extensive ablation, comparison, and visualization experiments are conducted on an RS-SSDA benchmark built upon four widely-used RS scene classification datasets. Experimental results indicate that in comparison with some state-of-the-art methods, our BSCA achieves the superior cross-domain classification performance with compact feature representation and low-entropy classification boundary. Our code will be available at https://github.com/hw2hwei/BSCA.

A cross-domain fruit classification method based on lightweight attention networks and unsupervised domain adaptation

Image-based fruit classification offers many useful applications in industrial production and daily life, such as self-checkout in the supermarket, automatic fruit sorting and dietary guidance. However, fruit classification task will have different data distributions due to different application scenarios. One feasible solution to solve this problem is to use domain adaptation that adapts knowledge from the original training data (source domain) to the new testing data (target domain). In this paper, we propose a novel deep learning-based unsupervised domain adaptation method for cross-domain fruit classification. A hybrid attention module is proposed and added to MobileNet V3 to construct the HAM-MobileNet that can suppress the impact of complex backgrounds and extract more discriminative features. A hybrid loss function combining subdomain alignment and implicit distribution metrics is used to reduce domain discrepancy during model training and improve model classification performance. Two fruit classification datasets covering several domains are established to simulate common industrial and daily life application scenarios. We validate the proposed method on our constructed grape classification dataset and general fruit classification dataset. The experimental results show that the proposed method achieves an average accuracy of 95.0% and 93.2% on the two datasets, respectively. The classification model after domain adaptation can well overcome the domain discrepancy brought by different fruit classification scenarios. Meanwhile, the proposed datasets and method can serve as a benchmark for future cross-domain fruit classification research.

Domain Adaptation Principal Component Analysis: Base Linear Method for Learning with Out-of-Distribution Data

Domain adaptation is a popular paradigm in modern machine learning which aims at tackling the problem of divergence (or shift) between the labeled training and validation datasets (source domain) and a potentially large unlabeled dataset (target domain). The task is to embed both datasets into a common space in which the source dataset is informative for training while the divergence between source and target is minimized. The most popular domain adaptation solutions are based on training neural networks that combine classification and adversarial learning modules, frequently making them both data-hungry and difficult to train. We present a method called Domain Adaptation Principal Component Analysis (DAPCA) that identifies a linear reduced data representation useful for solving the domain adaptation task. DAPCA algorithm introduces positive and negative weights between pairs of data points, and generalizes the supervised extension of principal component analysis. DAPCA is an iterative algorithm that solves a simple quadratic optimization problem at each iteration. The convergence of the algorithm is guaranteed, and the number of iterations is small in practice. We validate the suggested algorithm on previously proposed benchmarks for solving the domain adaptation task. We also show the benefit of using DAPCA in analyzing single-cell omics datasets in biomedical applications. Overall, DAPCA can serve as a practical preprocessing step in many machine learning applications leading to reduced dataset representations, taking into account possible divergence between source and target domains.

Ambient air pollutants concentration prediction during the COVID-19: A method based on transfer learning

Research on the correlation analysis between COVID-19 and air pollution has attracted increasing attention since the COVID-19 pandemic. While many relevant issues have been widely studied, research into ambient air pollutant concentration prediction (APCP) during COVID-19 is still in its infancy. Most of the existing study on APCP is based on machine learning methods, which are not suitable for APCP during COVID-19 due to the different distribution of historical observations before and after the pandemic. Therefore, to fulfill the predictive task based on the historical observations with a different distribution, this paper proposes an improved transfer learning model combined with machine learning for APCP during COVID-19. Specifically, this paper employs the Gaussian mixture method and an optimization algorithm to obtain a new source domain similar to the target domain for further transfer learning. Then, several commonly used machine learning models are trained in the new source domain, and these well-trained models are transferred to the target domain to obtain APCP results. Based on the real-world dataset, the experimental results suggest that, by using the improved machine learning methods based on transfer learning, our method can achieve the prediction with significantly high accuracy. In terms of managerial insights, the effects of influential factors are analyzed according to the relationship between these influential factors and prediction results, while their importance is ranked through their average marginal contribution and partial dependence plots.

Classification of motor imagery electroencephalogram signals by using adaptive cross-subject transfer learning

Introduction

Electroencephalogram (EEG)-based motor imagery (MI) classification is an important aspect in brain-computer interfaces (BCIs), which bridges between neural system and computer devices decoding brain signals into recognizable machine commands. However, due to the small number of training samples of MI electroencephalogram (MI-EEG) for a single subject and the great individual differences of MI-EEG among different subjects, the generalization and accuracy of the model on the specific MI task may be poor.

Methods

To solve these problems, an adaptive cross-subject transfer learning algorithm is proposed, which is based on kernel mean matching (KMM) and transfer learning adaptive boosting (TrAdaBoost) method. First, the common spatial pattern (CSP) is used to extract the spatial features. Then, in order to make the feature distribution more similar among different subjects, the KMM algorithm is used to compute a sample weight matrix for aligning the mean between source and target domains and reducing distribution differences among different subjects. Finally, the sample weight matrix from KMM is used as the initialization weight of TrAdaBoost, and then TrAdaBoost is used to adaptively select source domain samples that are closer to the target task distribution to assist in building a classification model.

Results

In order to verify the effectiveness and feasibility of the proposed method, the algorithm is applied to BCI Competition IV datasets and in-house datasets. The results show that the average classification accuracy of the proposed method on the public datasets is 89.1%, and the average classification accuracy on the in-house datasets is 80.4%.

Discussion

Compared with the existing methods, the proposed method effectively improves the classification accuracy of MI-EEG signals. At the same time, this paper also applies the proposed algorithm to the in-house dataset, the results verify the effectiveness of the algorithm again, and the results of this study have certain clinical guiding significance for brain rehabilitation.

Probability-Based Graph Embedding Cross-Domain and Class Discriminative Feature Learning for Domain Adaptation

Feature-based domain adaptation methods project samples from different domains into the same feature space and try to align the distribution of two domains to learn an effective transferable model. The vital problem is how to find a proper way to reduce the domain shift and improve the discriminability of features. To address the above issues, we propose a unified Probability-based Graph embedding Cross-domain and class Discriminative feature learning framework for unsupervised domain adaptation (PGCD). Specifically, we propose novel graph embedding structures to be the class discriminative transfer feature learning item and cross-domain alignment item, which can make the same-category samples compact in each domain, and fully align the local and global geometric structure across domains. Besides, two theoretical analyses are given to prove the interpretability of the proposed graph structures, which can further describe the relationships between samples to samples in single-domain and cross-domain transfer feature learning scenarios. Moreover, we adopt novel weight strategies via probability information to generate robust centroids in each proposed item to enhance the accuracy of transfer feature learning and reduce the error accumulation. Compared with the advanced approaches by comprehensive experiments, the promising performance on the benchmark datasets verify the effectiveness of the proposed model.

Near infrared spectroscopy quantification based on Bi-LSTM and transfer learning for new scenarios

This study proposed a deep transfer learning methodology based on an improved Bi-directional Long Short-Term Memory (Bi-LSTM) network for the first time to address the near infrared spectroscopy (NIR) model transfer issue between samples. We tested its effectiveness on two datasets of manure and polyglutamic acid (γ-PGA) solution, respectively. First, the optimal primary Bi-LSTM networks for cattle manure and the first batch of γ-PGA were developed by ablation experiments and both proved to outperform one-dimensional convolutional neural network (1D-CNN), Partial Least Square (PLS) and Extreme Learning Machine (ELM) models. Then, two types of transfer learning approaches were carried out to determine model transferability to non-homologous samples. For poultry manure and the second batch of γ-PGA, the obtained predicting results verified that the second approach of fine-tuning Bi-LSTM layers and re-training FC layers transcended the first approach of fixing Bi-LSTM layers and only re-training FC layers by reducing the RMSEP_target of 23.4275% and 50.7343%, respectively. Finally, comparisons with fine-tuning 1D-CNN and other traditional model transfer methods further identified the superiority of the proposed methodology with exceeding accuracy and smaller variation, which decreased RMSEP_target of poultry manure and the second batch of γ-PGA of 7.2832% and 48.1256%, 67.1117% and 80.6924% when compared to that acquired by fine-tuning 1D-CNN, Tradaboost-ELM and CCA-PLS which were the best of five traditional methods, respectively. The study demonstrates the potential of the Fine-tuning-Bi-LSTM enabled NIR technology to be used as a simple, cost effective and reliable detection tool for a wide range of applications under various new scenarios.

Effective Transfer Learning Algorithm in Spiking Neural Networks

As the third generation of neural networks, spiking neural networks (SNNs) have gained much attention recently because of their high energy efficiency on neuromorphic hardware. However, training deep SNNs requires many labeled data that are expensive to obtain in real-world applications, as traditional artificial neural networks (ANNs). In order to address this issue, transfer learning has been proposed and widely used in traditional ANNs, but it has limited use in SNNs. In this article, we propose an effective transfer learning framework for deep SNNs based on the domain in-variance representation. Specifically, we analyze the rationality of centered kernel alignment (CKA) as a domain distance measurement relative to maximum mean discrepancy (MMD) in deep SNNs. In addition, we study the feature transferability across different layers by testing on the Office-31, Office-Caltech-10, and PACS datasets. The experimental results demonstrate the transferability of SNNs and show the effectiveness of the proposed transfer learning framework by using CKA in SNNs.

A Deep Probabilistic Transfer Learning Framework for Soft Sensor Modeling With Missing Data

Soft sensors have been extensively developed and applied in the process industry. One of the main challenges of the data-driven soft sensors is the lack of labeled data and the need to absorb the knowledge from a related source operating condition to enhance the soft sensing performance on the target application. This article introduces deep transfer learning to soft sensor modeling and proposes a deep probabilistic transfer regression (DPTR) framework. In DPTR, a deep generative regression model is first developed to learn Gaussian latent feature representations and model the regression relationship under the stochastic gradient variational Bayes framework. Then, a probabilistic latent space transfer strategy is designed to reduce the discrepancy between the source and target latent features such that the knowledge from the source data can be explored and transferred to enhance the target soft sensor performance. Besides, considering the missing values in the process data in the target operating condition, the DPTR is further extended to handle the missing data problem utilizing the strong generation and reconstruction capability of the deep generative model. The effectiveness of the proposed method is validated through an industrial multiphase flow process.

Optimal transport- and kernel-based early detection of mild cognitive impairment patients based on magnetic resonance and positron emission tomography images

Background

To help clinicians provide timely treatment and delay disease progression, it is crucial to identify dementia patients during the mild cognitive impairment (MCI) stage and stratify these MCI patients into early and late MCI stages before they progress to Alzheimer’s disease (AD). In the process of diagnosing MCI and AD in living patients, brain scans are collected using neuroimaging technologies such as computed tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET). These brain scans measure the volume and molecular activity within the brain resulting in a very promising avenue to diagnose patients early in a minimally invasive manner.

Methods

We have developed an optimal transport based transfer learning model to discriminate between early and late MCI. Combing this transfer learning model with bootstrap aggregation strategy, we overcome the overfitting problem and improve model stability and prediction accuracy.

Results

With the transfer learning methods that we have developed, we outperform the current state of the art MCI stage classification frameworks and show that it is crucial to leverage Alzheimer’s disease and normal control subjects to accurately predict early and late stage cognitive impairment.

Conclusions

Our method is the current state of the art based on benchmark comparisons. This method is a necessary technological stepping stone to widespread clinical usage of MRI-based early detection of AD.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13195-021-00915-3).

Semi-supervised adversarial discriminative domain adaptation

Domain adaptation is a potential method to train a powerful deep neural network across various datasets. More precisely, domain adaptation methods train the model on training data and test that model on a completely separate dataset. The adversarial-based adaptation method became popular among other domain adaptation methods. Relying on the idea of GAN, the adversarial-based domain adaptation tries to minimize the distribution between the training and testing dataset based on the adversarial learning process. We observe that the semi-supervised learning approach can combine with the adversarial-based method to solve the domain adaptation problem. In this paper, we propose an improved adversarial domain adaptation method called Semi-Supervised Adversarial Discriminative Domain Adaptation (SADDA), which can outperform other prior domain adaptation methods. We also show that SADDA has a wide range of applications and illustrate the promise of our method for image classification and sentiment classification problems.

Co-Occurrence Fingerprint Data-Based Heterogeneous Transfer Learning Framework for Indoor Positioning

Distribution discrepancy is an intrinsic challenge in existing fingerprint-based indoor positioning system(s) (FIPS) due to real-time environmental variations; thus, the positioning model needs to be reconstructed frequently based on newly collected training data. However, it is expensive or impossible to collect adequate training samples to reconstruct the fingerprint database. Fortunately, transfer learning has proven to be an effective solution to mitigate the distribution discrepancy, enabling us to update the positioning model using newly collected training data in real time. However, in practical applications, traditional transfer learning algorithms no longer act well to feature space heterogeneity caused by different types or holding postures of fingerprint collection devices (such as smartphones). Moreover, current heterogeneous transfer methods typically require enough accurately labeled samples in the target domain, which is practically expensive and even unavailable. Aiming to solve these problems, a heterogeneous transfer learning framework based on co-occurrence data (HTL-CD) is proposed for FIPS, which can realize higher positioning accuracy and robustness against environmental changes without reconstructing the fingerprint database repeatedly. Specifically, the source domain samples are mapped into the feature space in the target domain, then the marginal and conditional distributions of the source and target samples are aligned in order to minimize the distribution divergence caused by collection device heterogeneity and environmental changes. Moreover, the utilized co-occurrence fingerprint data enables us to calculate correlation coefficients between heterogeneous samples without accurately labeled target samples. Furthermore, by resorting to the adopted correlation restriction mechanism, more valuable knowledge will be transferred to the target domain if the source samples are related to the target ones, which remarkably relieves the "negative transfer" issue. Real-world experimental performance implies that, even without accurately labeled samples in the target domain, the proposed HTL-CD can obtain at least 17.15% smaller average localization errors (ALEs) than existing transfer learning-based positioning methods, which further validates the effectiveness and superiority of our algorithm.

Attention Regularized Laplace Graph for Domain Adaptation

In leveraging manifold learning in domain adaptation (DA), graph embedding-based DA methods have shown their effectiveness in preserving data manifold through the Laplace graph. However, current graph embedding DA methods suffer from two issues: 1). they are only concerned with preservation of the underlying data structures in the embedding and ignore sub-domain adaptation, which requires taking into account intra-class similarity and inter-class dissimilarity, thereby leading to negative transfer; 2). manifold learning is proposed across different feature/label spaces separately, thereby hindering unified comprehensive manifold learning. In this paper, starting from our previous DGA-DA, we propose a novel DA method, namely A ttention R egularized Laplace G raph-based D omain A daptation (ARG-DA), to remedy the aforementioned issues. Specifically, by weighting the importance across different sub-domain adaptation tasks, we propose the A ttention R egularized Laplace Graph for class aware DA, thereby generating the attention regularized DA. Furthermore, using a specifically designed FEEL strategy, our approach dynamically unifies alignment of the manifold structures across different feature/label spaces, thus leading to comprehensive manifold learning. Comprehensive experiments are carried out to verify the effectiveness of the proposed DA method, which consistently outperforms the state of the art DA methods on 7 standard DA benchmarks, i.e., 37 cross-domain image classification tasks including object, face, and digit images. An in-depth analysis of the proposed DA method is also discussed, including sensitivity, convergence, and robustness.