Semi-Supervised Domain Adaptation for Multi-Label Classification on Nonintrusive Load Monitoring

Nonintrusive load monitoring (NILM) is a technology that analyzes the load consumption and usage of an appliance from the total load. NILM is becoming increasingly important because residential and commercial power consumption account for about 60% of global energy consumption. Deep neural network-based NILM studies have increased rapidly as hardware computation costs have decreased. A significant amount of labeled data is required to train deep neural networks. However, installing smart meters on each appliance of all households for data collection requires the cost of geometric series. Therefore, it is urgent to detect whether the appliance is used from the total load without installing a separate smart meter. In other words, domain adaptation research, which can interpret the huge complexity of data and generalize information from various environments, has become a major challenge for NILM. In this research, we optimize domain adaptation by employing techniques such as robust knowledge distillation based on teacher-student structure, reduced complexity of feature distribution based on gkMMD, TCN-based feature extraction, and pseudo-labeling-based domain stabilization. In the experiments, we down-sample the UK-DALE and REDD datasets as in the real environment, and then verify the proposed model in various cases and discuss the results.

Concept Drift-Tolerant Transfer Learning in Dynamic Environments

Existing transfer learning methods that focus on problems in stationary environments are not usually applicable to dynamic environments, where concept drift may occur. To the best of our knowledge, the concept drift-tolerant transfer learning (CDTL), whose major challenge is the need to adapt the target model and knowledge of source domains to the changing environments, has yet to be well explored in the literature. This article, therefore, proposes a hybrid ensemble approach to deal with the CDTL problem provided that data in the target domain are generated in a streaming chunk-by-chunk manner from nonstationary environments. At each time step, a class-wise weighted ensemble is presented to adapt the model of target domains to new environments. It assigns a weight vector for each classifier generated from the previous data chunks to allow each class of the current data leveraging historical knowledge independently. Then, a domain-wise weighted ensemble is introduced to combine the source and target models to select useful knowledge of each domain. The source models are updated with the source instances performed by the proposed adaptive weighted CORrelation ALignment (AW-CORAL). AW-CORAL iteratively minimizes domain discrepancy meanwhile decreases the effect of unrelated source instances. In this way, positive knowledge of source domains can be potentially promoted while negative knowledge is reduced. Empirical studies on synthetic and real benchmark data sets demonstrate the effectiveness of the proposed algorithm.

Active Domain Adaptation With Application to Intelligent Logging Lithology Identification

Lithology identification plays an essential role in formation characterization and reservoir exploration. As an emerging technology, intelligent logging lithology identification has received great attention recently, which aims to infer the lithology type through the well-logging curves using machine-learning methods. However, the model trained on the interpreted logging data is not effective in predicting new exploration well due to the data distribution discrepancy. In this article, we aim to train a lithology identification model for the target well using a large amount of source-labeled logging data and a small amount of target-labeled data. The challenges of this task lie in three aspects: 1) the distribution misalignment; 2) the data divergence; and 3) the cost limitation. To solve these challenges, we propose a novel active adaptation for logging lithology identification (AALLI) framework that combines active learning (AL) and domain adaptation (DA). The contributions of this article are three-fold: 1) the domain-discrepancy problem in intelligent logging lithology identification is first investigated in this article, and a novel framework that incorporates AL and DA into lithology identification is proposed to handle the problem; 2) we design a discrepancy-based AL and pseudolabeling (PL) module and an instance importance weighting module to query the most uncertain target information and retain the most confident source information, which solves the challenges of cost limitation and distribution misalignment; and 3) we develop a reliability detecting module to improve the reliability of target pseudolabels, which, together with the discrepancy-based AL and PL module, solves the challenge of data divergence. Extensive experiments on three real-world well-logging datasets demonstrate the effectiveness of the proposed method compared to the baselines.

A New Belief-Based Bidirectional Transfer Classification Method

In pattern classification, we may have a few labeled data points in the target domain, but a number of labeled samples are available in another related domain (called the source domain). Transfer learning can solve such classification problems via the knowledge transfer from source to target domains. The source and target domains can be represented by heterogeneous features. There may exist uncertainty in domain transformation, and such uncertainty is not good for classification. The effective management of uncertainty is important for improving classification accuracy. So, a new belief-based bidirectional transfer classification (BDTC) method is proposed. In BDTC, the intraclass transformation matrix is estimated at first for mapping the patterns from source to target domains, and this matrix can be learned using the labeled patterns of the same class represented by heterogeneous domains (features). The labeled patterns in the source domain are transferred to the target domain by the corresponding transformation matrix. Then, we learn a classifier using all the labeled patterns in the target domain to classify the objects. In order to take full advantage of the complementary knowledge of different domains, we transfer the query patterns from target to source domains using the K-NN technique and do the classification task in the source domain. Thus, two pieces of classification results can be obtained for each query pattern in the source and target domains, but the classification results may have different reliabilities/weights. A weighted combination rule is developed to combine the two classification results based on the belief functions theory, which is an expert at dealing with uncertain information. We can efficiently reduce the uncertainty of transfer classification via the combination strategy. Experiments on some domain adaptation benchmarks show that our method can effectively improve classification accuracy compared with other related methods.

Random Channel Masks for Regularization of Least Squares-Based Finger EMG-Force Modeling to Improve Cross-Day Performance

Estimating the finger forces from surface electromyography (sEMG) is essential for diverse applications (e.g., human-machine interfacing). The performance of pre-trained sEMG-force models degenerates significantly when applied on a second day, due to the large cross-day variation of sEMG characteristics. Previous studies mainly employed transfer learning algorithms to tackle this problem. However, transfer learning algorithms normally require data collected on the second day for model calibration, increasing the inconvenience in practical use. In this work, we investigated the effect of model regularization on this issue. Specifically, 256-channel high-density sEMG (HDsEMG) signals with varying finger forces were collected on different days (3-25 days apart). We applied randomly generated channel perturbations ("masks") to feature maps of randomly selected channels in training dataset. The channel masks of the training set were generated randomly and independently in each narrow time window (~20 ms). We assumed that by learning from randomly masked feature maps (randomness is the central aspect), the model would not be biased by a small number of features but would be based on learning from a global perspective, therefore avoiding overfitting to the within-day EMG patterns. Moore-Penrose inverse model regularization was also employed as a baseline method, with results showing that cross-day EMG-force models require a higher tolerance parameter compared with within-day applications. In combination with the Moore-Penrose inverse model regularization, further applying random channel masks to the training set significantly improved model performance in cross-day validation.

Single-Source to Single-Target Cross-Subject Motor Imagery Classification Based on Multisubdomain Adaptation Network

In the electroencephalography (EEG) based cross-subject motor imagery (MI) classification task, the device and subject problems can cause the time-related data distribution shift problem. In a single-source to single-target (STS) MI classification task, such a shift problem will certainly provoke an increase in the overall data distribution difference between the source and target domains, giving rise to poor classification accuracy. In this paper, a novel multi-subdomain adaptation method (MSDAN) is proposed to solve the shift problem and improve the classification accuracy of the traditional approaches. In the proposed MSDAN, the adaptation losses in both class-related and time-related subdomains (that are divided by different data labels and session labels) are obtained by measuring the distribution differences between the source and target subdomains. Then, the adaptation and classification losses in the loss function of MSDAN are minimized concurrently. To illustrate the application value of the proposed method, our method is applied to solve the STS MI classification task about data analysis with respect to the brain-computer interface (BCI) competition III-IVa dataset. The resultant experiment results demonstrate that compared with other well-known domain adaptation and deep learning methods, the proposed method is capable of solving the time-related data distribution problem at higher classification accuracy.

TSTELM: Two-Stage Transfer Extreme Learning Machine for Unsupervised Domain Adaptation

As a single-layer feedforward network (SLFN), extreme learning machine (ELM) has been successfully applied for classification and regression in machine learning due to its faster training speed and better generalization. However, it will perform poorly for domain adaptation in which the distributions between training data and testing data are inconsistent. In this article, we propose a novel ELM called two-stage transfer extreme learning machine (TSTELM) to solve this problem. At the statistical matching stage, we adopt maximum mean discrepancy (MMD) to narrow the distribution difference of the output layer between domains. In addition, at the subspace alignment stage, we align the source and target model parameters, design target cross-domain mean approximation, and add the output weight approximation to further promote the knowledge transferring across domains. Moreover, the prediction of test sample is jointly determined by the ELM parameters generated at the two stages. Finally, we investigate the proposed approach in classification task and conduct experiments on four public domain adaptation datasets. The result indicates that TSTELM could effectively enhance the knowledge transfer ability of ELM with higher accuracy than other existing transfer and non-transfer classifiers.

Domain Adaptation with Data Uncertainty Measure Based on Evidence Theory

Domain adaptation aims to learn a classifier for a target domain task by using related labeled data from the source domain. Because source domain data and target domain task may be mismatched, there is an uncertainty of source domain data with respect to the target domain task. Ignoring the uncertainty may lead to models with unreliable and suboptimal classification results for the target domain task. However, most previous works focus on reducing the gap in data distribution between the source and target domains. They do not consider the uncertainty of source domain data about the target domain task and cannot apply the uncertainty to learn an adaptive classifier. Aimed at this problem, we revisit the domain adaptation from source domain data uncertainty based on evidence theory and thereby devise an adaptive classifier with the uncertainty measure. Based on evidence theory, we first design an evidence net to estimate the uncertainty of source domain data about the target domain task. Second, we design a general loss function with the uncertainty measure for the adaptive classifier and extend the loss function to support vector machine. Finally, numerical experiments on simulation datasets and real-world applications are given to comprehensively demonstrate the effectiveness of the adaptive classifier with the uncertainty measure.

Feature Extraction and Intelligent Text Generation of Digital Music

Because the current network music operation mechanism is constantly improving and the matching of music platforms and users is poor, in this paper, the characteristics of digital music are analyzed, and the music features, rhythm, tune, intensity, and timbre with the MIDI format are extracted. Then, a music feature information extraction algorithm based on neural networks is proposed, and according to the extracted information of the music style, the B2T model is adopted for intelligent text generation. Finally, test results are given by the style matching rate and ROUGE value, which show that the model is accurate and effective for classification of music and description of related text, and the extraction of music feature information has a certain influence on its intelligent text generation.

MetaDetector: Meta Event Knowledge Transfer for Fake News Detection

The blooming of fake news on social networks has devastating impacts on society, economy, and public security. Although numerous studies are conducted for the automatic detection of fake news, the majority tend to utilize deep neural networks to learn event-specific features for superior detection performance on specific datasets. However, the trained models heavily rely on the training datasets and are infeasible to apply to upcoming events due to the discrepancy between event distributions. Inspired by domain adaptation theories, we propose an end-to-end adversarial adaptation network, dubbed as MetaDetector , to transfer meta knowledge (event-shared features) between different events. Specifically, MetaDetector pushes the feature extractor and event discriminator to eliminate event-specific features and preserve required meta knowledge by adversarial training. Furthermore, the pseudo-event discriminator is utilized to evaluate the importance of news records in historical events to obtain partial knowledge that are discriminative for detecting fake news. Under the coordinated optimization among all the submodules, MetaDetector accurately transfers the meta knowledge of historical events to the upcoming event for fact checking. We conduct extensive experiments on two real-world datasets collected from Sina Weibo and Twitter. The experimental results demonstrate that MetaDetector outperforms the state-of-the-art methods, especially when the distribution discrepancy between events is significant.

Sliding-Window Normalization to Improve the Performance of Machine-Learning Models for Real-Time Motion Prediction Using Electromyography

Many researchers have used machine learning models to control artificial hands, walking aids, assistance suits, etc., using the biological signal of electromyography (EMG). The use of such devices requires high classification accuracy. One method for improving the classification performance of machine learning models is normalization, such as z-score. However, normalization is not used in most EMG-based motion prediction studies because of the need for calibration and fluctuation of reference value for calibration (cannot re-use). Therefore, in this study, we proposed a normalization method that combines sliding-window and z-score normalization that can be implemented in real-time processing without need for calibration. The effectiveness of this normalization method was confirmed by conducting a single-joint movement experiment of the elbow and predicting its rest, flexion, and extension movements from the EMG signal. The proposed method achieved 77.7% accuracy, an improvement of 21.5% compared to the non-normalization (56.2%). Furthermore, when using a model trained by other people’s data for application without calibration, the proposed method achieved 63.1% accuracy, an improvement of 8.8% compared to the z-score (54.4%). These results showed the effectiveness of the simple and easy-to-implement method, and that the classification performance of the machine learning model could be improved.

Evolutionary Dynamic Multiobjective Optimization via Learning From Historical Search Process

Dynamic multiobjective optimization problems are challenging due to their fast convergence and diversity maintenance requirements. Prediction-based evolutionary algorithms currently gain much attention for meeting these requirements. However, it is not always the case that an elaborate predictor is suitable for different problems and the quality of historical solutions is sufficient to support prediction, which limits the availability of prediction-based methods over various problems. Faced with these issues, this article proposes a knowledge learning strategy for change response in the dynamic multiobjective optimization. Unlike prediction approaches that estimate the future optima from previously obtained solutions, in the proposed strategy, we react to changes via learning from the historical search process. We introduce a method to extract the knowledge within the previous search experience. The extracted knowledge can accelerate convergence as well as introduce diversity for the optimization of the future environment. We conduct a comprehensive experiment on comparing the proposed strategy with the state-of-the-art algorithms. Results demonstrate the better performance of the proposed strategy in terms of solution quality and computational efficiency.

A meta-learning BCI for estimating decision confidence

OBJECTIVE

We investigated whether a recently introduced transfer-learning technique based on meta-learning could improve the performance of Brain-Computer Interfaces (BCIs) for decision-confidence prediction with respect to more traditional machine learning methods.

APPROACH

We adapted the meta-learning by biased regularisation algorithm to the problem of predicting decision confidence from EEG and EOG data on a decision-by-decision basis in a difficult target discrimination task based on video feeds. The method exploits previous participants' data to produce a prediction algorithm that is then quickly tuned to new participants. We compared it with with the traditional single-subject training almost universally adopted in BCIs, a state-of-the-art transfer learning technique called Domain Adversarial Neural Networks (DANN), a transfer-learning adaptation of a zero-training method we used recently for a similar task, and with a simple baseline algorithm.

MAIN RESULTS

The meta-learning approach was significantly better than other approaches in most conditions, and much better in situations where limited data from a new participant are available for training/tuning. Meta-learning by biased regularisation allowed our BCI to seamlessly integrate information from past participants with data from a specific user to produce high-performance predictors. Its robustness in the presence of small training sets is a real-plus in BCI applications, as new users need to train the BCI for a much shorter period.

SIGNIFICANCE

Due to the variability and noise of EEG/EOG data, BCIs need to be normally trained with data from a specific participant. This work shows that even better performance can be obtained using our version of meta-learning by biased regularisation.

Transfer Adaptation Learning: A Decade Survey

The world we see is ever-changing and it always changes with people, things, and the environment. Domain is referred to as the state of the world at a certain moment. A research problem is characterized as transfer adaptation learning (TAL) when it needs knowledge correspondence between different moments/domains. TAL aims to build models that can perform tasks of target domain by learning knowledge from a semantic-related but distribution different source domain. It is an energetic research field of increasing influence and importance, which is presenting a blowout publication trend. This article surveys the advances of TAL methodologies in the past decade, and the technical challenges and essential problems of TAL have been observed and discussed with deep insights and new perspectives. Broader solutions of TAL being created by researchers are identified, i.e., instance reweighting adaptation, feature adaptation, classifier adaptation, deep network adaptation, and adversarial adaptation, which are beyond the early semisupervised and unsupervised split. The survey helps researchers rapidly but comprehensively understand and identify the research foundation, research status, theoretical limitations, future challenges, and understudied issues (universality, interpretability, and credibility) to be broken in the field toward generalizable representation in open-world scenarios.

Online Domain Adaptation for Rolling Bearings Fault Diagnosis with Imbalanced Cross-Domain Data

Traditional machine learning methods rely on the training data and target data having the same feature space and data distribution. The performance may be unacceptable if there is a difference in data distribution between the training and target data, which is called cross-domain learning problem. In recent years, many domain adaptation methods have been proposed to solve this kind of problems and make much progress. However, existing domain adaptation approaches have a common assumption that the number of the data in source domain (labeled data) and target domain (unlabeled data) is matched. In this paper, the scenarios in real manufacturing site are considered, that the target domain data is much less than source domain data at the beginning, but the number of target domain data will increase as time goes by. A novel method is proposed for fault diagnosis of rolling bearing with online imbalanced cross-domain data. Finally, the proposed method which is tested on bearing dataset (CWRU) has achieved prediction accuracy of 95.89% with only 40 target samples. The results have been compared with other traditional methods. The comparisons show that the proposed online domain adaptation fault diagnosis method has achieved significant improvements. In addition, the deep transfer learning model by adaptive- network-based fuzzy inference system (ANFIS) is introduced to interpretation the results.

Adversarial Auto-encoder Domain Adaptation for Cold-start Recommendation with Positive and Negative Hypergraphs

This paper presents a novel model named Adversarial Auto-encoder Domain Adaptation (AADA) to handle the recommendation problem under cold-start settings. Specifically, we divide the hypergraph into two hypergraphs, i.e., a positive hypergraph and a negative one. Below, we adopt the cold-start user recommendation for illustration. After achieving positive and negative hypergraphs, we apply hypergraph auto-encoders to them to obtain positive and negative embeddings of warm users and items. Additionally, we employ a multi-layer perceptron to get warm and cold-start user embeddings called regular embeddings. Subsequently, for warm users, we assign positive and negative pseudo-labels to their positive and negative embeddings, respectively, and treat their positive and regular embeddings as the source and target domain data, respectively. Then, we develop a matching discriminator to jointly minimize the classification loss of the positive and negative warm user embeddings and the distribution gap between the positive and regular warm user embeddings. In this way, warm users’ positive and regular embeddings are connected. Since the positive hypergraph maintains the relations between positive warm user and item embeddings, and the regular warm and cold-start user embeddings follow a similar distribution, the regular cold-start user embedding and positive item embedding are bridged to discover their relationship. The proposed model can be easily extended to handle the cold-start item recommendation by changing inputs. We perform extensive experiments on real-world datasets for both cold-start user and cold-start item recommendations. Promising results in terms of precision, recall, NDCG, and hit rate verify the effectiveness of the proposed method.

Cross-scenario transfer diagnosis of reciprocating compressor based on CBAM and ResNet

To address data distribution discrepancy across scenarios, deep transfer learning is used to help the target scenario complete the recognition task using similar scenario data. However, fault misrecognition or low diagnostic accuracy occurs due to the weak expression of the deep transfer model in cross-scenario application. The Convolutional Block Attention Module (CBAM) can independently learn the importance of each channel and space features, recalibrate the channel and space features, and improve image classification performance. This study introduces the CBAM module using the Residual Network (ResNet), and proposes a transfer learning model that combines the CBAM module with an improved ResNet, denoted as TL_CBAM_ResNet17. A miniature ResNet17 deep model is constructed based on the ResNet50 model. The location of the CBAM module embedded in the ResNet17 model is determined to strengthen model expression. For effective cross-scenario transfer and reduced data distribution discrepancy between source and target domains, a multi-kernel Maximum Mean Discrepancy (MK–MMD) layer is added in front of the classifier layer in the ResNet17 model to select data with common domain features. Considering a reciprocating compressor as the research object, cross-scenario datasets are produced by the vibration signals from the simulation test bench and simulation signals from the dynamic simulation model. Mutual transfer experiments are conducted using these datasets. The proposed method (TL_CBAM_ResNet17) demonstrates better classification performance than TCA, JDA, the TL_ResNet50 model, the TL_ResNet17 model, and the TL_ResNet17 model integrated with other attention mechanism module, and greatly improves the accuracy of fault diagnosis and generalization of the model in cross-scenario applications.

Multi-fidelity surrogate modeling through hybrid machine learning for biomechanical and finite element analysis of soft tissues

Biomechanical simulation enables medical researchers to study complex mechano-biological conditions, although for soft tissue modeling, it may apply highly nonlinear multi-physics theories commonly implemented by expensive finite element (FE) solvers. This is a significantly time-consuming process on a regular computer and completely inefficient in urgent situations. One remedy is to first generate a dataset of the possible inputs and outputs of the solver in order to then train an efficient machine learning (ML) model, i.e., the supervised ML-based surrogate, replacing the expensive solver to speed up the simulation. But it still requires a large number of expensive numerical samples. In this regard, we propose a hybrid ML (HML) method that uses a reduced-order model defined by the simplification of the complex multi-physics equations to produce a dataset of the low-fidelity (LF) results. The surrogate then has this efficient numerical model and an ML model that should increase the fidelity of its outputs to the level of high-fidelity (HF) results. Based on our empirical tests via a group of diverse training and numerical modeling conditions, the proposed method can improve training convergence for very limited training samples. In particular, while considerable time gains comparing to the HF numerical models are observed, training of the HML models is also significantly more efficient than the purely ML-based surrogates. From this, we conclude that this non-destructive HML implementation may increase the accuracy and efficiency of surrogate modeling of soft tissues with complex multi-physics properties in small data regimes.

Online Unsupervised Domain Adaptation via Reducing Inter- and Intra-Domain Discrepancies

Unsupervised domain adaptation (UDA) transfers knowledge from a labeled source domain to an unlabeled target domain on cross-domain object recognition by reducing a distribution discrepancy between the source and target domains (interdomain discrepancy). Prevailing methods on UDA were presented based on the premise that target data are collected in advance. However, in online scenarios, the target data often arrive in a streamed manner, such as visual image recognition in daily monitoring, which means that there is a distribution discrepancy between incoming target data and collected target data (intradomain discrepancy). Consequently, most existing methods need to re-adapt the incoming data and retrain a new model on online data. This paradigm is difficult to meet the real-time requirements of online tasks. In this study, we propose an online UDA framework via jointly reducing interdomain and intradomain discrepancies on cross-domain object recognition where target data arrive in a streamed manner. Specifically, the proposed framework comprises two phases: classifier training and online recognition phases. In the former, we propose training a classifier on a shared subspace where there is a lower interdomain discrepancy between the two domains. In the latter, a low-rank subspace alignment method is introduced to adapt incoming data to the shared subspace by reducing the intradomain discrepancy. Finally, online recognition results can be obtained by the trained classifier. Extensive experiments on DA benchmarks and real-world datasets are employed to evaluate the performance of the proposed framework in online scenarios. The experimental results show the superiority of the proposed framework in online recognition tasks.

Visual-Depth Matching Network: Deep RGB-D Domain Adaptation With Unequal Categories

Existing domain adaptation (DA) methods generally assume that different domains have identical label space, and the training data are only sampled from a single domain. This unrealistic assumption is quite restricted for real-world applications, since it neglects the more practical scenario, where the source domain can contain the categories that are not shared by the target domain, and the training data can be collected from multiple modalities. In this article, we address a more difficult but practical problem, which recognizes RGB images through training on RGB-D data under the label space inequality scenario. There are three challenges in this task: 1) source and target domains are affected by the domain mismatch issue, which results in that the trained models perform imperfectly on the test data; 2) depth images are absent in the target domain (e.g., target images are captured by smartphones), when the source domain contains both the RGB and depth data. It makes the ordinary visual recognition approaches hardly applied to this task; and 3) in the real world, the source and target domains always have different numbers of categories, which would result in a negative transfer bottleneck being more prominent. Toward tackling the above challenges, we formulate a deep model, called visual-depth matching network (VDMN), where two new modules and a matching component can be trained in an end-to-end fashion jointly to identify the common and outlier categories effectively. The significance of VDMN is that it can take advantage of depth information and handle the domain distribution mismatch under label inequality simultaneously. The experimental results reveal that VDMN exceeds the state-of-the-art performance on various DA datasets, especially under the label inequality scenario.

Cross-Domain Remaining Useful Life Prediction Based on Adversarial Training

Remaining useful life prediction can assess the time to failure of degradation systems. Currently, numerous neural network-based prediction methods have been proposed by researchers. However, most of the work contains an implicit prerequisite: the network training and testing data have the same operating conditions. To solve this problem, an adversarial discriminative domain adaption prediction method based on adversarial training is proposed to improve the accuracy of cross-domain prediction under different working conditions. First, an LSTM feature extraction network is constructed to mine the source domain data and the target domain data for deep feature representation. Subsequently, the parameters of the target domain feature extraction network are adjusted based on the idea of adversarial training to achieve domain invariant feature mining. The proposed scheme is experimented on a publicly available dataset and achieves state-of-the-art prediction performance compared to recent unsupervised domain adaptation prediction methods.

Mental Workload Classification Method Based on EEG Cross-Session Subspace Alignment

Electroencephalogram (EEG) signals are sensitive to the level of Mental Workload (MW). However, the random non-stationarity of EEG signals will lead to low accuracy and a poor generalization ability for cross-session MW classification. To solve this problem of the different marginal distribution of EEG signals in different time periods, an MW classification method based on EEG Cross-Session Subspace Alignment (CSSA) is presented to identify the level of MW induced in visual manipulation tasks. The Independent Component Analysis (ICA) method is used to obtain the Independent Components (ICs) of labeled and unlabeled EEG signals. The energy features of ICs are extracted as source domains and target domains, respectively. The marginal distributions of source subspace base vectors are aligned with the target subspace base vectors based on the linear mapping. The Kullback–Leibler (KL) divergences between the two domains are calculated to select approximately similar transformed base vectors of source subspace. The energy features in all selected vectors are trained to build a new classifier using the Support Vector Machine (SVM). Then it can realize MW classification using the cross-session EEG signals, and has good classification accuracy.

Coupled Heterogeneous Tucker Decomposition: A Feature Extraction Method for Multisource Fusion and Domain Adaptation Using Multisource Heterogeneous Remote Sensing Data

To excavate adequately the rich information contained in multisource remote sensing data, feature extraction as basic yet important research has two typical applications: one of which is to extract complementary information of multisource data to improve classification; and the other is to extract shared information across sources for domain adaptation. However, typical feature extraction methods require the input represented as vectors or homogeneous tensors and fail to process multisource data represented as heterogeneous tensors. Therefore, the coupled heterogeneous Tucker decomposition (C-HTD) containing two sub-methods, namely coupled factor matrix-based HTD (CFM-HTD) and coupled core tensor-based HTD (CCT-HTD), is proposed to establish a unified feature extraction framework for multisource fusion and domain adaptation. To handle multisource heterogeneous tensors, multiple Tucker models were constructed to extract features of different sources separately. To cope with the supervised and semi-supervised cases, the class-indicator factor matrix was built to enhance the separability of features using known labels and learned labels. To mine the complementarity of paired multisource samples, coupling constraint was imposed on multiple factor matrices to form CFM-HTD to extract multisource information jointly. To extract domain-adapted features, coupling constraint was imposed on multiple core tensors to form CCT-HTD to encourage data from different sources to have the same class centroid. In addition, to reduce the impact of interference samples on domain adaptation, an adaptive sample-weighting matrix was designed to autonomously remove outliers. Using multiresolution multiangle optical and MSTAR datasets, experimental results show that the C-HTD outperforms typical multisource fusion and domain adaptation methods.

Not All Electrode Channels Are Needed: Knowledge Transfer From Only Stimulated Brain Regions for EEG Emotion Recognition

Emotion recognition from affective brain-computer interfaces (aBCI) has garnered a lot of attention in human-computer interactions. Electroencephalographic (EEG) signals collected and stored in one database have been mostly used due to their ability to detect brain activities in real time and their reliability. Nevertheless, large EEG individual differences occur amongst subjects making it impossible for models to share information across. New labeled data is collected and trained separately for new subjects which costs a lot of time. Also, during EEG data collection across databases, different stimulation is introduced to subjects. Audio-visual stimulation (AVS) is commonly used in studying the emotional responses of subjects. In this article, we propose a brain region aware domain adaptation (BRADA) algorithm to treat features from auditory and visual brain regions differently, which effectively tackle subject-to-subject variations and mitigate distribution mismatch across databases. BRADA is a new framework that works with the existing transfer learning method. We apply BRADA to both cross-subject and cross-database settings. The experimental results indicate that our proposed transfer learning method can improve valence-arousal emotion recognition tasks.

Motor Imagery Classification via Kernel-Based Domain Adaptation on an SPD Manifold

Background: Recording the calibration data of a brain–computer interface is a laborious process and is an unpleasant experience for the subjects. Domain adaptation is an effective technology to remedy the shortage of target data by leveraging rich labeled data from the sources. However, most prior methods have needed to extract the features of the EEG signal first, which triggers another challenge in BCI classification, due to small sample sets or a lack of labels for the target. Methods: In this paper, we propose a novel domain adaptation framework, referred to as kernel-based Riemannian manifold domain adaptation (KMDA). KMDA circumvents the tedious feature extraction process by analyzing the covariance matrices of electroencephalogram (EEG) signals. Covariance matrices define a symmetric positive definite space (SPD) that can be described by Riemannian metrics. In KMDA, the covariance matrices are aligned in the Riemannian manifold, and then are mapped to a high dimensional space by a log-Euclidean metric Gaussian kernel, where subspace learning is performed by minimizing the conditional distribution distance between the sources and the target while preserving the target discriminative information. We also present an approach to convert the EEG trials into 2D frames (E-frames) to further lower the dimension of covariance descriptors. Results: Experiments on three EEG datasets demonstrated that KMDA outperforms several state-of-the-art domain adaptation methods in classification accuracy, with an average Kappa of 0.56 for BCI competition IV dataset IIa, 0.75 for BCI competition IV dataset IIIa, and an average accuracy of 81.56% for BCI competition III dataset IVa. Additionally, the overall accuracy was further improved by 5.28% with the E-frames. KMDA showed potential in addressing subject dependence and shortening the calibration time of motor imagery-based brain–computer interfaces.

Domain Adaptation with Self-Guided Adaptive Sampling Strategy: Feature Alignment for Cross-User Myoelectric Pattern Recognition

Gestural interfaces based on surface electromyo-graphic (sEMG) signal have been widely explored. Nevertheless, due to the individual differences in the sEMG signals, it is very challenging for a myoelectric pattern recognition control system to adapt cross-user variability. Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness, and it is a promising approach to solve the cross-user challenge. Existing UDA methods largely ignore the instantaneous data distribution during model updating, thus deteriorating the feature representation given a large domain shift. To address this issue, a novel method is proposed based on a UDA model incorporated with a self-guided adaptive sampling (SGAS) strategy. This strategy is designed to utilize the domain distance in a kernel space as an indicator to screen out reliable instantaneous samples for updating the classifier. Thus, it enables improved alignment of feature representations of myoelectric patterns across users. To evaluate the performance of the proposed method, sEMG data were recorded from forearm muscles of nine subjects performing six finger and wrist gestures. Experiment results show that the UDA method with the SGAS strategy achieved a mean accuracy of 90.41% ± 14.44% in a cross-user classification manner, outperformed the state-of-the-art methods with statistical significance. This study demonstrates the effectiveness of the proposed UDA framework and offers a novel tool for implementing cross-user myoelectric pattern recognition towards a multi-user and user-independent control.

Transfer Discriminative Dictionary Pair Learning Approach for Across-Subject EEG Emotion Classification

Electroencephalogram (EEG) signals are not easily camouflaged, portable, and noninvasive. It is widely used in emotion recognition. However, due to the existence of individual differences, there will be certain differences in the data distribution of EEG signals in the same emotional state of different subjects. To obtain a model that performs well in classifying new subjects, traditional emotion recognition approaches need to collect a large number of labeled data of new subjects, which is often unrealistic. In this study, a transfer discriminative dictionary pair learning (TDDPL) approach is proposed for across-subject EEG emotion classification. The TDDPL approach projects data from different subjects into the domain-invariant subspace, and builds a transfer dictionary pair learning based on the maximum mean discrepancy (MMD) strategy. In the subspace, TDDPL learns shared synthesis and analysis dictionaries to build a bridge of discriminative knowledge from source domain (SD) to target domain (TD). By minimizing the reconstruction error and the inter-class separation term for each sub-dictionary, the learned synthesis dictionary is discriminative and the learned low-rank coding is sparse. Finally, a discriminative classifier in the TD is constructed on the classifier parameter, analysis dictionary and projection matrix, without the calculation of coding coefficients. The effectiveness of the TDDPL approach is verified on SEED and SEED IV datasets.