General Multiscenario Ultrasound Image Tumor Diagnosis Method Based on Unsupervised Domain Adaptation

OBJECTIVE

The utilization of computer-aided diagnosis (CAD) in breast ultrasound image classification has been limited by small sample sizes and domain shift. Current ultrasound classification methods perform inadequately when exposed to cross-domain scenarios, as they struggle with data sets from unobserved domains. In the medical field, there are situations in which all images must share the same networks as they capture the same symptom of the same participant, implying that they share identical structural content. Nevertheless, most domain adaptation methods are not suitable for medical images as they overlook the common features among the images.

METHODS

To overcome these challenges, we propose a novel diverse-domain 2-D feature selection network (FSN), which uses the similarities among medical images and extracts features with a reconstruction network with shared weights. Additionally, it penalizes the feature domain distance through two adversarial learning modules that align the feature space and select common features. Our experiments illustrate that the proposed method is robust and can be applied to ultrasound images of various diseases.

RESULTS

Compared with the latest domain adaptive methods, 2-D FSN markedly enhances the accuracy of classification of breast, thyroid and endoscopic ultrasound images, achieving accuracies of 82.4%, 96.4% and 89.7%, respectively. Furthermore, the model was evaluated on an unsupervised domain adaptation task using ultrasound images from multiple sources and achieved an average accuracy of 77.3% across widely varying domains.

CONCLUSION

In general, 2-D FSN improves the classification ability of the model on multidomain ultrasound data sets through the learning of common features and the combination of multimodule intelligence. The algorithm has good clinical guidance value.

Detecting Cheating in Large-Scale Assessment: The Transfer of Detectors to New Tests

Recent approaches to the detection of cheaters in tests employ detectors from the field of machine learning. Detectors based on supervised learning algorithms achieve high accuracy but require labeled data sets with identified cheaters for training. Labeled data sets are usually not available at an early stage of the assessment period. In this article, we discuss the approach of adapting a detector that was trained previously with a labeled training data set to a new unlabeled data set. The training and the new data set may contain data from different tests. The adaptation of detectors to new data or tasks is denominated as transfer learning in the field of machine learning. We first discuss the conditions under which a detector of cheating can be transferred. We then investigate whether the conditions are met in a real data set. We finally evaluate the benefits of transferring a detector of cheating. We find that a transferred detector has higher accuracy than an unsupervised detector of cheating. A naive transfer that consists of a simple reuse of the detector increases the accuracy considerably. A transfer via a self-labeling (SETRED) algorithm increases the accuracy slightly more than the naive transfer. The findings suggest that the detection of cheating might be improved by using existing detectors of cheating at an early stage of an assessment period.

Transfer Learning with Kernel Methods

Transfer learning refers to the process of adapting a model trained on a source task to a target task. While kernel methods are conceptually and computationally simple models that are competitive on a variety of tasks, it has been unclear how to develop scalable kernel-based transfer learning methods across general source and target tasks with possibly differing label dimensions. In this work, we propose a transfer learning framework for kernel methods by projecting and translating the source model to the target task. We demonstrate the effectiveness of our framework in applications to image classification and virtual drug screening. For both applications, we identify simple scaling laws that characterize the performance of transfer-learned kernels as a function of the number of target examples. We explain this phenomenon in a simplified linear setting, where we are able to derive the exact scaling laws.

Domain-adaptive neural networks improve supervised machine learning based on simulated population genetic data

Investigators have recently introduced powerful methods for population genetic inference that rely on supervised machine learning from simulated data. Despite their performance advantages, these methods can fail when the simulated training data does not adequately resemble data from the real world. Here, we show that this "simulation mis-specification" problem can be framed as a "domain adaptation" problem, where a model learned from one data distribution is applied to a dataset drawn from a different distribution. By applying an established domain-adaptation technique based on a gradient reversal layer (GRL), originally introduced for image classification, we show that the effects of simulation mis-specification can be substantially mitigated. We focus our analysis on two state-of-the-art deep-learning population genetic methods-SIA, which infers positive selection from features of the ancestral recombination graph (ARG), and ReLERNN, which infers recombination rates from genotype matrices. In the case of SIA, the domain adaptive framework also compensates for ARG inference error. Using the domain-adaptive SIA (dadaSIA) model, we estimate improved selection coefficients at selected loci in the 1000 Genomes CEU population. We anticipate that domain adaptation will prove to be widely applicable in the growing use of supervised machine learning in population genetics.

Open Set Domain Adaptation via Joint Alignment and Category Separation

Prevalent domain adaptation approaches are suitable for a close-set scenario where the source domain and the target domain are assumed to share the same data categories. However, this assumption is often violated in real-world conditions where the target domain usually contains samples of categories that are not presented in the source domain. This setting is termed as open set domain adaptation (OSDA). Most existing domain adaptation approaches do not work well in this situation. In this article, we propose an effective method, named joint alignment and category separation (JACS), for OSDA. Specifically, JACS learns a latent shared space, where the marginal and conditional divergence of feature distributions for commonly known classes across domains is alleviated (Joint Alignment), the distribution discrepancy between the known classes and the unknown class is enlarged, and the distance between different known classes is also maximized (Category Separation). These two aspects are unified into an objective to reinforce the optimization of each part simultaneously. The classifier is achieved based on the learned new feature representations by minimizing the structural risk in the reproducing kernel Hilbert space. Extensive experiment results verify that our method outperforms other state-of-the-art approaches on several benchmark datasets.

A deep domain adaptation framework with correlation alignment for EEG-based motor imagery classification

It is impractical to collect sufficient and well-labeled EEG data in Brain-computer interface because of the time-consuming data acquisition and costly annotation. Conventional classification methods reusing EEG data from different subjects and time periods (across domains) significantly decrease the classification accuracy of motor imagery. In this paper, we propose a deep domain adaptation framework with correlation alignment (DDAF-CORAL) to solve the problem of distribution divergence for motor imagery classification across domains. Specifically, a two-stage framework is adopted to extract deep features for raw EEG data. The distribution divergence caused by subjected-related and time-related variations is further minimized by aligning the covariance of the source and target EEG feature distributions. Finally, the classification loss and adaptation loss are optimized simultaneously to achieve sufficient discriminative classification performance and low feature distribution divergence. Extensive experiments on three EEG datasets demonstrate that our proposed method can effectively reduce the distribution divergence between the source and target EEG data. The results show that our proposed method delivers outperformance (an average classification accuracy of 92.9% for within-session, an average kappa value of 0.761 for cross-session, and an average classification accuracy of 83.3% for cross-subject) in two-class classification tasks compared to other state-of-the-art methods.

From single to multiple: Generalized detection of Covid-19 under limited classes samples

Amid the unfolding Covid-19 pandemic, there is a critical need for rapid and accurate diagnostic methods. In this context, the field of deep learning-based medical image diagnosis has witnessed a swift evolution. However, the prevailing methodologies often rely on large amounts of labeled data and require comprehensive medical knowledge. Both of these prerequisites pose significant challenges in real clinical settings, given the high cost of data labeling and the complexities of disease representations. Addressing this gap, we propose a novel problem setting, the Open-Set Single-Domain Generalization for Medical Image Diagnosis (OSSDG-MID). In OSSDG-MID, our aim is to train a model exclusively on a single source domain, so it can classify samples from the target domain accurately, designating them as 'unknown' if they don't belong to the source domain sample category space. Our innovative solution, the Multiple Cross-Matching method (MCM), enhances the identification of these 'unknown' categories by generating auxiliary samples that fall outside the category space of the source domain. Experimental evaluations on two diverse cross-domain image classification tasks demonstrate that our approach outperforms existing methodologies in both single-domain generalization and open-set image classification.

Using adversarial networks to extend brain computer interface decoding accuracy over time

Existing intracortical brain computer interfaces (iBCIs) transform neural activity into control signals capable of restoring movement to persons with paralysis. However, the accuracy of the 'decoder' at the heart of the iBCI typically degrades over time due to turnover of recorded neurons. To compensate, decoders can be recalibrated, but this requires the user to spend extra time and effort to provide the necessary data, then learn the new dynamics. As the recorded neurons change, one can think of the underlying movement intent signal being expressed in changing coordinates. If a mapping can be computed between the different coordinate systems, it may be possible to stabilize the original decoder's mapping from brain to behavior without recalibration. We previously proposed a method based on Generalized Adversarial Networks (GANs), called 'Adversarial Domain Adaptation Network' (ADAN), which aligns the distributions of latent signals within underlying low-dimensional neural manifolds. However, we tested ADAN on only a very limited dataset. Here we propose a method based on Cycle-Consistent Adversarial Networks (Cycle-GAN), which aligns the distributions of the full-dimensional neural recordings. We tested both Cycle-GAN and ADAN on data from multiple monkeys and behaviors and compared them to a third, quite different method based on Procrustes alignment of axes provided by Factor Analysis. All three methods are unsupervised and require little data, making them practical in real life. Overall, Cycle-GAN had the best performance and was easier to train and more robust than ADAN, making it ideal for stabilizing iBCI systems over time.

Privacy-preserving continual learning methods for medical image classification: a comparative analysis

Background

The implementation of deep learning models for medical image classification poses significant challenges, including gradual performance degradation and limited adaptability to new diseases. However, frequent retraining of models is unfeasible and raises concerns about healthcare privacy due to the retention of prior patient data. To address these issues, this study investigated privacy-preserving continual learning methods as an alternative solution.

Methods

We evaluated twelve privacy-preserving non-storage continual learning algorithms based deep learning models for classifying retinal diseases from public optical coherence tomography (OCT) images, in a class-incremental learning scenario. The OCT dataset comprises 108,309 OCT images. Its classes include normal (47.21%), drusen (7.96%), choroidal neovascularization (CNV) (34.35%), and diabetic macular edema (DME) (10.48%). Each class consisted of 250 testing images. For continuous training, the first task involved CNV and normal classes, the second task focused on DME class, and the third task included drusen class. All selected algorithms were further experimented with different training sequence combinations. The final model's average class accuracy was measured. The performance of the joint model obtained through retraining and the original finetune model without continual learning algorithms were compared. Additionally, a publicly available medical dataset for colon cancer detection based on histology slides was selected as a proof of concept, while the CIFAR10 dataset was included as the continual learning benchmark.

Results

Among the continual learning algorithms, Brain-inspired-replay (BIR) outperformed the others in the continual learning-based classification of retinal diseases from OCT images, achieving an accuracy of 62.00% (95% confidence interval: 59.36-64.64%), with consistent top performance observed in different training sequences. For colon cancer histology classification, Efficient Feature Transformations (EFT) attained the highest accuracy of 66.82% (95% confidence interval: 64.23-69.42%). In comparison, the joint model achieved accuracies of 90.76% and 89.28%, respectively. The finetune model demonstrated catastrophic forgetting in both datasets.

Conclusion

Although the joint retraining model exhibited superior performance, continual learning holds promise in mitigating catastrophic forgetting and facilitating continual model updates while preserving privacy in healthcare deep learning models. Thus, it presents a highly promising solution for the long-term clinical deployment of such models.

Omics data integration in computational biology viewed through the prism of machine learning paradigms

Important quantities of biological data can today be acquired to characterize cell types and states, from various sources and using a wide diversity of methods, providing scientists with more and more information to answer challenging biological questions. Unfortunately, working with this amount of data comes at the price of ever-increasing data complexity. This is caused by the multiplication of data types and batch effects, which hinders the joint usage of all available data within common analyses. Data integration describes a set of tasks geared towards embedding several datasets of different origins or modalities into a joint representation that can then be used to carry out downstream analyses. In the last decade, dozens of methods have been proposed to tackle the different facets of the data integration problem, relying on various paradigms. This review introduces the most common data types encountered in computational biology and provides systematic definitions of the data integration problems. We then present how machine learning innovations were leveraged to build effective data integration algorithms, that are widely used today by computational biologists. We discuss the current state of data integration and important pitfalls to consider when working with data integration tools. We eventually detail a set of challenges the field will have to overcome in the coming years.

Domain Adaptation Based on Semi-Supervised Cross-Domain Mean Discriminative Analysis and Kernel Transfer Extreme Learning Machine

Good data feature representation and high precision classifiers are the key steps for pattern recognition. However, when the data distributions between testing samples and training samples do not match, the traditional feature extraction methods and classification models usually degrade. In this paper, we propose a domain adaptation approach to handle this problem. In our method, we first introduce cross-domain mean approximation (CDMA) into semi-supervised discriminative analysis (SDA) and design semi-supervised cross-domain mean discriminative analysis (SCDMDA) to extract shared features across domains. Secondly, a kernel extreme learning machine (KELM) is applied as a subsequent classifier for the classification task. Moreover, we design a cross-domain mean constraint term on the source domain into KELM and construct a kernel transfer extreme learning machine (KTELM) to further promote knowledge transfer. Finally, the experimental results from four real-world cross-domain visual datasets prove that the proposed method is more competitive than many other state-of-the-art methods.

Transforming Complex Problems Into K-Means Solutions

K-means is a fundamental clustering algorithm widely used in both academic and industrial applications. Its popularity can be attributed to its simplicity and efficiency. Studies show the equivalence of K-means to principal component analysis, non-negative matrix factorization, and spectral clustering. However, these studies focus on standard K-means with squared euclidean distance. In this review paper, we unify the available approaches in generalizing K-means to solve challenging and complex problems. We show that these generalizations can be seen from four aspects: data representation, distance measure, label assignment, and centroid updating. As concrete applications of transforming problems into modified K-means formulation, we review the following applications: iterative subspace projection and clustering, consensus clustering, constrained clustering, domain adaptation, and outlier detection.

Domain Adaptation Multitask Optimization

Multitask optimization (MTO) is a new optimization paradigm that leverages useful information contained in multiple tasks to help solve each other. It attracts increasing attention in recent years and gains significant performance improvements. However, the solutions of distinct tasks usually obey different distributions. To avoid that individuals after intertask learning are not suitable for the original task due to the distribution differences and even impede overall solution efficiency, we propose a novel multitask evolutionary framework that enables knowledge aggregation and online learning among distinct tasks to solve MTO problems. Our proposal designs a domain adaptation-based mapping strategy to reduce the difference across solution domains and find more genetic traits to improve the effectiveness of information interactions. To further improve the algorithm performance, we propose a smart way to divide initial population into different subpopulations and choose suitable individuals to learn. By ranking individuals in target subpopulation, worse-performing individuals can learn from other tasks. The significant advantage of our proposed paradigm over the state of the art is verified via a series of MTO benchmark studies.

KL Divergence-based transfer learning for cross-subject eye movement recognition with EOG signals

Human-machine interfaces (HMIs) based on Electro-oculogram (EOG) signals have been widely explored. However, due to the individual variability, it is still challenging for an EOG-based eye movement recognition model to achieve favorable results among cross-subjects. The classical transfer learning methods such as CORrelation Alignment (CORAL), Transfer Component Analysis (TCA), and Joint Distribution Adaptation (JDA) are mainly based on feature transformation and distribution alignment, which do not consider similarities/dissimilarities between target subject and source subjects. In this paper, the Kullback-Leibler (KL) divergence of the log-Power Spectral Density (log-PSD) features of horizontal EOG (HEOG) between the target subject and each source subject is calculated for adaptively selecting partial subjects that suppose to have similar distribution with target subject for further training. It not only consider the similarity but also reduce computational consumption. The results show that the proposed approach is superior to the baseline and classical transfer learning methods, and significantly improves the performance of target subjects who have poor performance with the primary classifiers. The best improvement of Support Vector Machines (SVM) classifier has improved by 13.1% for subject 31 compared with baseline result. The preliminary results of this study demonstrate the effectiveness of the proposed transfer framework and provide a promising tool for implementing cross-subject eye movement recognition models in real-life scenarios.

Development and Optimization of a Novel Soft Sensor Modeling Method for Fermentation Process of Pichia pastoris

This paper introduces a novel soft sensor modeling method based on BDA-IPSO-LSSVM designed to address the issue of model failure caused by varying fermentation data distributions resulting from different operating conditions during the fermentation of different batches of Pichia pastoris. First, the problem of significant differences in data distribution among different batches of the fermentation process is addressed by adopting the balanced distribution adaptation (BDA) method from transfer learning. This method reduces the data distribution differences among batches of the fermentation process, while the fuzzy set concept is employed to improve the BDA method by transforming the classification problem into a regression prediction problem for the fermentation process. Second, the soft sensor model for the fermentation process is developed using the least squares support vector machine (LSSVM). The model parameters are optimized by an improved particle swarm optimization (IPSO) algorithm based on individual differences. Finally, the data obtained from the Pichia pastoris fermentation experiment are used for simulation, and the developed soft sensor model is applied to predict the cell concentration and product concentration during the fermentation process of Pichia pastoris. Simulation results demonstrate that the IPSO algorithm has good convergence performance and optimization performance compared with other algorithms. The improved BDA algorithm can make the soft sensor model adapt to different operating conditions, and the proposed soft sensor method outperforms existing methods, exhibiting higher prediction accuracy and the ability to accurately predict the fermentation process of Pichia pastoris under different operating conditions.

Harmonizing Healthy Cohorts to Support Multicenter Studies on Migraine Classification using Brain MRI Data

Multicenter and multi-scanner imaging studies might be needed to provide sample sizes large enough for developing accurate predictive models. However, multicenter studies, which likely include confounding factors due to subtle differences in research participant characteristics, MRI scanners, and imaging acquisition protocols, might not yield generalizable machine learning models, that is, models developed using one dataset may not be applicable to a different dataset. The generalizability of classification models is key for multi-scanner and multicenter studies, and for providing reproducible results. This study developed a data harmonization strategy to identify healthy controls with similar (homogenous) characteristics from multicenter studies to validate the generalization of machine-learning techniques for classifying individual migraine patients and healthy controls using brain MRI data. The Maximum Mean Discrepancy (MMD) was used to compare the two datasets represented in Geodesic Flow Kernel (GFK) space, capturing the data variabilities for identifying a "healthy core". A set of homogeneous healthy controls can assist in overcoming some of the unwanted heterogeneity and allow for the development of classification models that have high accuracy when applied to new datasets. Extensive experimental results show the utilization of a healthy core. One dataset consists of 120 individuals (66 with migraine and 54 healthy controls) and another dataset consists of 76 (34 with migraine and 42 healthy controls) individuals. A homogeneous dataset derived from a cohort of healthy controls improves the performance of classification models by about 25% accuracy improvements for both episodic and chronic migraineurs.

Subject-Independent EEG Emotion Recognition Based on Genetically Optimized Projection Dictionary Pair Learning

One of the primary challenges in Electroencephalogram (EEG) emotion recognition lies in developing models that can effectively generalize to new unseen subjects, considering the significant variability in EEG signals across individuals. To address the issue of subject-specific features, a suitable approach is to employ projection dictionary learning, which enables the identification of emotion-relevant features across different subjects. To accomplish the objective of pattern representation and discrimination for subject-independent EEG emotion recognition, we utilized the fast and efficient projection dictionary pair learning (PDPL) technique. PDPL involves the joint use of a synthesis dictionary and an analysis dictionary to enhance the representation of features. Additionally, to optimize the parameters of PDPL, which depend on experience, we applied the genetic algorithm (GA) to obtain the optimal solution for the model. We validated the effectiveness of our algorithm using leave-one-subject-out cross validation on three EEG emotion databases: SEED, MPED, and GAMEEMO. Our approach outperformed traditional machine learning methods, achieving an average accuracy of 69.89% on the SEED database, 24.11% on the MPED database, 64.34% for the two-class GAMEEMO, and 49.01% for the four-class GAMEEMO. These results highlight the potential of subject-independent EEG emotion recognition algorithms in the development of intelligent systems capable of recognizing and responding to human emotions in real-world scenarios.

Improve the performance of CT-based pneumonia classification via source data reweighting

Pneumonia is a life-threatening disease. Computer tomography (CT) imaging is broadly used for diagnosing pneumonia. To assist radiologists in accurately and efficiently detecting pneumonia from CT scans, many deep learning methods have been developed. These methods require large amounts of annotated CT scans, which are difficult to obtain due to privacy concerns and high annotation costs. To address this problem, we develop a three-level optimization based method which leverages CT data from a source domain to mitigate the lack of labeled CT scans in a target domain. Our method automatically identifies and downweights low-quality source CT data examples which are noisy or have large domain discrepancy with target data, by minimizing the validation loss of a target model trained on reweighted source data. On a target dataset with 2218 CT scans and a source dataset with 349 CT images, our method achieves an F1 score of 91.8% in detecting pneumonia and an F1 score of 92.4% in detecting other types of pneumonia, which are significantly better than those achieved by state-of-the-art baseline methods.

Adaptive Transfer Kernel Learning for Transfer Gaussian Process Regression

Transfer regression is a practical and challenging problem with important applications in various domains, such as engineering design and localization. Capturing the relatedness of different domains is the key of adaptive knowledge transfer. In this paper, we investigate an effective way of explicitly modelling domain relatedness through transfer kernel, a transfer-specified kernel that considers domain information in the covariance calculation. Specifically, we first give the formal definition of transfer kernel, and introduce three basic general forms that well cover existing related works. To cope with the limitations of the basic forms in handling complex real-world data, we further propose two advanced forms. Corresponding instantiations of the two forms are developed, namely Trk_αβ and Trk_ω based on multiple kernel learning and neural networks, respectively. For each instantiation, we present a condition with which the positive semi-definiteness is guaranteed and a semantic meaning is interpreted to the learned domain relatedness. Moreover, the condition can be easily used in the learning of TrGP _αβ and TrGP _ω that are the Gaussian process models with the transfer kernels Trk_αβ and Trk_ω respectively. Extensive empirical studies show the effectiveness of TrGP _αβ and TrGP _ω on domain relatedness modelling and transfer adaptiveness.

SSTM-IS: simplified STM method based on instance selection for real-time EEG emotion recognition

Introduction

EEG signals can non-invasively monitor the brain activities and have been widely used in brain-computer interfaces (BCI). One of the research areas is to recognize emotions objectively through EEG. In fact, the emotion of people changes over time, however, most of the existing affective BCIs process data and recognize emotions offline, and thus cannot be applied to real-time emotion recognition.

Methods

In order to solve this problem, we introduce the instance selection strategy into transfer learning and propose a simplified style transfer mapping algorithm. In the proposed method, the informative instances are firstly selected from the source domain data, and then the update strategy of hyperparameters is also simplified for style transfer mapping, making the model training more quickly and accurately for a new subject.

Results

To verify the effectiveness of our algorithm, we carry out the experiments on SEED, SEED-IV and the offline dataset collected by ourselves, and achieve the recognition accuracies up to 86.78%, 82.55% and 77.68% in computing time of 7s, 4s and 10s, respectively. Furthermore, we also develop a real-time emotion recognition system which integrates the modules of EEG signal acquisition, data processing, emotion recognition and result visualization.

Discussion

Both the results of offline and online experiments show that the proposed algorithm can accurately recognize emotions in a short time, meeting the needs of real-time emotion recognition applications.

A regression method for EEG-based cross-dataset fatigue detection

Introduction: Fatigue is dangerous for certain jobs requiring continuous concentration. When faced with new datasets, the existing fatigue detection model needs a large amount of electroencephalogram (EEG) data for training, which is resource-consuming and impractical. Although the cross-dataset fatigue detection model does not need to be retrained, no one has studied this problem previously. Therefore, this study will focus on the design of the cross-dataset fatigue detection model. Methods: This study proposes a regression method for EEG-based cross-dataset fatigue detection. This method is similar to self-supervised learning and can be divided into two steps: pre-training and the domain-specific adaptive step. To extract specific features for different datasets, a pretext task is proposed to distinguish data on different datasets in the pre-training step. Then, in the domain-specific adaptation stage, these specific features are projected into a shared subspace. Moreover, the maximum mean discrepancy (MMD) is exploited to continuously narrow the differences in the subspace so that an inherent connection can be built between datasets. In addition, the attention mechanism is introduced to extract continuous information on spatial features, and the gated recurrent unit (GRU) is used to capture time series information. Results: The accuracy and root mean square error (RMSE) achieved by the proposed method are 59.10% and 0.27, respectively, which significantly outperforms state-of-the-art domain adaptation methods. Discussion: In addition, this study discusses the effect of labeled samples. When the number of labeled samples is 10% of the total number, the accuracy of the proposed model can reach 66.21%. This study fills a vacancy in the field of fatigue detection. In addition, the EEG-based cross-dataset fatigue detection method can be used for reference by other EEG-based deep learning research practices.

Deep Learning-Based Anomaly Detection in Video Surveillance: A Survey

Anomaly detection in video surveillance is a highly developed subject that is attracting increased attention from the research community. There is great demand for intelligent systems with the capacity to automatically detect anomalous events in streaming videos. Due to this, a wide variety of approaches have been proposed to build an effective model that would ensure public security. There has been a variety of surveys of anomaly detection, such as of network anomaly detection, financial fraud detection, human behavioral analysis, and many more. Deep learning has been successfully applied to many aspects of computer vision. In particular, the strong growth of generative models means that these are the main techniques used in the proposed methods. This paper aims to provide a comprehensive review of the deep learning-based techniques used in the field of video anomaly detection. Specifically, deep learning-based approaches have been categorized into different methods by their objectives and learning metrics. Additionally, preprocessing and feature engineering techniques are discussed thoroughly for the vision-based domain. This paper also describes the benchmark databases used in training and detecting abnormal human behavior. Finally, the common challenges in video surveillance are discussed, to offer some possible solutions and directions for future research.

Unsupervised model adaptation for multivariate calibration by domain adaptation-regularization based kernel partial least square

In chemometrics, calibration model adaptation is desired when training- and test-samples come from different distributions. Domain-invariant feature representation is currently a successful strategy to realize model adaptation and has received wide attention. The paper presents a nonlinear unsupervised model adaptation method termed as domain adaption regularization-based kernel partial least squares regression (DarKPLS). DarKPLS aims to minimize the source and target distributions in a low-dimensional latent space projected from the reproducing kernel Hilbert space (RKHS) generated with the labeled source data and unlabeled target data. Specially, the distributional means and variances between source and target latent variables are aligned in the RKHS. By extending existing domain invariant partial least square regression (di-PLS) with the projected maximum mean discrepancy (PMMD) to reduce the mean discrepancy in the RKHS further, DarKPLS could realize fine-grained domain alignment that further improves the adaptation performance. DarKPLS is applied to the γ-polyglutamic acid fermentation dataset, tobacco dataset and corn dataset, and it demonstrates improved prediction results in comparison with No adaptation partial least squares (PLS), null augmented regression (NAR), extended linear joint trained framework (ExtJT), scatter component analysis (SCA) and domain-invariant iterative partial least squares (DIPALS).

Towards Accurate and Robust Domain Adaptation Under Multiple Noisy Environments

In many non-stationary environments, machine learning algorithms usually confront the distribution shift scenarios. Previous domain adaptation methods have achieved great success. However, they would lose algorithm robustness in multiple noisy environments where the examples of source domain become corrupted by label noise, feature noise, or open-set noise. In this paper, we report our attempt toward achieving noise-robust domain adaptation. We first give a theoretical analysis and find that different noises have disparate impacts on the expected target risk. To eliminate the effect of source noises, we propose offline curriculum learning minimizing a newly-defined empirical source risk. We suggest a proxy distribution-based margin discrepancy to gradually decrease the noisy distribution distance to reduce the impact of source noises. We propose an energy estimator for assessing the outlier degree of open-set-noise examples to defeat the harmful influence. We also suggest robust parameter learning to mitigate the negative effect further and learn domain-invariant feature representations. Finally, we seamlessly transform these components into an adversarial network that performs efficient joint optimization for them. A series of empirical studies on the benchmark datasets and the COVID-19 screening task show that our algorithm remarkably outperforms the state-of-the-art, with over 10% accuracy improvements in some transfer tasks.

Reconstruction of the cell pseudo-space from single-cell RNA sequencing data with scSpace

Tissues are highly complicated with spatial heterogeneity in gene expression. However, the cutting-edge single-cell RNA-seq technology eliminates the spatial information of individual cells, which contributes to the characterization of cell identities. Herein, we propose single-cell spatial position associated co-embeddings (scSpace), an integrative method to identify spatially variable cell subpopulations by reconstructing cells onto a pseudo-space with spatial transcriptome references (Visium, STARmap, Slide-seq, etc.). We benchmark scSpace with both simulated and biological datasets, and demonstrate that scSpace can accurately and robustly identify spatially variated cell subpopulations. When employed to reconstruct the spatial architectures of complex tissue such as the brain cortex, the small intestinal villus, the liver lobule, the kidney, the embryonic heart, and others, scSpace shows promising performance on revealing the pairwise cellular spatial association within single-cell data. The application of scSpace in melanoma and COVID-19 exhibits a broad prospect in the discovery of spatial therapeutic markers.

TIToK: A solution for bi-imbalanced unsupervised domain adaptation

Unsupervised domain adaptation (UDA) aims to transfer knowledge via domain alignment, and typically assumes balanced data distribution. When deployed in real tasks, however, (i) each domain usually suffers from class imbalance, and (ii) different domains may have different class imbalance ratios. In such bi-imbalanced cases with both within-domain and across-domain imbalance, source knowledge transfer may degenerate the target performance. Some recent efforts have adopted source re-weighting to this issue, in order to align label distributions across domains. However, since target label distribution is unknown, the alignment might be incorrect or even risky. In this paper, we propose an alternative solution named TIToK for bi-imbalanced UDA, by directly Transferring Imbalance-Tolerant Knowledge across domains. In TIToK, a class contrastive loss is presented for classification, in order to alleviate the sensitivity to imbalance in knowledge transfer. Meanwhile, knowledge of class correlation is transferred as a supplementary, which is commonly invariant to imbalance. Finally, discriminative feature alignment is developed for a more robust classifier boundary. Experiments over benchmark datasets show that TIToK achieves competitive performance with the state-of-the-arts, and its performance is less sensitive to imbalance.

Overcoming Challenges Associated with Developing Industrial Prognostics and Health Management Solutions

The development of prognostics and health management solutions in the manufacturing industry has lagged behind academic advances due to a number of practical challenges. This work proposes a framework for the initial development of industrial PHM solutions that is based on the system development life cycle commonly used for software-based applications. Methodologies for completing the planning and design stages, which are critical for industrial solutions, are presented. Two challenges that are inherent to health modeling in manufacturing environments, data quality and modeling systems that experience trend-based degradation, are then identified and methods to overcome them are proposed. Additionally included is a case study documenting the development of an industrial PHM solution for a hyper compressor at a manufacturing facility operated by The Dow Chemical Company. This case study demonstrates the value of the proposed development process and provides guidelines for utilizing it in other applications.

Cross-subject EEG emotion recognition using multi-source domain manifold feature selection

Recent researches on emotion recognition suggests that domain adaptation, a form of transfer learning, has the capability to solve the cross-subject problem in Affective brain-computer interface (aBCI) field. However, traditional domain adaptation methods perform single to single domain transfer or simply merge different source domains into a larger domain to realize the transfer of knowledge, resulting in negative transfer. In this study, a multi-source transfer learning framework was proposed to promote the performance of multi-source electroencephalogram (EEG) emotion recognition. The method first used the data distribution similarity ranking (DDSA) method to select the appropriate source domain for each target domain off-line, and reduced data drift between domains through manifold feature mapping on Grassmann manifold. Meanwhile, the minimum redundancy maximum correlation algorithm (mRMR) was employed to select more representative manifold features and minimized the conditional distribution and marginal distribution of the manifold features, and then learned the domain-invariant classifier by summarizing structural risk minimization (SRM). Finally, the weighted fusion criterion was applied to further improve recognition performance. We compared our method with several state-of-the-art domain adaptation techniques using the SEED and DEAP dataset. Results showed that, compared with the conventional MEDA algorithm, the recognition accuracy of our proposed algorithm on SEED and DEAP dataset were improved by 6.74% and 5.34%, respectively. Besides, compared with TCA, JDA, and other state-of-the-art algorithms, the performance of our proposed method was also improved with the best average accuracy of 86.59% on SEED and 64.40% on DEAP. Our results demonstrated that the proposed multi-source transfer learning framework is more effective and feasible than other state-of-the-art methods in recognizing different emotions by solving the cross-subject problem.

STGATE: Spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition

Electroencephalogram (EEG) is a crucial and widely utilized technique in neuroscience research. In this paper, we introduce a novel graph neural network called the spatial-temporal graph attention network with a transformer encoder (STGATE) to learn graph representations of emotion EEG signals and improve emotion recognition performance. In STGATE, a transformer-encoder is applied for capturing time-frequency features which are fed into a spatial-temporal graph attention for emotion classification. Using a dynamic adjacency matrix, the proposed STGATE adaptively learns intrinsic connections between different EEG channels. To evaluate the cross-subject emotion recognition performance, leave-one-subject-out experiments are carried out on three public emotion recognition datasets, i.e., SEED, SEED-IV, and DREAMER. The proposed STGATE model achieved a state-of-the-art EEG-based emotion recognition performance accuracy of 90.37% in SEED, 76.43% in SEED-IV, and 76.35% in DREAMER dataset, respectively. The experiments demonstrated the effectiveness of the proposed STGATE model for cross-subject EEG emotion recognition and its potential for graph-based neuroscience research.

Semantically preserving adversarial unsupervised domain adaptation network for improving disease recognition from chest x-rays

Supervised deep learning methods have been successfully applied in medical imaging. However, training deep learning systems often requires ample annotated data. Due to cost and time restrictions, not all collected medical images, e.g., chest x-rays (CXRs), can be labeled in practice. To classify these unlabeled images, a solution may involve adopting a model trained with sufficient labeled data in relevant domains (with both source and target being CXRs). However, domain shift may cause the trained model not able to generalize well on unlabeled target datasets. This work aims to develop a novel unsupervised domain adaptation (UDA) framework to improve recognition performance on unlabeled target data. We present a semantically preserving adversarial UDA network, i.e., SPA-UDA net, with the potential to bridge the domain gap, by reconstructing the images in the target domain via an adversarial encode-and-reconstruct translation architecture. To preserve the class-specific semantic information (i.e., with or without disease) of the original images when translating, a semantically consistent framework is embedded. This framework is designed to guarantee that fine-grained disease-related information on the original images can be safely transferred. Furthermore, the proposed SPA-UDA net does not require paired images from source and target domains when training, which reduces the cost of arranging data significantly and is ideal for UDA. We evaluate the proposed SPA-UDA net on two public CXR datasets for lung disease recognition. The experimental results show that the proposed framework achieves significant performance improvements compared to other state-of-the-art UDA methods.

Cross-subject aesthetic preference recognition of Chinese dance posture using EEG

Due to the differences in knowledge, experience, background, and social influence, people have subjective characteristics in the process of dance aesthetic cognition. To explore the neural mechanism of the human brain in the process of dance aesthetic preference, and to find a more objective determining criterion for dance aesthetic preference, this paper constructs a cross-subject aesthetic preference recognition model of Chinese dance posture. Specifically, Dai nationality dance (a classic Chinese folk dance) was used to design dance posture materials, and an experimental paradigm for aesthetic preference of Chinese dance posture was built. Then, 91 subjects were recruited for the experiment, and their EEG signals were collected. Finally, the transfer learning method and convolutional neural networks were used to identify the aesthetic preference of the EEG signals. Experimental results have shown the feasibility of the proposed model, and the objective aesthetic measurement in dance appreciation has been implemented. Based on the classification model, the accuracy of aesthetic preference recognition is 79.74%. Moreover, the recognition accuracies of different brain regions, different hemispheres, and different model parameters were also verified by the ablation study. Additionally, the experimental results reflected the following two facts: (1) in the visual aesthetic processing of Chinese dance posture, the occipital and frontal lobes are more activated and participate in dance aesthetic preference; (2) the right brain is more involved in the visual aesthetic processing of Chinese dance posture, which is consistent with the common knowledge that the right brain is responsible for processing artistic activities.

D-BIN: A Generalized Disentangling Batch Instance Normalization for Domain Adaptation

Pattern recognition is significantly challenging in real-world scenarios by the variability of visual statistics. Therefore, most existing algorithms relying on the independent identically distributed assumption of training and test data suffer from the poor generalization capability of inference on unseen testing datasets. Although numerous studies, including domain discriminator or domain-invariant feature learning, are proposed to alleviate this problem, the data-driven property and lack of interpretation of their principle throw researchers and developers off. Consequently, this dilemma incurs us to rethink the essence of networks' generalization. An observation that visual patterns cannot be discriminative after style transfer inspires us to take careful consideration of the importance of style features and content features. Does the style information related to the domain bias? How to effectively disentangle content and style features across domains? In this article, we first investigate the effect of feature normalization on domain adaptation. Based on it, we propose a novel normalization module to adaptively leverage the propagated information through each channel and batch of features called disentangling batch instance normalization (D-BIN). In this module, we explicitly explore domain-specific and domaininvariant feature disentanglement. We maneuver contrastive learning to encourage images with the same semantics from different domains to have similar content representations while having dissimilar style representations. Furthermore, we construct both self-form and dual-form regularizers for preserving the mutual information (MI) between feature representations of the normalization layer in order to compensate for the loss of discriminative information and effectively match the distributions across domains. D-BIN and the constrained term can be simply plugged into state-of-the-art (SOTA) networks to improve their performance. In the end, experiments, including domain adaptation and generalization, conducted on different datasets have proven their effectiveness.

BuresNet: Conditional Bures Metric for Transferable Representation Learning

As a fundamental manner for learning and cognition, transfer learning has attracted widespread attention in recent years. Typical transfer learning tasks include unsupervised domain adaptation (UDA) and few-shot learning (FSL), which both attempt to sufficiently transfer discriminative knowledge from the training environment to the test environment to improve the model's generalization performance. Previous transfer learning methods usually ignore the potential conditional distribution shift between environments. This leads to the discriminability degradation in the test environments. Therefore, how to construct a learnable and interpretable metric to measure and then reduce the gap between conditional distributions is very important in the literature. In this article, we design the Conditional Kernel Bures (CKB) metric for characterizing conditional distribution discrepancy, and derive an empirical estimation with convergence guarantee. CKB provides a statistical and interpretable approach, under the optimal transportation framework, to understand the knowledge transfer mechanism. It is essentially an extension of optimal transportation from the marginal distributions to the conditional distributions. CKB can be used as a plug-and-play module and placed onto the loss layer in deep networks, thus, it plays the bottleneck role in representation learning. From this perspective, the new method with network architecture is abbreviated as BuresNet, and it can be used extract conditional invariant features for both UDA and FSL tasks. BuresNet can be trained in an end-to-end manner. Extensive experiment results on several benchmark datasets validate the effectiveness of BuresNet.

From Laboratory to Field: Unsupervised Domain Adaptation for Plant Disease Recognition in the Wild

Plant disease recognition is of vital importance to monitor plant development and predicting crop production. However, due to data degradation caused by different conditions of image acquisition, e.g., laboratory vs. field environment, machine learning-based recognition models generated within a specific dataset (source domain) tend to lose their validity when generalized to a novel dataset (target domain). To this end, domain adaptation methods can be leveraged for the recognition by learning invariant representations across domains. In this paper, we aim at addressing the issues of domain shift existing in plant disease recognition and propose a novel unsupervised domain adaptation method via uncertainty regularization, namely, Multi-Representation Subdomain Adaptation Network with Uncertainty Regularization for Cross-Species Plant Disease Classification (MSUN). Our simple but effective MSUN makes a breakthrough in plant disease recognition in the wild by using a large amount of unlabeled data and via nonadversarial training. Specifically, MSUN comprises multirepresentation, subdomain adaptation modules and auxiliary uncertainty regularization. The multirepresentation module enables MSUN to learn the overall structure of features and also focus on capturing more details by using the multiple representations of the source domain. This effectively alleviates the problem of large interdomain discrepancy. Subdomain adaptation is used to capture discriminative properties by addressing the issue of higher interclass similarity and lower intraclass variation. Finally, the auxiliary uncertainty regularization effectively suppresses the uncertainty problem due to domain transfer. MSUN was experimentally validated to achieve optimal results on the PlantDoc, Plant-Pathology, Corn-Leaf-Diseases, and Tomato-Leaf-Diseases datasets, with accuracies of 56.06%, 72.31%, 96.78%, and 50.58%, respectively, surpassing other state-of-the-art domain adaptation techniques considerably.

Unsupervised Domain Adaptation with Differentially Private Gradient Projection

Domain adaptation is a viable solution for deep learning with small data. However, domain adaptation models trained on data with sensitive information may be a violation of personal privacy. In this article, we proposed a solution for unsupervised domain adaptation, called DP-CUDA, which is based on differentially private gradient projection and contradistinguisher. Compared with the traditional domain adaptation process, DP-CUDA involves searching for domain-invariant features between the source domain and target domain first and then transferring knowledge. Specifically, the model is trained in the source domain by supervised learning from labeled data. During the training of the target model, feature learning is used to solve the classification task in an end-to-end manner using unlabeled data directly, and the differentially private noise is injected into the gradient. We conducted extensive experiments on a variety of benchmark datasets, including MNIST, USPS, SVHN, VisDA-2017, Office-31, and Amazon Review, to demonstrate our proposed method’s utility and privacy-preserving properties.

A transfer predictive control method based on inter-domain mapping learning with application to industrial roasting process

As a critical variable in the roasting process, the roasting temperature has a significant influence on operating conditions. Model predictive control (MPC) provides a path to stabilize the roasting temperature. However, process data collected at different periods usually follow different distributions due to the fluctuation of feed composition for the roasting process, result in a model mismatch on online control. For this reason, a transfer predictive control method based on inter-domain mapping learning (IDML-MPC) is proposed. The proposed method first treat historical and online data as two domains. Then, a distribution mapping function from one domain to another domain is learned to make the distribution of the historical data follow that of the online data. Finally, an accurate online prediction model is built, roasting temperature control is achieved by minimizing the cost function with respect to the predicted value and the control input. The effectiveness of the proposed method is demonstrated by comparative experiments based on a numerical example and a simulation platform of the roasting process. Experimental results compared with some state-of-the-art methods show that it is necessary to take into account the distribution differences between historical data and online data when production conditions change. The IDML-MPC improved the control performance for the roasting temperature with an average 56.98% reduction in the root mean square error.

DomainATM: Domain adaptation toolbox for medical data analysis

Domain adaptation (DA) is an important technique for modern machine learning-based medical data analysis, which aims at reducing distribution differences between different medical datasets. A proper domain adaptation method can significantly enhance the statistical power by pooling data acquired from multiple sites/centers. To this end, we have developed the Domain Adaptation Toolbox for Medical data analysis (DomainATM) - an open-source software package designed for fast facilitation and easy customization of domain adaptation methods for medical data analysis. The DomainATM is implemented in MATLAB with a user-friendly graphical interface, and it consists of a collection of popular data adaptation algorithms that have been extensively applied to medical image analysis and computer vision. With DomainATM, researchers are able to facilitate fast feature-level and image-level adaptation, visualization and performance evaluation of different adaptation methods for medical data analysis. More importantly, the DomainATM enables the users to develop and test their own adaptation methods through scripting, greatly enhancing its utility and extensibility. An overview characteristic and usage of DomainATM is presented and illustrated with three example experiments, demonstrating its effectiveness, simplicity, and flexibility. The software, source code, and manual are available online.

Deep learning-based EEG emotion recognition: Current trends and future perspectives

Automatic electroencephalogram (EEG) emotion recognition is a challenging component of human-computer interaction (HCI). Inspired by the powerful feature learning ability of recently-emerged deep learning techniques, various advanced deep learning models have been employed increasingly to learn high-level feature representations for EEG emotion recognition. This paper aims to provide an up-to-date and comprehensive survey of EEG emotion recognition, especially for various deep learning techniques in this area. We provide the preliminaries and basic knowledge in the literature. We review EEG emotion recognition benchmark data sets briefly. We review deep learning techniques in details, including deep belief networks, convolutional neural networks, and recurrent neural networks. We describe the state-of-the-art applications of deep learning techniques for EEG emotion recognition in detail. We analyze the challenges and opportunities in this field and point out its future directions.

An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN

OBJECTIVES

Electroencephalogram (EEG) is often used to detect mental fatigue because of its real-time characteristic and objective nature. However, because of the individual variability of EEG among different individuals, tedious and time-consuming calibration sessions are needed.

METHODS

Therefore, we propose a multi-source domain adaptation network for inter-subject mental fatigue detection named FLDANN, which is short for focal loss based domain-adversarial training of neural network. As for mental state feature extraction, power spectrum density is extracted based on the Welch method from four sub-bands of EEG signals. The features of the source domain and target domain are fed into the FLDANN network. The contributions of FLDANN include: (1) It uses the idea of adversarial to reduce feature differences between the source and target domain. (2) A loss function named focal loss is used to assign weights to source and target domain samples.

RESULTS

The experiment result shows that when the number of the source domains increases, the classification accuracy of domain-adversarial training of neural network (DANN) gradually decreases and finally tends to be stable. The proposed method achieves an accuracy of 84.10% ± 8.75% on the SEED-VIG dataset and 65.42% ± 7.47% on the self-designed dataset. In addition, the proposed method is compared with other domain adaptation methods and the results show that the proposed method outperforms those state-of-the-art methods.

CONCLUSIONS

The result proves that the proposed method is able to solve the problem of individual differences across subjects and to solve the problem of low classification performance of multi-source domain transfer learning.

A Multi-Source Transfer Joint Matching Method for Inter-Subject Motor Imagery Decoding

Individual differences among different subjects pose a great challenge to motor imagery (MI) decoding. Multi-source transfer learning (MSTL) is one of the most promising ways to reduce individual differences, which can utilize rich information and align the data distribution among different subjects. However, most MSTL methods in MI-BCI combine all data in the source subjects into a single mixed domain, which will ignore the effect of important samples and the large differences in multiple source subjects. To address these issues, we introduce transfer joint matching and improve it to multi-source transfer joint matching (MSTJM) and weighted MSTJM (wMSTJM). Different from previous MSTL methods in MI, our methods align the data distribution for each pair of subjects, and then integrate the results by decision fusion. Besides that, we design an inter-subject MI decoding framework to verify the effectiveness of these two MSTL algorithms. It mainly consists of three modules: covariance matrix centroid alignment in the Riemannian space, source selection in the Euclidean space after tangent space mapping to reduce negative transfer and computation overhead, and further distribution alignment by MSTJM or wMSTJM. The superiority of this framework is verified on two common public MI datasets from BCI competition IV. The average classification accuracy of the MSTJM and wMSTJ methods outperformed other state-of-the-art methods by at least 4.24% and 2.62% respectively. It's promising to advance the practical applications of MI-BCI.