Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments

Evolution strategies (ESs), as a family of black-box optimization algorithms, recently emerge as a scalable alternative to reinforcement learning (RL) approaches such as Q-learning or policy gradient and are much faster when many central processing units (CPUs) are available due to better parallelization. In this article, we propose a systematic incremental learning method for ES in dynamic environments. The goal is to adjust previously learned policy to a new one incrementally whenever the environment changes. We incorporate an instance weighting mechanism with ES to facilitate its learning adaptation while retaining scalability of ES. During parameter updating, higher weights are assigned to instances that contain more new knowledge, thus encouraging the search distribution to move toward new promising areas of parameter space. We propose two easy-to-implement metrics to calculate the weights: instance novelty and instance quality. Instance novelty measures an instance's difference from the previous optimum in the original environment, while instance quality corresponds to how well an instance performs in the new environment. The resulting algorithm, instance weighted incremental evolution strategies (IW-IESs), is verified to achieve significantly improved performance on challenging RL tasks ranging from robot navigation to locomotion. This article thus introduces a family of scalable ES algorithms for RL domains that enables rapid learning adaptation to dynamic environments.

KNNENS: A k-Nearest Neighbor Ensemble-Based Method for Incremental Learning Under Data Stream With Emerging New Classes

In this brief, we investigate the problem of incremental learning under data stream with emerging new classes (SENC). In the literature, existing approaches encounter the following problems: 1) yielding high false positive for the new class; i) having long prediction time; and 3) having access to true labels for all instances, which is unrealistic and unacceptable in real-life streaming tasks. Therefore, we propose the k -Nearest Neighbor ENSemble-based method (KNNENS) to handle these problems. The KNNENS is effective to detect the new class and maintains high classification performance for known classes. It is also efficient in terms of run time and does not require true labels of new class instances for model update, which is desired in real-life streaming classification tasks. Experimental results show that the KNNENS achieves the best performance on four benchmark datasets and three real-world data streams in terms of accuracy and F1-measure and has a relatively fast run time compared to four reference methods. Codes are available at https://github.com/Ntriver/KNNENS.

Model Behavior Preserving for Class-Incremental Learning

Deep models have shown to be vulnerable to catastrophic forgetting, a phenomenon that the recognition performance on old data degrades when a pre-trained model is fine-tuned on new data. Knowledge distillation (KD) is a popular incremental approach to alleviate catastrophic forgetting. However, it usually fixes the absolute values of neural responses for isolated historical instances, without considering the intrinsic structure of the responses by a convolutional neural network (CNN) model. To overcome this limitation, we recognize the importance of the global property of the whole instance set and treat it as a behavior characteristic of a CNN model relevant to model incremental learning. On this basis: 1) we design an instance neighborhood-preserving (INP) loss to maintain the order of pair-wise instance similarities of the old model in the feature space; 2) we devise a label priority-preserving (LPP) loss to preserve the label ranking lists within instance-wise label probability vectors in the output space; and 3) we introduce an efficient derivable ranking algorithm for calculating the two loss functions. Extensive experiments conducted on CIFAR100 and ImageNet show that our approach achieves the state-of-the-art performance.

IWDA: Importance Weighting for Drift Adaptation in Streaming Supervised Learning Problems

Distribution drift is an important issue for practical applications of machine learning (ML). In particular, in streaming ML, the data distribution may change over time, yielding the problem of concept drift, which affects the performance of learners trained with outdated data. In this article, we focus on supervised problems in an online nonstationary setting, introducing a novel learner-agnostic algorithm for drift adaptation, namely importance weighting for drift adaptation (IWDA), with the goal of performing efficient retraining of the learner when drift is detected. IWDA incrementally estimates the joint probability density of input and target for the incoming data and, as soon as drift is detected, retrains the learner using importance-weighted empirical risk minimization. The importance weights are computed for all the samples observed so far, employing the estimated densities, thus, using all available information efficiently. After presenting our approach, we provide a theoretical analysis in the abrupt drift setting. Finally, we present numerical simulations that illustrate how IWDA competes and often outperforms state-of-the-art stream learning techniques, including adaptive ensemble methods, on both synthetic and real-world data benchmarks.

Adaptive Multigradient Recursive Reinforcement Learning Event-Triggered Tracking Control for Multiagent Systems

This article proposes a fault-tolerant adaptive multigradient recursive reinforcement learning (RL) event-triggered tracking control scheme for strict-feedback discrete-time multiagent systems. The multigradient recursive RL algorithm is used to avoid the local optimal problem that may exist in the gradient descent scheme. Different from the existing event-triggered control results, a new lemma about the relative threshold event-triggered control strategy is proposed to handle the compensation error, which can improve the utilization of communication resources and weaken the negative impact on tracking accuracy and closed-loop system stability. To overcome the difficulty caused by sensor fault, a distributed control method is introduced by adopting the adaptive compensation technique, which can effectively decrease the number of online estimation parameters. Furthermore, by using the multigradient recursive RL algorithm with less learning parameters, the online estimation time can be effectively reduced. The stability of closed-loop multiagent systems is proved by using the Lyapunov stability theorem, and it is verified that all signals are semiglobally uniformly ultimately bounded. Finally, two simulation examples are given to show the availability of the presented control scheme.

Command Filter-Based Adaptive Neural Control Design for Nonstrict-Feedback Nonlinear Systems With Multiple Actuator Constraints

This article proposes an adaptive neural-network command-filtered tracking control scheme of nonlinear systems with multiple actuator constraints. An equivalent transformation method is introduced to address the impediment from actuator nonlinearity. By utilizing the command filter method, the explosion of complexity problem is addressed. With the help of neural-network approximation, an adaptive neural-network tracking backstepping control strategy via the command filter technique and the backstepping design algorithm is proposed. Based on this scheme, the boundedness of all variables is guaranteed and the output tracking error fluctuates near the origin within a small bounded area. Simulations testify the availability of the designed control strategy.

Lifelong Incremental Reinforcement Learning With Online Bayesian Inference

A central capability of a long-lived reinforcement learning (RL) agent is to incrementally adapt its behavior as its environment changes and to incrementally build upon previous experiences to facilitate future learning in real-world scenarios. In this article, we propose lifelong incremental reinforcement learning (LLIRL), a new incremental algorithm for efficient lifelong adaptation to dynamic environments. We develop and maintain a library that contains an infinite mixture of parameterized environment models, which is equivalent to clustering environment parameters in a latent space. The prior distribution over the mixture is formulated as a Chinese restaurant process (CRP), which incrementally instantiates new environment models without any external information to signal environmental changes in advance. During lifelong learning, we employ the expectation-maximization (EM) algorithm with online Bayesian inference to update the mixture in a fully incremental manner. In EM, the E-step involves estimating the posterior expectation of environment-to-cluster assignments, whereas the M-step updates the environment parameters for future learning. This method allows for all environment models to be adapted as necessary, with new models instantiated for environmental changes and old models retrieved when previously seen environments are encountered again. Simulation experiments demonstrate that LLIRL outperforms relevant existing methods and enables effective incremental adaptation to various dynamic environments for lifelong learning.

Open-environment machine learning

Conventional machine learning studies generally assume close-environment scenarios where important factors of the learning process hold invariant. With the great success of machine learning, nowadays, more and more practical tasks, particularly those involving open-environment scenarios where important factors are subject to change, called open-environment machine learning in this article, are present to the community. Evidently, it is a grand challenge for machine learning turning from close environment to open environment. It becomes even more challenging since, in various big data tasks, data are usually accumulated with time, like streams, while it is hard to train the machine learning model after collecting all data as in conventional studies. This article briefly introduces some advances in this line of research, focusing on techniques concerning emerging new classes, decremental/incremental features, changing data distributions and varied learning objectives, and discusses some theoretical issues.

Optimal Feature Selection with Weight Optimised Deep Neural Network for Incremental Learning-Based Intrusion Detection in Fog Environment

Fog computing acts as an intermediate component to reduce the delays in communication among end-users and the cloud that offer local processing of requests among end-users through fog devices. Thus, the primary aim of fog devices is to ensure the authenticity of incoming network traffic. Anyhow, these fog devices are susceptible to malicious attacks. An efficient Intrusion Detection System (IDS) or Intrusion Prevention System (IPS) is necessary to offer secure functioning of fog for improving efficiency. IDSs are a fundamental component for any security system like the Internet of things (IoT) and fog networks for ensuring the Quality of Service (QoS). Even though different machine learning and deep learning models have shown their efficiency in intrusion detection, the deep insight of managing the incremental data is a complex part. Therefore, the main intent of this paper is to implement an effective model for intrusion detection in a fog computing platform. Initially, the data dealing with intrusion are collected from diverse benchmark sources. Further, data cleaning is performed, which is to identify and remove errors and duplicate data, to create a reliable dataset. This improves the quality of the training data for analytics and enables accurate decision making. The conceptual and temporal features are extracted. Concerning reducing the data length for reducing the training complexity, optimal feature selection is performed based on an improved meta-heuristic concept termed Modified Active Electrolocation-based Electric Fish Optimization (MAE-EFO). With the optimally selected features or data, incremental learning-based detection is accomplished by Incremental Deep Neural Network (I-DNN). This deep learning model optimises the testing weight using the proposed MAE-EFO by concerning the objective as to minimise the error difference between the predicted and actual results, thus enhancing the performance of new incremental data. The validation of the proposed model on the benchmark datasets and other datasets achieves an attractive performance when compared over other state-of-the-art IDSs.

Temporal and spatial patterns in the detection of veterinary drug residues in poultry and swine in Brazil

Abstract Food Safety is an important topic for public health and international trade in food. Residues of veterinary drugs and environmental contaminants in animal products can cause diseases and acute toxicity in organisms exposed to these substances. This study evaluated official monitoring data of veterinary drug residues from the Brazilian Ministry of Agriculture, Livestock and Supply in tissues of poultry and swine in the period between 2002 and 2014 to check for hidden patterns in the occurrence of six common drugs (Closantel, Diclazuril, Nicarbazin, Sulfaquinoxaline, Doxycycline and Sulfamethazinein). The analysis of data was performed by using two machine learning methods: decision tree and neural networks, in addition to visual evaluation through graphs and maps. Contamination rates were low, varying from 0 to 0.66%. A spatial distribution pattern of detections of substances by region was identified, but no pattern of temporal distribution was observed. Nevertless, regressions showed an increase in levels when these substances were detected, so monitoring should continue. However, the results show that the products monitored during the study period presented a low risk to public health.

Padrões temporais e espaciais na detecção de resíduos de medicamentos veterinários em aves e suínos no Brasil

Resumo A Segurança Alimentar é um tema importante para a saúde pública e o comércio internacional de alimentos. Resíduos de medicamentos veterinários e contaminantes ambientais em produtos de origem animal podem causar doenças e toxicidade aguda em organismos expostos a essas substâncias. Este estudo avaliou dados oficiais de monitoramento de resíduos de medicamentos veterinários do Ministério da Agricultura, Pecuária e Abastecimento em tecidos de aves e suínos no período de 2002 a 2014 para verificar padrões ocultos na ocorrência de seis medicamentos comuns (Closantel, Diclazuril, Nicarbazina, Sulfaquinoxalina, Doxiciclina e Sulfametazina). A análise dos dados foi realizada por meio de dois métodos de aprendizado de máquina: árvore de decisão e redes neurais, além da avaliação visual por meio de gráficos e mapas. As taxas de contaminação foram baixas, variando de 0 a 0,66%. Foi identificado um padrão de distribuição espacial das detecções de substâncias por região, mas nenhum padrão de distribuição temporal foi observado. No entanto, as regressões mostraram um aumento nos níveis quando essas substâncias foram detectadas, portanto, o monitoramento deve continuar. No entanto, os resultados mostram que os produtos monitorados durante o período do estudo apresentaram baixo risco à saúde pública.

Adaptive Resilient Event-Triggered Control Design of Autonomous Vehicles With an Iterative Single Critic Learning Framework

This article investigates the adaptive resilient event-triggered control for rear-wheel-drive autonomous (RWDA) vehicles based on an iterative single critic learning framework, which can effectively balance the frequency/changes in adjusting the vehicle's control during the running process. According to the kinematic equation of RWDA vehicles and the desired trajectory, the tracking error system during the autonomous driving process is first built, where the denial-of-service (DoS) attacking signals are injected into the networked communication and transmission. Combining the event-triggered sampling mechanism and iterative single critic learning framework, a new event-triggered condition is developed for the adaptive resilient control algorithm, and the novel utility function design is considered for driving the autonomous vehicle, where the control input can be guaranteed into an applicable saturated bound. Finally, we apply the new adaptive resilient control scheme to a case of driving the RWDA vehicles, and the simulation results illustrate the effectiveness and practicality successfully.

ADAW: Age decay accuracy weighted ensemble method for drifting data stream mining

Dynamic environment data generators are very often in real-world that produce data streams. A data source of a dynamic environment generates data streams in which the underlying data distribution changes very frequently with respect to time and hence results in concept drifts. As compared to the stationary environment, learning in the dynamic environment is very difficult due to the presence of concept drifts. Learning in dynamic environment requires evolutionary and adaptive approaches to be accommodated with the learning algorithms. Ensemble methods are commonly used to build classifiers for learning in a dynamic environment. The ensemble methods of learning are generally described at three very crucial aspects, namely, the learning and testing method employed, result integration method and forgetting mechanism for old concepts. In this paper, we propose a novel approach called Age Decay Accuracy Weighted (ADAW) ensemble architecture for learning in concept drifting data streams. The ADAW method assigned weights to the component classifiers based on its accuracy and its remaining life-time in the ensemble is such a way that ensures maximum accuracy. We empirically evaluated ADAW on benchmark artificial drifting data stream generators and real datasets and compared its performance with ten well-known state-of-the-art existing methods. The experimental results show that ADAW outperforms over the existing methods.

Incremental multi-view spectral clustering with sparse and connected graph learning

In recent years, a lot of excellent multi-view clustering methods have been proposed. Because most of them need to fuse all views at one time, they are infeasible as the number of views increases over time. If the present multi-view clustering methods are employed directly to re-fuse all views at each time, it is too expensive to store all historical views. In this paper, we proposed an efficient incremental multi-view spectral clustering method with sparse and connected graph learning (SCGL). In our method, only one consensus similarity matrix is stored to represent the structural information of all historical views. Once the newly collected view is available, the consensus similarity matrix is reconstructed by learning from its previous version and the current new view. To further improve the incremental multi-view clustering performance, the sparse graph learning and the connected graph learning are integrated into our model, which can not only reduce the noises, but also preserve the correct connections within clusters. Experiments on several multi-view datasets demonstrate that our method is superior to traditional methods in clustering accuracy, and is more suitable to deal with the multi-view clustering with the number of views increasing over time.

IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection

Elastic weight consolidation (EWC) has been successfully applied for general incremental learning to overcome the catastrophic forgetting issue. It adaptively constrains each parameter of the new model not to deviate much from its counterpart in the old model during fine-tuning on new class data sets, according to its importance weight for old tasks. However, the previous study demonstrates that it still suffers from catastrophic forgetting when directly used in object detection. In this article, we show EWC is effective for incremental object detection if with critical adaptations. First, we conduct controlled experiments to identify two core issues why EWC fails if trivially applied to incremental detection: 1) the absence of old class annotations in new class images makes EWC misclassify objects of old classes in these images as background and 2) the quadratic regularization loss in EWC easily leads to gradient explosion when balancing old and new classes. Then, based on the abovementioned findings, we propose the corresponding solutions to tackle these issues: 1) utilize pseudobounding box annotations of old classes on new data sets to compensate for the absence of old class annotations and 2) adopt a novel Huber regularization instead of the original quadratic loss to prevent from unstable training. Finally, we propose a general EWC-based incremental object detection framework and implement it under both Fast R-CNN and Faster R-CNN, showing its flexibility and versatility. In terms of either the final performance or the performance drop with respect to the upper bound of joint training on all seen classes, evaluations on the PASCAL VOC and COCO data sets show that our method achieves a new state of the art.

Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception

The material attribute of an object's surface is critical to enable robots to perform dexterous manipulations or actively interact with their surrounding objects. Tactile sensing has shown great advantages in capturing material properties of an object's surface. However, the conventional classification method based on tactile information may not be suitable to estimate or infer material properties, particularly during interacting with unfamiliar objects in unstructured environments. Moreover, it is difficult to intuitively obtain material properties from tactile data as the tactile signals about material properties are typically dynamic time sequences. In this article, a visual-tactile cross-modal learning framework is proposed for robotic material perception. In particular, we address visual-tactile cross-modal learning in the lifelong learning setting, which is beneficial to incrementally improve the ability of robotic cross-modal material perception. To this end, we proposed a novel lifelong cross-modal learning model. Experimental results on the three publicly available data sets demonstrate the effectiveness of the proposed method.

Data-driven adaptive modeling method for industrial processes and its application in flotation reagent control

In real industrial processes, new process "excitation" patterns that largely deviate from previously collected training data will appear due to disturbances caused by process inputs. To reduce model mismatch, it is important for a data-driven process model to adapt to new process "excitation" patterns. Although efforts have been devoted to developing adaptive process models to deal with this problem, few studies have attempted to develop an adaptive process model that can incrementally learn new process "excitation" patterns without performance degradation on old patterns. In this study, efforts are devoted to enabling data-driven process models with incremental learning ability. First, a novel incremental learning method is proposed for process model updating. Second, an adaptive neural network process model is developed based on the novel incremental learning method. Third, a nonlinear model predictive control based on the adaptive process model is implemented and applied for flotation reagent control. Experiments based on historical data provide evidence that the newly developed adaptive process model can accommodate new process "excitation" patterns and preserve its performance on old patterns. Furthermore, industry experiments carried out in a real-world lead-zinc froth flotation plant provide industrial evidence and show that the newly designed controller is promising for practical flotation reagent control.

Precision health data: Requirements, challenges and existing techniques for data security and privacy

Precision health leverages information from various sources, including omics, lifestyle, environment, social media, medical records, and medical insurance claims to enable personalized care, prevent and predict illness, and precise treatments. It extensively uses sensing technologies (e.g., electronic health monitoring devices), computations (e.g., machine learning), and communication (e.g., interaction between the health data centers). As health data contain sensitive private information, including the identity of patient and carer and medical conditions of the patient, proper care is required at all times. Leakage of these private information affects the personal life, including bullying, high insurance premium, and loss of job due to the medical history. Thus, the security, privacy of and trust on the information are of utmost importance. Moreover, government legislation and ethics committees demand the security and privacy of healthcare data. Besides, the public, who is the data source, always expects the security, privacy, and trust of their data. Otherwise, they can avoid contributing their data to the precision health system. Consequently, as the public is the targeted beneficiary of the system, the effectiveness of precision health diminishes. Herein, in the light of precision health data security, privacy, ethical and regulatory requirements, finding the best methods and techniques for the utilization of the health data, and thus precision health is essential. In this regard, firstly, this paper explores the regulations, ethical guidelines around the world, and domain-specific needs. Then it presents the requirements and investigates the associated challenges. Secondly, this paper investigates secure and privacy-preserving machine learning methods suitable for the computation of precision health data along with their usage in relevant health projects. Finally, it illustrates the best available techniques for precision health data security and privacy with a conceptual system model that enables compliance, ethics clearance, consent management, medical innovations, and developments in the health domain.

Beyond Cross-Validation—Accuracy Estimation for Incremental and Active Learning Models

For incremental machine-learning applications it is often important to robustly estimate the system accuracy during training, especially if humans perform the supervised teaching. Cross-validation and interleaved test/train error are here the standard supervised approaches. We propose a novel semi-supervised accuracy estimation approach that clearly outperforms these two methods. We introduce the Configram Estimation (CGEM) approach to predict the accuracy of any classifier that delivers confidences. By calculating classification confidences for unseen samples, it is possible to train an offline regression model, capable of predicting the classifier’s accuracy on novel data in a semi-supervised fashion. We evaluate our method with several diverse classifiers and on analytical and real-world benchmark data sets for both incremental and active learning. The results show that our novel method improves accuracy estimation over standard methods and requires less supervised training data after deployment of the model. We demonstrate the application of our approach to a challenging robot object recognition task, where the human teacher can use our method to judge sufficient training.

Context-Aware Query Selection for Active Learning in Event Recognition

Activity recognition is a challenging problem with many practical applications. In addition to the visual features, recent approaches have benefited from the use of context, e.g., inter-relationships among the activities and objects. However, these approaches require data to be labeled, entirely available beforehand, and not designed to be updated continuously, which make them unsuitable for surveillance applications. In contrast, we propose a continuous-learning framework for context-aware activity recognition from unlabeled video, which has two distinct advantages over existing methods. First, it employs a novel active-learning technique that not only exploits the informativeness of the individual activities but also utilizes their contextual information during query selection; this leads to significant reduction in expensive manual annotation effort. Second, the learned models can be adapted online as more data is available. We formulate a conditional random field model that encodes the context and devise an information-theoretic approach that utilizes entropy and mutual information of the nodes to compute the set of most informative queries, which are labeled by a human. These labels are combined with graphical inference techniques for incremental updates. We provide a theoretical formulation of the active learning framework with an analytic solution. Experiments on six challenging datasets demonstrate that our framework achieves superior performance with significantly less manual labeling.

Adaptive Neural Output-Feedback Decentralized Control for Large-Scale Nonlinear Systems With Stochastic Disturbances

This paper addresses the problem of adaptive neural output-feedback decentralized control for a class of strongly interconnected nonlinear systems suffering stochastic disturbances. An state observer is designed to approximate the unmeasurable state signals. Using the approximation capability of radial basis function neural networks (NNs) and employing classic adaptive control strategy, an observer-based adaptive backstepping decentralized controller is developed. In the control design process, NNs are applied to model the uncertain nonlinear functions, and adaptive control and backstepping are combined to construct the controller. The developed control scheme can guarantee that all signals in the closed-loop systems are semiglobally uniformly ultimately bounded in fourth-moment. The simulation results demonstrate the effectiveness of the presented control scheme.

Incremental Learning for Classification of Unstructured Data Using Extreme Learning Machine

Unstructured data are irregular information with no predefined data model. Streaming data which constantly arrives over time is unstructured, and classifying these data is a tedious task as they lack class labels and get accumulated over time. As the data keeps growing, it becomes difficult to train and create a model from scratch each time. Incremental learning, a self-adaptive algorithm uses the previously learned model information, then learns and accommodates new information from the newly arrived data providing a new model, which avoids the retraining. The incrementally learned knowledge helps to classify the unstructured data. In this paper, we propose a framework CUIL (Classification of Unstructured data using Incremental Learning) which clusters the metadata, assigns a label for each cluster and then creates a model using Extreme Learning Machine (ELM), a feed-forward neural network, incrementally for each batch of data arrived. The proposed framework trains the batches separately, reducing the memory resources, training time significantly and is tested with metadata created for the standard image datasets like MNIST, STL-10, CIFAR-10, Caltech101, and Caltech256. Based on the tabulated results, our proposed work proves to show greater accuracy and efficiency.

Concept Drift Adaptation by Exploiting Historical Knowledge

Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods.

Adaptive Neural Output-Feedback Control for a Class of Nonlower Triangular Nonlinear Systems With Unmodeled Dynamics

This paper presents the development of an adaptive neural controller for a class of nonlinear systems with unmodeled dynamics and immeasurable states. An observer is designed to estimate system states. The structure consistency of virtual control signals and the variable partition technique are combined to overcome the difficulties appearing in a nonlower triangular form. An adaptive neural output-feedback controller is developed based on the backstepping technique and the universal approximation property of the radial basis function (RBF) neural networks. By using the Lyapunov stability analysis, the semiglobally and uniformly ultimate boundedness of all signals within the closed-loop system is guaranteed. The simulation results show that the controlled system converges quickly, and all the signals are bounded. This paper is novel at least in the two aspects: 1) an output-feedback control strategy is developed for a class of nonlower triangular nonlinear systems with unmodeled dynamics and 2) the nonlinear disturbances and their bounds are the functions of all states, which is in a more general form than existing results.

New Splitting Criteria for Decision Trees in Stationary Data Streams

The most popular tools for stream data mining are based on decision trees. In previous 15 years, all designed methods, headed by the very fast decision tree algorithm, relayed on Hoeffding's inequality and hundreds of researchers followed this scheme. Recently, we have demonstrated that although the Hoeffding decision trees are an effective tool for dealing with stream data, they are a purely heuristic procedure; for example, classical decision trees such as ID3 or CART cannot be adopted to data stream mining using Hoeffding's inequality. Therefore, there is an urgent need to develop new algorithms, which are both mathematically justified and characterized by good performance. In this paper, we address this problem by developing a family of new splitting criteria for classification in stationary data streams and investigating their probabilistic properties. The new criteria, derived using appropriate statistical tools, are based on the misclassification error and the Gini index impurity measures. The general division of splitting criteria into two types is proposed. Attributes chosen based on type- splitting criteria guarantee, with high probability, the highest expected value of split measure. Type- criteria ensure that the chosen attribute is the same, with high probability, as it would be chosen based on the whole infinite data stream. Moreover, in this paper, two hybrid splitting criteria are proposed, which are the combinations of single criteria based on the misclassification error and Gini index.

Airline Passenger Profiling Based on Fuzzy Deep Machine Learning

Passenger profiling plays a vital part of commercial aviation security, but classical methods become very inefficient in handling the rapidly increasing amounts of electronic records. This paper proposes a deep learning approach to passenger profiling. The center of our approach is a Pythagorean fuzzy deep Boltzmann machine (PFDBM), whose parameters are expressed by Pythagorean fuzzy numbers such that each neuron can learn how a feature affects the production of the correct output from both the positive and negative sides. We propose a hybrid algorithm combining a gradient-based method and an evolutionary algorithm for training the PFDBM. Based on the novel learning model, we develop a deep neural network (DNN) for classifying normal passengers and potential attackers, and further develop an integrated DNN for identifying group attackers whose individual features are insufficient to reveal the abnormality. Experiments on data sets from Air China show that our approach provides much higher learning ability and classification accuracy than existing profilers. It is expected that the fuzzy deep learning approach can be adapted for a variety of complex pattern analysis tasks.

Enhancing the Prediction of Transmembrane β-Barrel Segments with Chain Learning and Feature Sparse Representation

Transmembrane β-barrels (TMBs) are one important class of membrane proteins that play crucial functions in the cell. Membrane proteins are difficult wet-lab targets of structural biology, which call for accurate computational prediction approaches. Here, we developed a novel method named MemBrain-TMB to predict the spanning segments of transmembrane β-barrel from amino acid sequence. MemBrain-TMB is a statistical machine learning-based model, which is constructed using a new chain learning algorithm with input features encoded by the image sparse representation approach. We considered the relative status information between neighboring residues for enhancing the performance, and the matrix of features was translated into feature image by sparse coding algorithm for noise and dimension reduction. To deal with the diverse loop length problem, we applied a dynamic threshold method, which is particularly useful for enhancing the recognition of short loops and tight turns. Our experiments demonstrate that the new protocol designed in MemBrain-TMB effectively helps improve prediction performance.

Perception Evolution Network Based on Cognition Deepening Model--Adapting to the Emergence of New Sensory Receptor

The proposed perception evolution network (PEN) is a biologically inspired neural network model for unsupervised learning and online incremental learning. It is able to automatically learn suitable prototypes from learning data in an incremental way, and it does not require the predefined prototype number or the predefined similarity threshold. Meanwhile, being more advanced than the existing unsupervised neural network model, PEN permits the emergence of a new dimension of perception in the perception field of the network. When a new dimension of perception is introduced, PEN is able to integrate the new dimensional sensory inputs with the learned prototypes, i.e., the prototypes are mapped to a high-dimensional space, which consists of both the original dimension and the new dimension of the sensory inputs. In the experiment, artificial data and real-world data are used to test the proposed PEN, and the results show that PEN can work effectively.

Constructing query-driven dynamic machine learning model with application to protein-ligand binding sites prediction

We are facing an era with annotated biological data rapidly and continuously generated. How to effectively incorporate new annotated data into the learning step is crucial for enhancing the performance of a bioinformatics prediction model. Although machine-learning-based methods have been extensively used for dealing with various biological problems, existing approaches usually train static prediction models based on fixed training datasets. The static approaches are found having several disadvantages such as low scalability and impractical when training dataset is huge. In view of this, we propose a dynamic learning framework for constructing query-driven prediction models. The key difference between the proposed framework and the existing approaches is that the training set for the machine learning algorithm of the proposed framework is dynamically generated according to the query input, as opposed to training a general model regardless of queries in traditional static methods. Accordingly, a query-driven predictor based on the smaller set of data specifically selected from the entire annotated base dataset will be applied on the query. The new way for constructing the dynamic model enables us capable of updating the annotated base dataset flexibly and using the most relevant core subset as the training set makes the constructed model having better generalization ability on the query, showing "part could be better than all" phenomenon. According to the new framework, we have implemented a dynamic protein-ligand binding sites predictor called OSML (On-site model for ligand binding sites prediction). Computer experiments on 10 different ligand types of three hierarchically organized levels show that OSML outperforms most existing predictors. The results indicate that the current dynamic framework is a promising future direction for bridging the gap between the rapidly accumulated annotated biological data and the effective machine-learning-based predictors. OSML web server and datasets are freely available at: http://www.csbio.sjtu.edu.cn/bioinf/OSML/ for academic use.

Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine

In this paper, the well known stagewise additive modeling using a multiclass exponential (SAMME) boosting algorithm is extended to address problems where there exists a natural order in the targets using a cost-sensitive approach. The proposed ensemble model uses an extreme learning machine (ELM) model as a base classifier (with the Gaussian kernel and the additional regularization parameter). The closed form of the derived weighted least squares problem is provided, and it is employed to estimate analytically the parameters connecting the hidden layer to the output layer at each iteration of the boosting algorithm. Compared to the state-of-the-art boosting algorithms, in particular those using ELM as base classifier, the suggested technique does not require the generation of a new training dataset at each iteration. The adoption of the weighted least squares formulation of the problem has been presented as an unbiased and alternative approach to the already existing ELM boosting techniques. Moreover, the addition of a cost model for weighting the patterns, according to the order of the targets, enables the classifier to tackle ordinal regression problems further. The proposed method has been validated by an experimental study by comparing it with already existing ensemble methods and ELM techniques for ordinal regression, showing competitive results.

Mining recurring concepts in a dynamic feature space

Most data stream classification techniques assume that the underlying feature space is static. However, in real-world applications the set of features and their relevance to the target concept may change over time. In addition, when the underlying concepts reappear, reusing previously learnt models can enhance the learning process in terms of accuracy and processing time at the expense of manageable memory consumption. In this paper, we propose mining recurring concepts in a dynamic feature space (MReC-DFS), a data stream classification system to address the challenges of learning recurring concepts in a dynamic feature space while simultaneously reducing the memory cost associated with storing past models. MReC-DFS is able to detect and adapt to concept changes using the performance of the learning process and contextual information. To handle recurring concepts, stored models are combined in a dynamically weighted ensemble. Incremental feature selection is performed to reduce the combined feature space. This contribution allows MReC-DFS to store only the features most relevant to the learnt concepts, which in turn increases the memory efficiency of the technique. In addition, an incremental feature selection method is proposed that dynamically determines the threshold between relevant and irrelevant features. Experimental results demonstrating the high accuracy of MReC-DFS compared with state-of-the-art techniques on a variety of real datasets are presented. The results also show the superior memory efficiency of MReC-DFS.

COMPOSE: A semisupervised learning framework for initially labeled nonstationary streaming data

An increasing number of real-world applications are associated with streaming data drawn from drifting and nonstationary distributions that change over time. These applications demand new algorithms that can learn and adapt to such changes, also known as concept drift. Proper characterization of such data with existing approaches typically requires substantial amount of labeled instances, which may be difficult, expensive, or even impractical to obtain. In this paper, we introduce compacted object sample extraction (COMPOSE), a computational geometry-based framework to learn from nonstationary streaming data, where labels are unavailable (or presented very sporadically) after initialization. We introduce the algorithm in detail, and discuss its results and performances on several synthetic and real-world data sets, which demonstrate the ability of the algorithm to learn under several different scenarios of initially labeled streaming environments. On carefully designed synthetic data sets, we compare the performance of COMPOSE against the optimal Bayes classifier, as well as the arbitrary subpopulation tracker algorithm, which addresses a similar environment referred to as extreme verification latency. Furthermore, using the real-world National Oceanic and Atmospheric Administration weather data set, we demonstrate that COMPOSE is competitive even with a well-established and fully supervised nonstationary learning algorithm that receives labeled data in every batch.

Efficient online subspace learning with an indefinite kernel for visual tracking and recognition

We propose an exact framework for online learning with a family of indefinite (not positive) kernels. As we study the case of nonpositive kernels, we first show how to extend kernel principal component analysis (KPCA) from a reproducing kernel Hilbert space to Krein space. We then formulate an incremental KPCA in Krein space that does not require the calculation of preimages and therefore is both efficient and exact. Our approach has been motivated by the application of visual tracking for which we wish to employ a robust gradient-based kernel. We use the proposed nonlinear appearance model learned online via KPCA in Krein space for visual tracking in many popular and difficult tracking scenarios. We also show applications of our kernel framework for the problem of face recognition.