Evolving logic networks with real-valued inputs for fast incremental learning

In this paper, we present a neural network structure and a fast incremental learning algorithm using this network. The proposed network structure, named Evolving Logic Networks for Real-valued inputs (ELN-R), is a data structure for storing and using the knowledge. A distinctive feature of ELN-R is that the previously learned knowledge stored in ELN-R can be used as a kind of building block in constructing new knowledge. Using this feature, the proposed learning algorithm can enhance the stability and plasticity at the same time, and as a result, the fast incremental learning can be realized. The performance of the proposed scheme is shown by a theoretical analysis and an experimental study on two benchmark problems.

Incremental linear discriminant analysis for face recognition

Dimensionality reduction methods have been successfully employed for face recognition. Among the various dimensionality reduction algorithms, linear (Fisher) discriminant analysis (LDA) is one of the popular supervised dimensionality reduction methods, and many LDA-based face recognition algorithms/systems have been reported in the last decade. However, the LDA-based face recognition systems suffer from the scalability problem. To overcome this limitation, an incremental approach is a natural solution. The main difficulty in developing the incremental LDA (ILDA) is to handle the inverse of the within-class scatter matrix. In this paper, based on the generalized singular value decomposition LDA (LDA/GSVD), we develop a new ILDA algorithm called GSVD-ILDA. Different from the existing techniques in which the new projection matrix is found in a restricted subspace, the proposed GSVD-ILDA determines the projection matrix in full space. Extensive experiments are performed to compare the proposed GSVD-ILDA with the LDA/GSVD as well as the existing ILDA methods using the face recognition technology face database and the Carneggie Mellon University Pose, Illumination, and Expression face database. Experimental results show that the proposed GSVD-ILDA algorithm gives the same performance as the LDA/GSVD with much smaller computational complexity. The experimental results also show that the proposed GSVD-ILDA gives better classification performance than the other recently proposed ILDA algorithms.

An ensemble-based incremental learning approach to data fusion

This paper introduces Learn++, an ensemble of classifiers based algorithm originally developed for incremental learning, and now adapted for information/data fusion applications. Recognizing the conceptual similarity between incremental learning and data fusion, Learn++ follows an alternative approach to data fusion, i.e., sequentially generating an ensemble of classifiers that specifically seek the most discriminating information from each data set. It was observed that Learn++ based data fusion consistently outperforms a similarly configured ensemble classifier trained on any of the individual data sources across several applications. Furthermore, even if the classifiers trained on individual data sources are fine tuned for the given problem, Learn++ can still achieve a statistically significant improvement by combining them, if the additional data sets carry complementary information. The algorithm can also identify-albeit indirectly-those data sets that do not carry such additional information. Finally, it was shown that the algorithm can consecutively learn both the supplementary novel information coming from additional data of the same source, and the complementary information coming from new data sources without requiring access to any of the previously seen data.

Comparative multiresolution wavelet analysis of ERP spectral bands using an ensemble of classifiers approach for early diagnosis of Alzheimer's disease

Early diagnosis of Alzheimer's disease (AD) is becoming an increasingly important healthcare concern. Prior approaches analyzing event-related potentials (ERPs) had varying degrees of success, primarily due to smaller study cohorts, and the inherent difficulty of the problem. A new effort using multiresolution analysis of ERPs is described. Distinctions of this study include analyzing a larger cohort, comparing different wavelets and different frequency bands, using ensemble-based decisions and, most importantly, aiming the earliest possible diagnosis of the disease. Surprising yet promising outcomes indicate that ERPs in response to novel sounds of oddball paradigm may be more reliable as a biomarker than the more commonly used responses to target sounds.

Ensemble based data fusion for early diagnosis of Alzheimer's disease

We describe an ensemble of classifiers based data fusion approach to combine information from two sources, believed to contain complimentary information, for early diagnosis of Alzheimer's disease. Specifically, we use the event related potentials recorded from the Pz and Cz electrodes of the EEG, which are further analyzed using multiresolution wavelet analysis. The proposed data fusion approach includes generating multiple classifiers trained with strategically selected subsets of the training data from each source, which are then combined through a weighted majority voting. Several factors set this study apart from similar prior efforts: we use a larger cohort, specifically target early diagnosis of the disease, use an ensemble based approach rather then a single classifier, and most importantly, we combine information from multiple sources, rather then using a single modality. We present promising results obtained from the first 35 (of 80) patients whose data are analyzed thus far.

Boosting based classification of event related potentials for early diagnosis of Alzheimer's disease

With the number of the elderly population affected by Alzheimer's disease (AD) rising, the need to find an accurate, inexpensive and non-intrusive procedure that can be made available to community healthcare providers for early diagnosis of Alzheimer's disease is becoming more and more urgent as a major health concern. Several recent studies have looked at analyzing electroencephalogram signals through the use of wavelets and neural networks. In this study, multiresolution wavelet analysis, coupled with the ensemble of classifiers based boosting algorithm is used on the P300 component of the event related potentials (ERP) to determine the feasibility of the approach as a diagnostic tool for early diagnosis of AD. The technique and its promising initial results are presented.

Parameter Incremental Learning Algorithm for Neural Networks

In this paper, a novel stochastic (or online) training algorithm for neural networks, named parameter incremental learning (PIL) algorithm, is proposed and developed. The main idea of the PIL strategy is that the learning algorithm should not only adapt to the newly presented input-output training pattern by adjusting parameters, but also preserve the prior results. A general PIL algorithm for feedforward neural networks is accordingly presented as the first-order approximate solution to an optimization problem, where the performance index is the combination of proper measures of preservation and adaptation. The PIL algorithms for the multilayer perceptron (MLP) are subsequently derived. Numerical studies show that for all the three benchmark problems used in this paper the PIL algorithm for MLP is measurably superior to the standard online backpropagation (BP) algorithm and the stochastic diagonal Levenberg-Marquardt (SDLM) algorithm in terms of the convergence speed and accuracy. Other appealing features of the PIL algorithm are that it is computationally as simple as the BP algorithm, and as easy to use as the BP algorithm. It, therefore, can be applied, with better performance, to any situations where the standard online BP algorithm is applicable.

Fuzzy ARTMAP with input relevances

We introduce a new fuzzy ARTMAP (FAM) neural network: Fuzzy ARTMAP with relevance factor (FAMR). The FAMR architecture is able to incrementally "grow" and to sequentially accommodate input-output sample pairs. Each training pair has a relevance factor assigned to it, proportional to the importance of that pair during the learning phase. The relevance factors are user-defined or computed. The FAMR can be trained as a classifier and, at the same time, as a nonparametric estimator of the probability that an input belongs to a given class. The FAMR probability estimation converges almost surely and in the mean square to the posterior probability. Our theoretical results also characterize the convergence rate of the approximation. Using a relevance factor adds more flexibility to the training phase, allowing ranking of sample pairs according to the confidence we have in the information source. We analyze the FAMR capability for mapping noisy functions when training data originates from multiple sources with known levels of noise.

A neural network method for automatic and incremental learning applied to patient-dependent seizure detection

OBJECTIVE

Describe and evaluate a neural network method for automatic and incremental learning applied to patient-dependent seizure detection. Compare the classification ability of various time-frequency methods including FFT spectrogram, spectral edge frequency and bicoherence.

METHODS

57 seizures from 10 epilepsy patients are used. A probabilistic neural network (PNN) is trained and incrementally updated in a novel fashion. The speed and accuracy of the method is evaluated with different training parameters and time-frequency methods.

RESULTS

Training the PNN on a single seizure from each record offers better performance (sensitivity = 0.89 and false-positive-rate = 0.56/h) than 3 patient-independent seizure detection algorithms. The method is virtually unaffected by the settings of various training parameters. Training is very fast (0.9 s), and the accuracy improves as more examples are added incrementally (without retraining). The overall best time-frequency method was the FFT spectrogram. The bicoherence plus the FFT spectrogram was the best method on 4 records, improving the correlation from 0.111 to 0.940 on one and from 0.288 to 0.612 on another.

CONCLUSIONS

The proposed method offers accurate, robust and virtually instantaneous training and incremental learning when applied to patient-dependent seizure detection.

SIGNIFICANCE

Accurate seizure detection can improve patient care in both the epilepsy monitoring unit and the intensive care unit. Future applications include patient-independent algorithms that continue to learn as new examples are encountered.

An incremental approach to genetic-algorithms-based classification

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multiagent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an "integration" operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed.

An incremental learning algorithm with confidence estimation for automated identification of NDE signals

An incremental learning algorithm is introduced for learning new information from additional data that may later become available, after a classifier has already been trained using a previously available database. The proposed algorithm is capable of incrementally learning new information without forgetting previously acquired knowledge and without requiring access to the original database, even when new data include examples of previously unseen classes. Scenarios requiring such a learning algorithm are encountered often in nondestructive evaluation (NDE) in which large volumes of data are collected in batches over a period of time, and new defect types may become available in subsequent databases. The algorithm, named Learn++, takes advantage of synergistic generalization performance of an ensemble of classifiers in which each classifier is trained with a strategically chosen subset of the training databases that subsequently become available. The ensemble of classifiers then is combined through a weighted majority voting procedure. Learn++ is independent of the specific classifier(s) comprising the ensemble, and hence may be used with any supervised learning algorithm. The voting procedure also allows Learn++ to estimate the confidence in its own decision. We present the algorithm and its promising results on two separate ultrasonic weld inspection applications.

Neocognitron capable of incremental learning

This paper proposes a new neocognitron that accepts incremental learning, without giving a severe damage to old memories or reducing learning speed. The new neocognitron uses a competitive learning, and the learning of all stages of the hierarchical network progresses simultaneously. To increase the learning speed, conventional neocognitrons of recent versions sacrificed the ability of incremental learning, and used a technique of sequential construction of layers, by which the learning of a layer started after the learning of the preceding layers had completely finished. If the learning speed is simply set high for the conventional neocognitron, simultaneous construction of layers produces many garbage cells, which become always silent after having finished the learning. The proposed neocognitron with a new learning method can prevent the generation of such garbage cells even with a high learning speed, allowing incremental learning.