Reformulated radial basis neural networks trained by gradient descent

Selective Memory Recursive Least Squares: Recast Forgetting Into Memory in RBF Neural Network-Based Real-Time Learning

In radial basis function neural network (RBFNN)-based real-time learning tasks, forgetting mechanisms are widely used such that the neural network can keep its sensitivity to new data. However, with forgetting mechanisms, some useful knowledge will get lost simply because they are learned a long time ago, which we refer to as the passive knowledge forgetting phenomenon. To address this problem, this article proposes a real-time training method named selective memory recursive least squares (SMRLS) in which the classical forgetting mechanisms are recast into a memory mechanism. Different from the forgetting mechanism, which mainly evaluates the importance of samples according to the time when samples are collected, the memory mechanism evaluates the importance of samples through both temporal and spatial distribution of samples. With SMRLS, the input space of the RBFNN is evenly divided into a finite number of partitions, and a synthesized objective function is developed using synthesized samples from each partition. In addition to the current approximation error, the neural network also updates its weights according to the recorded data from the partition being visited. Compared with classical training methods including the forgetting factor recursive least squares (FFRLS) and stochastic gradient descent (SGD) methods, SMRLS achieves improved learning speed and generalization capability, which are demonstrated by corresponding simulation results.

Generalized M-sparse algorithms for constructing fault tolerant RBF networks

In the construction process of radial basis function (RBF) networks, two common crucial issues arise: the selection of RBF centers and the effective utilization of the given source without encountering the overfitting problem. Another important issue is the fault tolerant capability. That is, when noise or faults exist in a trained network, it is crucial that the network's performance does not undergo significant deterioration or decrease. However, without employing a fault tolerant procedure, a trained RBF network may exhibit significantly poor performance. Unfortunately, most existing algorithms are unable to simultaneously address all of the aforementioned issues. This paper proposes fault tolerant training algorithms that can simultaneously select RBF nodes and train RBF output weights. Additionally, our algorithms can directly control the number of RBF nodes in an explicit manner, eliminating the need for a time-consuming procedure to tune the regularization parameter and achieve the target RBF network size. Based on simulation results, our algorithms demonstrate improved test set performance when more RBF nodes are used, effectively utilizing the given source without encountering the overfitting problem. This paper first defines a fault tolerant objective function, which includes a term to suppress the effects of weight faults and weight noise. This term also prevents the issue of overfitting, resulting in better test set performance when more RBF nodes are utilized. With the defined objective function, the training process is designed to solve a generalized M-sparse problem by incorporating an ℓ0-norm constraint. The ℓ0-norm constraint allows us to directly and explicitly control the number of RBF nodes. To address the generalized M-sparse problem, we introduce the noise-resistant iterative hard thresholding (NR-IHT) algorithm. The convergence properties of the NR-IHT algorithm are subsequently discussed theoretically. To further enhance performance, we incorporate the momentum concept into the NR-IHT algorithm, referring to the modified version as "NR-IHT-Mom". Simulation results show that both the NR-IHT algorithm and the NR-IHT-Mom algorithm outperform several state-of-the-art comparison algorithms.

Artificial Intelligence Models for the Mass Loss of Copper-Based Alloys under Cavitation

Cavitation is a physical process that produces different negative effects on the components working in conditions where it acts. One is the materials' mass loss by corrosion-erosion when it is introduced into fluids under cavitation. This research aims at modeling the mass variation of three samples (copper, brass, and bronze) in a cavitation field produced by ultrasound in water, using four artificial intelligence methods-SVR, GRNN, GEP, and RBF networks. Utilizing six goodness-of-fit indicators (R2, MAE, RMSE, MAPE, CV, correlation between the recorded and computed values), it is shown that the best results are provided by GRNN, followed by SVR. The novelty of the approach resides in the experimental data collection and analysis.

AutoML-ID: automated machine learning model for intrusion detection using wireless sensor network

Momentous increase in the popularity of explainable machine learning models coupled with the dramatic increase in the use of synthetic data facilitates us to develop a cost-efficient machine learning model for fast intrusion detection and prevention at frontier areas using Wireless Sensor Networks (WSNs). The performance of any explainable machine learning model is driven by its hyperparameters. Several approaches have been developed and implemented successfully for optimising or tuning these hyperparameters for skillful predictions. However, the major drawback of these techniques, including the manual selection of the optimal hyperparameters, is that they depend highly on the problem and demand application-specific expertise. In this paper, we introduced Automated Machine Learning (AutoML) model to automatically select the machine learning model (among support vector regression, Gaussian process regression, binary decision tree, bagging ensemble learning, boosting ensemble learning, kernel regression, and linear regression model) and to automate the hyperparameters optimisation for accurate prediction of numbers of k-barriers for fast intrusion detection and prevention using Bayesian optimisation. To do so, we extracted four synthetic predictors, namely, area of the region, sensing range of the sensor, transmission range of the sensor, and the number of sensors using Monte Carlo simulation. We used 80% of the datasets to train the models and the remaining 20% for testing the performance of the trained model. We found that the Gaussian process regression performs prodigiously and outperforms all the other considered explainable machine learning models with correlation coefficient (R = 1), root mean square error (RMSE = 0.007), and bias = − 0.006. Further, we also tested the AutoML performance on a publicly available intrusion dataset, and we observed a similar performance. This study will help the researchers accurately predict the required number of k-barriers for fast intrusion detection and prevention.

Human Activity Classification Using Multilayer Perceptron

The number of smart homes is rapidly increasing. Smart homes typically feature functions such as voice-activated functions, automation, monitoring, and tracking events. Besides comfort and convenience, the integration of smart home functionality with data processing methods can provide valuable information about the well-being of the smart home residence. This study is aimed at taking the data analysis within smart homes beyond occupancy monitoring and fall detection. This work uses a multilayer perceptron neural network to recognize multiple human activities from wrist- and ankle-worn devices. The developed models show very high recognition accuracy across all activity classes. The cross-validation results indicate accuracy levels above 98% across all models, and scoring evaluation methods only resulted in an average accuracy reduction of 10%.

A New General Maximum Intensity Projection Technology via the Hybrid of U-Net and Radial Basis Function Neural Network

Maximum intensity projection (MIP) technology is a computer visualization method that projects three-dimensional spatial data on a visualization plane. According to the specific purposes, the specific lab thickness and direction can be selected. This technology can better show organs, such as blood vessels, arteries, veins, and bronchi and so forth, from different directions, which could bring more intuitive and comprehensive results for doctors in the diagnosis of related diseases. However, in this traditional projection technology, the details of the small projected target are not clearly visualized when the projected target is not much different from the surrounding environment, which could lead to missed diagnosis or misdiagnosis. Therefore, it is urgent to develop a new technology that can better and clearly display the angiogram. However, to the best of our knowledge, research in this area is scarce. To fill this gap in the literature, in the present study, we propose a new method based on the hybrid of convolutional neural network (CNN) and radial basis function neural network (RBFNN) to synthesize the projection image. We first adopted the U-net to obtain feature or enhanced images to be projected; subsequently, the RBF neural network performed further synthesis processing for these data; finally, the projection images were obtained. For experimental data, in order to increase the robustness of the proposed algorithm, the following three different types of datasets were adopted: the vascular projection of the brain, the bronchial projection of the lung parenchyma, and the vascular projection of the liver. In addition, radiologist evaluation and five classic metrics of image definition were implemented for effective analysis. Finally, compared to the traditional MIP technology and other structures, the use of a large number of different types of data and superior experimental results proved the versatility and robustness of the proposed method.

An Exact Reformulation of Feature-Vector-Based Radial-Basis-Function Networks for Graph-Based Observations

Radial basis function (RBF) networks are traditionally defined for sets of vector-based observations. In this brief, we reformulate such networks so that they can be applied to adjacency-matrix representations of weighted, directed graphs that represent the relationships between object pairs. We restate the sum-of-squares objective function so that it is purely dependent on entries from the adjacency matrix. From this objective function, we derive a gradient descent update for the network weights. We also derive a gradient update that simulates the repositioning of the radial basis prototypes and changes in the radial basis prototype parameters. An important property of our radial basis function networks is that they are guaranteed to yield the same responses as conventional radial basis networks trained on a corresponding vector realization of the relationships encoded by the adjacency matrix. Such a vector realization only needs to provably exist for this property to hold, which occurs whenever the relationships correspond to distances from some arbitrary metric applied to a latent set of vectors. We, therefore, completely avoid needing to actually construct vectorial realizations via multidimensional scaling, which ensures that the underlying relationships are totally preserved.

Predicting MicroRNA Sequence Using CNN and LSTM Stacked in Seq2Seq Architecture

CNN and LSTM have proven their ability in feature extraction and natural language processing, respectively. So, we tried to use their ability to process the language of RNAs, i.e., predicting sequence of microRNAs using the sequence of mRNA. The idea is to extract the features from sequence of mRNA using CNN and use LSTM network for prediction of miRNA. The model has learned the basic features such as seed match at first 2-8 nucleotides starting at the 5' end and counting toward the 3' end. Also, it was able to predict G-U wobble base pair in seed region. While validating on experimentally validated data, the model was able to predict on average 72 percent of miRNAs for specific mRNA and shows highest positive expression fold change of predicted targets on a microarray data generated using anti 25 miRNAs compare to other predicted tools. Codes are available at https://github.com/rajkumar1501/sequence-prediction-using-CNN-and-LSTMs.

Improving land-use change modeling by integrating ANN with Cellular Automata-Markov Chain model

Urban growth and land-use change are a few of many puzzling factors affecting our future cities. Creating a precise simulation for future land change is a challenging process that requires temporal and spatial modeling. Many recent studies developed and trained models to predict urban expansion patterns using Artificial Intelligence (AI). This study aims to enhance the simulation capability of Cellular Automata Markov Chain (CA-MC) model in predicting changes in land-use. This study integrates the Artificial Neural Network (ANN) into CA-MC to incorporate several driving forces that highly impact land-use change. The research utilizes different socio-economic, spatial, and environmental variables (slope, distance to road, distance to urban centers, distance to commercial, density, elevation, and land fertility) to generate potential transition maps using ANN Data-driven model. The generated maps are fed to CA-MC as additional inputs. We calibrated the original CA-MC and our models for 2015 cross-comparing simulated maps and actual maps obtained for Irbid city, Jordan in 2015. Validation of our model was assessed and compared to the CA-MC model using Kappa indices including the agreement in terms of quantity and location. The results elucidated that our model with an accuracy of 90.04% substantially outperforms CA-MC (86.29%) model. The improvement we obtained from integrating ANN with CA-MC suggested that the influence imposed by the driving force was necessary to be taken into account for more accurate prediction. In addition to the improved model prediction, the predicted maps of Irbid for the years 2021 and 2027 will guide local authorities in the development of management strategies that balance urban expansion and protect agricultural regions. This will play a vital role in sustaining Jordan's food security.

Removal of computed tomography ring artifacts via radial basis function artificial neural networks

Ring artifacts in computed tomography (CT) images are caused by the undesirable response of detector pixels, which leads to the degradation of CT images. Accordingly, it affects the image interpretation, post-processing, and quantitative analysis. In this study, a radial basis function neural network (RBFNN) was used to remove ring artifacts. The proposed method employs polar coordinate transformation. First, ring artifacts were transformed into linear artifacts by polar coordinate transformation. Then, smoothing operators were applied to locate these artifacts exactly. Subsequently, RBFNN was operated on each linear artifact. The neuron numbers of the input, hidden, and output layers of the neural network were 8, 40, and 1, respectively. Neurons in the input layer were selected according to the characteristics of the artifact itself and its relationship with the surrounding normal pixels. For the training of the neural network, a hybrid of adaptive gradient descent algorithm (AGDA) and gravitational search algorithm (GSA) was adopted. After the corrected image was obtained using the updated neural network, the inverse coordinate transformation was implemented. The experimental data were divided into simulated ring artifacts and real ring artifacts, which were based on brain and abdomen CT images. Compared with current artifact removal methods, the proposed method removed ring artifacts more effectively and retained the maximum detail of normal tissues. In addition, for index analysis, the performance of proposed method was superior to that of the other methods.

Reference-shaping adaptive control by using gradient descent optimizers

This study presents a model reference adaptive control scheme based on reference-shaping approach. The proposed adaptive control structure includes two optimizer processes that perform gradient descent optimization. The first process is the control optimizer that generates appropriate control signal for tracking of the controlled system output to a reference model output. The second process is the adaptation optimizer that performs for estimation of a time-varying adaptation gain, and it contributes to improvement of control signal generation. Numerical update equations derived for adaptation gain and control signal perform gradient descent optimization in order to decrease the model mismatch errors. To reduce noise sensitivity of the system, a dead zone rule is applied to the adaptation process. Simulation examples show the performance of the proposed Reference-Shaping Adaptive Control (RSAC) method for several test scenarios. An experimental study demonstrates application of method for rotor control.

Two-Stage Orthogonal Least Squares Methods for Neural Network Construction

A number of neural networks can be formulated as the linear-in-the-parameters models. Training such networks can be transformed to a model selection problem where a compact model is selected from all the candidates using subset selection algorithms. Forward selection methods are popular fast subset selection approaches. However, they may only produce suboptimal models and can be trapped into a local minimum. More recently, a two-stage fast recursive algorithm (TSFRA) combining forward selection and backward model refinement has been proposed to improve the compactness and generalization performance of the model. This paper proposes unified two-stage orthogonal least squares methods instead of the fast recursive-based methods. In contrast to the TSFRA, this paper derives a new simplified relationship between the forward and the backward stages to avoid repetitive computations using the inherent orthogonal properties of the least squares methods. Furthermore, a new term exchanging scheme for backward model refinement is introduced to reduce computational demand. Finally, given the error reduction ratio criterion, effective and efficient forward and backward subset selection procedures are proposed. Extensive examples are presented to demonstrate the improved model compactness constructed by the proposed technique in comparison with some popular methods.

Sparse multivariate gaussian mixture regression

Fitting a multivariate Gaussian mixture to data represents an attractive, as well as challenging problem, in especial when sparsity in the solution is demanded. Achieving this objective requires the concurrent update of all parameters (weight, centers, and precisions) of all multivariate Gaussian functions during the learning process. Such is the focus of this paper, which presents a novel method founded on the minimization of the error of the generalized logarithmic utility function (GLUF). This choice, which allows us to move smoothly from the mean square error (MSE) criterion to the one based on the logarithmic error, yields an optimization problem that resembles a locally convex problem and can be solved with a quasi-Newton method. The GLUF framework also facilitates the comparative study between both extremes, concluding that the classical MSE optimization is not the most adequate for the task. The performance of the proposed novel technique is demonstrated on simulated as well as realistic scenarios.

A new discrete-continuous algorithm for radial basis function networks construction

The construction of a radial basis function (RBF) network involves the determination of the model size, hidden nodes, and output weights. Least squares-based subset selection methods can determine a RBF model size and its parameters simultaneously. Although these methods are robust, they may not achieve optimal results. Alternatively, gradient methods are widely used to optimize all the parameters. The drawback is that most algorithms may converge slowly as they treat hidden nodes and output weights separately and ignore their correlations. In this paper, a new discrete-continuous algorithm is proposed for the construction of a RBF model. First, the orthogonal least squares (OLS)-based forward stepwise selection constructs an initial model by selecting model terms one by one from a candidate term pool. Then a new Levenberg-Marquardt (LM)-based parameter optimization is proposed to further optimize the hidden nodes and output weights in the continuous space. To speed up the convergence, the proposed parameter optimization method considers the correlation between the hidden nodes and output weights, which is achieved by translating the output weights to dependent parameters using the OLS method. The correlation is also used by the previously proposed continuous forward algorithm (CFA). However, unlike the CFA, the new method optimizes all the parameters simultaneously. In addition, an equivalent recursive sum of squared error is derived to reduce the computation demanding for the first derivatives used in the LM method. Computational complexity is given to confirm the new method is much more computationally efficient than the CFA. Different numerical examples are presented to illustrate the effectiveness of the proposed method. Further, Friedman statistical tests on 13 classification problems are performed, and the results demonstrate that RBF networks built by the new method are very competitive in comparison with some popular classifiers.

A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation

Radial basis function (RBF) neural network is constructed of certain number of RBF neurons, and these networks are among the most used neural networks for modeling of various nonlinear problems in engineering. Conventional RBF neuron is usually based on Gaussian type of activation function with single width for each activation function. This feature restricts neuron performance for modeling the complex nonlinear problems. To accommodate limitation of a single scale, this paper presents neural network with similar but yet different activation function-hyper basis function (HBF). The HBF allows different scaling of input dimensions to provide better generalization property when dealing with complex nonlinear problems in engineering practice. The HBF is based on generalization of Gaussian type of neuron that applies Mahalanobis-like distance as a distance metrics between input training sample and prototype vector. Compared to the RBF, the HBF neuron has more parameters to optimize, but HBF neural network needs less number of HBF neurons to memorize relationship between input and output sets in order to achieve good generalization property. However, recent research results of HBF neural network performance have shown that optimal way of constructing this type of neural network is needed; this paper addresses this issue and modifies sequential learning algorithm for HBF neural network that exploits the concept of neuron's significance and allows growing and pruning of HBF neuron during learning process. Extensive experimental study shows that HBF neural network, trained with developed learning algorithm, achieves lower prediction error and more compact neural network.

Deterministic global optimization for FNN training

This paper addresses the issue of training feedforward neural networks by global optimization. The main contributions include characterization of global optimality of a network error function, and formulation of a global descent algorithm to solve the network training problem. A network with a single hidden-layer and a single-output unit is considered. By means of a monotonic transformation, a sufficient condition for global optimality of a network error function is presented. Based on this, a penalty-based algorithm is derived directing the search towards possible regions containing the global minima. Numerical comparison with benchmark problems from the neural network literature shows superiority of the proposed algorithm over some local methods, in terms of the percentage of trials attaining the desired solutions. The algorithm is also shown to be effective for several pattern recognition problems.

Fast and efficient second-order method for training radial basis function networks

This paper proposes an improved second order (ISO) algorithm for training radial basis function (RBF) networks. Besides the traditional parameters, including centers, widths and output weights, the input weights on the connections between input layer and hidden layer are also adjusted during the training process. More accurate results can be obtained by increasing variable dimensions. Initial centers are chosen from training patterns and other parameters are generated randomly in limited range. Taking the advantages of fast convergence and powerful search ability of second order algorithms, the proposed ISO algorithm can normally reach smaller training/testing error with much less number of RBF units. During the computation process, quasi Hessian matrix and gradient vector are accumulated as the sum of related sub matrices and vectors, respectively. Only one Jacobian row is stored and used for multiplication, instead of the entire Jacobian matrix storage and multiplication. Memory reduction benefits the computation speed and allows the training of problems with basically unlimited number of patterns. Several practical discrete and continuous classification problems are applied to test the properties of the proposed ISO training algorithm.

Using Radial Basis Function Networks for Function Approximation and Classification

The radial basis function (RBF) network has its foundation in the conventional approximation theory. It has the capability of universal approximation. The RBF network is a popular alternative to the well-known multilayer perceptron (MLP), since it has a simpler structure and a much faster training process. In this paper, we give a comprehensive survey on the RBF network and its learning. Many aspects associated with the RBF network, such as network structure, universal approimation capability, radial basis functions, RBF network learning, structure optimization, normalized RBF networks, application to dynamic system modeling, and nonlinear complex-valued signal processing, are described. We also compare the features and capability of the two models.

A Comparison of RBF Neural Network Training Algorithms for Inertial Sensor Based Terrain Classification

This paper introduces a comparison of training algorithms of radial basis function (RBF) neural networks for classification purposes. RBF networks provide effective solutions in many science and engineering fields. They are especially popular in the pattern classification and signal processing areas. Several algorithms have been proposed for training RBF networks. The Artificial Bee Colony (ABC) algorithm is a new, very simple and robust population based optimization algorithm that is inspired by the intelligent behavior of honey bee swarms. The training performance of the ABC algorithm is compared with the Genetic algorithm, Kalman filtering algorithm and gradient descent algorithm. In the experiments, not only well known classification problems from the UCI repository such as the Iris, Wine and Glass datasets have been used, but also an experimental setup is designed and inertial sensor based terrain classification for autonomous ground vehicles was also achieved. Experimental results show that the use of the ABC algorithm results in better learning than those of others.

A radial basis function neural network approach to determine the survival of Listeria monocytogenes in Katiki, a traditional Greek soft cheese

A radial basis function neural network was developed to determine the kinetic behavior of Listeria monocytogenes in Katiki, a traditional white acid-curd soft spreadable cheese. The applicability of the neural network approach was compared with the reparameterized Gompertz, the modified Weibull, and the Geeraerd primary models. Model performance was assessed with the root mean square error of the residuals of the model (RMSE), the regression coefficient (R2), and the F test. Commercially prepared cheese samples were artificially inoculated with a five-strain cocktail of L. monocytogenes, with an initial concentration of 10(6) CFU g(-1) and stored at 5, 10, 15, and 20 degrees C for 40 days. At each storage temperature, a pathogen viability loss profile was evident and included a shoulder, a log-linear phase, and a tailing phase. The developed neural network described the survival of L. monocytogenes equally well or slightly better than did the three primary models. The performance indices for the training subset of the network were R2 = 0.993 and RMSE = 0.214. The relevant mean values for all storage temperatures were R2 = 0.981, 0.986, and 0.985 and RMSE = 0.344, 0.256, and 0.262 for the reparameterized Gompertz, modified Weibull, and Geeraerd models, respectively. The results of the F test indicated that none of the primary models were able to describe accurately the survival of the pathogen at 5 degrees C, whereas with the neural network all fvalues were significant. The neural network and primary models all were validated under constant temperature storage conditions (12 and 17 degrees C). First or second order polynomial models were used to relate the inactivation parameters to temperature, whereas the neural network was used a one-step modeling approach. Comparison of the prediction capability was based on bias and accuracy factors and on the goodness-of-fit index. The prediction performance of the neural network approach was equal to that of the primary models at both validation temperatures. The results of this work could increase the knowledge basis for the applicability of neural networks as an alternative tool in predictive microbiology.

Modelling fungal growth using radial basis function neural networks: The case of the ascomycetous fungus Monascus ruber van Tieghem

A radial basis function (RBF) neural network was developed and evaluated against a quadratic response surface model to predict the maximum specific growth rate of the ascomycetous fungus Monascus ruber in relation to temperature (20-40 degrees C), water activity (0.937-0.970) and pH (3.5-5.0), based on the data of Panagou et al. [Panagou, E.Z., Skandamis, P.N., Nychas, G.-J.E., 2003. Modelling the combined effect of temperature, pH and aw on the growth rate of M. ruber, a heat-resistant fungus isolated from green table olives. J. Appl. Microbiol. 94, 146-156]. Both RBF network and polynomial model were compared against the experimental data using five statistical indices namely, coefficient of determination (R(2)), root mean square error (RMSE), standard error of prediction (SEP), bias (B(f)) and accuracy (A(f)) factors. Graphical plots were also used for model comparison. For training data set the RBF network predictions outperformed the classical statistical model, whereas in the case of test data set the network gave reasonably good predictions, considering its performance for unseen data. Sensitivity analysis showed that from the three environmental factors the most influential on fungal growth was temperature, followed by water activity and pH to a lesser extend. Neural networks offer an alternative and powerful technique to model microbial kinetic parameters and could thus become an additional tool in predictive mycology.

Automated Detection of Videotaped Neonatal Seizures Based on Motion Tracking Methods

This study was carried out during the second phase of the project "Video Technologies for Neonatal Seizures" and aimed at the development of a seizure detection system by training neural networks, using quantitative motion information extracted by motion tracking methods from short video segments of infants monitored for seizures. The motion of the infants' body parts was quantified by temporal motion trajectory signals extracted from video recordings by robust motion trackers, based on block motion models. These motion trackers were developed to autonomously adjust to illumination and contrast changes that may occur during the video frame sequence. The computational tools and procedures developed for automated seizure detection were evaluated on short video segments selected and labeled by physicians from a set of 240 video recordings of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments). This evaluation provided the basis for selecting the most effective strategy for training neural networks to detect neonatal seizures as well as the decision scheme used for interpreting the responses of the trained neural networks. The best neural networks exhibited sensitivity and specificity above 90%. The best among the motion trackers developed in this study produced quantitative features that constitute a reliable basis for detecting myoclonic and focal clonic neonatal seizures. The performance targets of the second phase of the project may be achieved by combining the quantitative features described in this paper with those obtained by analyzing motion strength signals produced by motion segmentation methods.

A Hybrid Forward Algorithm for RBF Neural Network Construction

This paper proposes a novel hybrid forward algorithm (HFA) for the construction of radial basis function (RBF) neural networks with tunable nodes. The main objective is to efficiently and effectively produce a parsimonious RBF neural network that generalizes well. In this study, it is achieved through simultaneous network structure determination and parameter optimization on the continuous parameter space. This is a mixed integer hard problem and the proposed HFA tackles this problem using an integrated analytic framework, leading to significantly improved network performance and reduced memory usage for the network construction. The computational complexity analysis confirms the efficiency of the proposed algorithm, and the simulation results demonstrate its effectiveness.

Training reformulated radial basis function neural networks capable of identifying uncertainty in data classification

This paper introduces a learning algorithm that can be used for training reformulated radial basis function neural networks (RBFNNs) capable of identifying uncertainty in data classification. This learning algorithm trains a special class of reformulated RBFNNs, known as cosine RBFNNs, by updating selected adjustable parameters to minimize the class-conditional variances at the outputs of their radial basis functions (RBFs). The experiments verify that quantum neural networks (QNNs) and cosine RBFNNs trained by the proposed learning algorithm are capable of identifying uncertainty in data classification, a property that is not shared by cosine RBFNNs trained by the original learning algorithm and conventional feed-forward neural networks (FFNNs). Finally, this study leads to a simple classification strategy that can be used to improve the classification accuracy of QNNs and cosine RBFNNs by rejecting ambiguous feature vectors based on their responses.

Automated Detection of Videotaped Neonatal Seizures of Epileptic Origin

PURPOSE

This study aimed at the development of a seizure-detection system by training neural networks with quantitative motion information extracted from short video segments of neonatal seizures of the myoclonic and focal clonic types and random infant movements.

METHODS

The motion of the infants' body parts was quantified by temporal motion-strength signals extracted from video segments by motion-segmentation methods based on optical flow computation. The area of each frame occupied by the infants' moving body parts was segmented by clustering the motion parameters obtained by fitting an affine model to the pixel velocities. The motion of the infants' body parts also was quantified by temporal motion-trajectory signals extracted from video recordings by robust motion trackers based on block-motion models. These motion trackers were developed to adjust autonomously to illumination and contrast changes that may occur during the video-frame sequence. Video segments were represented by quantitative features obtained by analyzing motion-strength and motion-trajectory signals in both the time and frequency domains. Seizure recognition was performed by conventional feed-forward neural networks, quantum neural networks, and cosine radial basis function neural networks, which were trained to detect neonatal seizures of the myoclonic and focal clonic types and to distinguish them from random infant movements.

RESULTS

The computational tools and procedures developed for automated seizure detection were evaluated on a set of 240 video segments of 54 patients exhibiting myoclonic seizures (80 segments), focal clonic seizures (80 segments), and random infant movements (80 segments). Regardless of the decision scheme used for interpreting the responses of the trained neural networks, all the neural network models exhibited sensitivity and specificity>90%. For one of the decision schemes proposed for interpreting the responses of the trained neural networks, the majority of the trained neural-network models exhibited sensitivity>90% and specificity>95%. In particular, cosine radial basis function neural networks achieved the performance targets of this phase of the project (i.e., sensitivity>95% and specificity>95%).

CONCLUSIONS

The best among the motion segmentation and tracking methods developed in this study produced quantitative features that constitute a reliable basis for detecting neonatal seizures. The performance targets of this phase of the project were achieved by combining the quantitative features obtained by analyzing motion-strength signals with those produced by analyzing motion-trajectory signals. The computational procedures and tools developed in this study to perform off-line analysis of short video segments will be used in the next phase of this project, which involves the integration of these procedures and tools into a system that can process and analyze long video recordings of infants monitored for seizures in real time.

An Ensemble of Cooperative Extended Kohonen Maps for Complex Robot Motion Tasks

Self-organizing feature maps such as extended Kohonen maps (EKMs) have been very successful at learning sensorimotor control for mobile robot tasks. This letter presents a new ensemble approach, cooperative EKMs with indirect mapping, to achieve complex robot motion. An indirect-mapping EKM self-organizes to map from the sensory input space to the motor control space indirectly via a control parameter space. Quantitative evaluation reveals that indirect mapping can provide finer, smoother, and more efficient motion control than does direct mapping by operating in a continuous, rather than discrete, motor control space. It is also shown to outperform basis function neural networks. Furthermore, training its control parameters with recursive least squares enables faster convergence and better performance compared to gradient descent. The cooperation and competition of multiple self-organized EKMs allow a nonholonomic mobile robot to negotiate unforeseen, concave, closely spaced, and dynamic obstacles. Qualitative and quantitative comparisons with neural network ensembles employing weighted sum reveal that our method can achieve more sophisticated motion tasks even though the weighted-sum ensemble approach also operates in continuous motor control space.

Analysis of augmented-input-layer RBFNN

In this paper we present and analyze a new structure for designing a radial basis function neural network (RBFNN). In the training phase, input layer of RBFNN is augmented with desired output vector. Generalization phase involves the following steps: (1) identify the cluster to which a previously unseen input vector belongs; (2) augment the input layer with an average of the targets of the input vectors in the identified cluster; and (3) use the augmented network to estimate the unknown target. It is shown that, under some reasonable assumptions, the generalization error function admits an upper bound in terms of the quantization errors minimized when determining the centers of the proposed method over the training set and the difference between training samples and generalization samples in a deterministic setting. When the difference between the training and generalization samples goes to zero, the upper bound can be made arbitrarily small by increasing the number of hidden neurons. Computer simulations verified the effectiveness of the proposed method.