Relaxed fault-tolerant hardware implementation of neural networks in the presence of multiple transient errors

Generalized M-sparse algorithms for constructing fault tolerant RBF networks

In the construction process of radial basis function (RBF) networks, two common crucial issues arise: the selection of RBF centers and the effective utilization of the given source without encountering the overfitting problem. Another important issue is the fault tolerant capability. That is, when noise or faults exist in a trained network, it is crucial that the network's performance does not undergo significant deterioration or decrease. However, without employing a fault tolerant procedure, a trained RBF network may exhibit significantly poor performance. Unfortunately, most existing algorithms are unable to simultaneously address all of the aforementioned issues. This paper proposes fault tolerant training algorithms that can simultaneously select RBF nodes and train RBF output weights. Additionally, our algorithms can directly control the number of RBF nodes in an explicit manner, eliminating the need for a time-consuming procedure to tune the regularization parameter and achieve the target RBF network size. Based on simulation results, our algorithms demonstrate improved test set performance when more RBF nodes are used, effectively utilizing the given source without encountering the overfitting problem. This paper first defines a fault tolerant objective function, which includes a term to suppress the effects of weight faults and weight noise. This term also prevents the issue of overfitting, resulting in better test set performance when more RBF nodes are utilized. With the defined objective function, the training process is designed to solve a generalized M-sparse problem by incorporating an ℓ0-norm constraint. The ℓ0-norm constraint allows us to directly and explicitly control the number of RBF nodes. To address the generalized M-sparse problem, we introduce the noise-resistant iterative hard thresholding (NR-IHT) algorithm. The convergence properties of the NR-IHT algorithm are subsequently discussed theoretically. To further enhance performance, we incorporate the momentum concept into the NR-IHT algorithm, referring to the modified version as "NR-IHT-Mom". Simulation results show that both the NR-IHT algorithm and the NR-IHT-Mom algorithm outperform several state-of-the-art comparison algorithms.

Adaptive Neural Network-Based Observer Design for Switched Systems With Quantized Measurements

This study is concerned with the adaptive neural network (NN) observer design problem for continuous-time switched systems via quantized output signals. A novel NN observer is presented in which the adaptive laws are constructed using quantized measurements. Then, persistent dwell time (PDT) switching is considered in the observer design to describe fast and slow switching in a unified framework. Accurate estimations of state and actuator efficiency factor can be obtained by the proposed observer technique despite actuator degradation. Finally, a simulation example is provided to illustrate the effectiveness of the developed NN observer design approach.

Event-Triggered Cooperative Model-Free Adaptive Iterative Learning Control for Multiple Subway Trains With Actuator Faults

This article investigates the issue of speed tracking and dynamic adjustment of headway for the repeatable multiple subway trains (MSTs) system in the case of actuator faults. First, the repeatable nonlinear subway train system is transformed into an iteration-related full-form dynamic linearization (IFFDL) data model. Then, the event-triggered cooperative model-free adaptive iterative learning control (ET-CMFAILC) scheme based on the IFFDL data model for MSTs is designed. The control scheme includes the following four parts: 1) the cooperative control algorithm is derived by the cost function to realize cooperation of MSTs; 2) the radial basis function neural network (RBFNN) algorithm along the iteration axis is constructed to compensate the effects of iteration-time-varying actuator faults; 3) the projection algorithm is employed to estimate unknown complex nonlinear terms; and 4) the asynchronous event-triggered mechanism operated along the time domain and iteration domain is applied to lessen the communication and computational burden. Theoretical analysis and simulation results show that the effectiveness of the proposed ET-CMFAILC scheme, which can ensure that the speed tracking errors of MSTs are bounded and the distances of adjacent subway trains are stabilized in the safe range.

ADMM-Based Algorithm for Training Fault Tolerant RBF Networks and Selecting Centers

In the training stage of radial basis function (RBF) networks, we need to select some suitable RBF centers first. However, many existing center selection algorithms were designed for the fault-free situation. This brief develops a fault tolerant algorithm that trains an RBF network and selects the RBF centers simultaneously. We first select all the input vectors from the training set as the RBF centers. Afterward, we define the corresponding fault tolerant objective function. We then add an -norm term into the objective function. As the -norm term is able to force some unimportant weights to zero, center selection can be achieved at the training stage. Since the -norm term is nondifferentiable, we formulate the original problem as a constrained optimization problem. Based on the alternating direction method of multipliers framework, we then develop an algorithm to solve the constrained optimization problem. The convergence proof of the proposed algorithm is provided. Simulation results show that the proposed algorithm is superior to many existing center selection algorithms.

Integral Sliding Mode Fault-Tolerant Control for Uncertain Linear Systems Over Networks With Signals Quantization

In this paper, a new robust fault-tolerant compensation control method for uncertain linear systems over networks is proposed, where only quantized signals are assumed to be available. This approach is based on the integral sliding mode (ISM) method where two kinds of integral sliding surfaces are constructed. One is the continuous-state-dependent surface with the aim of sliding mode stability analysis and the other is the quantization-state-dependent surface, which is used for ISM controller design. A scheme that combines the adaptive ISM controller and quantization parameter adjustment strategy is then proposed. Through utilizing H_∞ control analytical technique, once the system is in the sliding mode, the nature of performing disturbance attenuation and fault tolerance from the initial time can be found without requiring any fault information. Finally, the effectiveness of our proposed ISM control fault-tolerant schemes against quantization errors is demonstrated in the simulation.

A Regularizer Approach for RBF Networks Under the Concurrent Weight Failure Situation

Many existing results on fault-tolerant algorithms focus on the single fault source situation, where a trained network is affected by one kind of weight failure. In fact, a trained network may be affected by multiple kinds of weight failure. This paper first studies how the open weight fault and the multiplicative weight noise degrade the performance of radial basis function (RBF) networks. Afterward, we define the objective function for training fault-tolerant RBF networks. Based on the objective function, we then develop two learning algorithms, one batch mode and one online mode. Besides, the convergent conditions of our online algorithm are investigated. Finally, we develop a formula to estimate the test set error of faulty networks trained from our approach. This formula helps us to optimize some tuning parameters, such as RBF width.

The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.

Objective Function and Learning Algorithm for the General Node Fault Situation

Fault tolerance is one interesting property of artificial neural networks. However, the existing fault models are able to describe limited node fault situations only, such as stuck-at-zero and stuck-at-one. There is no general model that is able to describe a large class of node fault situations. This paper studies the performance of faulty radial basis function (RBF) networks for the general node fault situation. We first propose a general node fault model that is able to describe a large class of node fault situations, such as stuck-at-zero, stuck-at-one, and the stuck-at level being with arbitrary distribution. Afterward, we derive an expression to describe the performance of faulty RBF networks. An objective function is then identified from the formula. With the objective function, a training algorithm for the general node situation is developed. Finally, a mean prediction error (MPE) formula that is able to estimate the test set error of faulty networks is derived. The application of the MPE formula in the selection of basis width is elucidated. Simulation experiments are then performed to demonstrate the effectiveness of the proposed method.