Additive Fuzzy Systems: From Generalized Mixtures to Rule Continua

Parameter-coupled state space models based on quasi-Gaussian fuzzy approximation

The accuracy of a fuzzy system's approximation is closely tied to the performance of fuzzy control systems design, while this system's interpretability depends on the description of a mechanical model using human language. This research introduces a quasi-Gaussian membership function characterized by a pair of parameters to achieve the sensitivity of a triangular membership function along with the interpretability of Gaussian membership functions. Consequently, a two-dimensional (2-D) quasi-Gaussian membership function is derived, and a method for establishing quasi-Gaussian fuzzy systems (QGFS) using a rectangular grid is proposed. After validating the approximation properties using the sine function for the one-dimensional (1-D) and 2-D QGFS, the systems are applied to approximate the depyrogenation tunnel, a significant piece of equipment in the pharmaceutical industry with various mechanical designs. Validation results indicate that the 1-D and 2-D QGFS can achieve an approximation error varying within a ± 5% range. Meanwhile, the 1-D and 2-D QGFSs are applied to mechanical models of the depyrogenation tunnel with satisfactory final approximation results. Lastly, the 2-D QGFS is capable of demonstrating an excellent description of models with coupled parameters.

Noise can speed backpropagation learning and deep bidirectional pretraining

We show that the backpropagation algorithm is a special case of the generalized Expectation-Maximization (EM) algorithm for iterative maximum likelihood estimation. We then apply the recent result that carefully chosen noise can speed the average convergence of the EM algorithm as it climbs a hill of probability or log-likelihood. Then injecting such noise can speed the average convergence of the backpropagation algorithm for both the training and pretraining of multilayer neural networks. The beneficial noise adds to the hidden and visible neurons and related parameters. The noise also applies to regularized regression networks. This beneficial noise is just that noise that makes the current signal more probable. We show that such noise also tends to improve classification accuracy. The geometry of the noise-benefit region depends on the probability structure of the neurons in a given layer. The noise-benefit region in noise space lies above the noisy-EM (NEM) hyperplane for classification and involves a hypersphere for regression. Simulations demonstrate these noise benefits using MNIST digit classification. The NEM noise benefits substantially exceed those of simply adding blind noise to the neural network. We further prove that the noise speed-up applies to the deep bidirectional pretraining of neural-network bidirectional associative memories (BAMs) or their functionally equivalent restricted Boltzmann machines. We then show that learning with basic contrastive divergence also reduces to generalized EM for an energy-based network probability. The optimal noise adds to the input visible neurons of a BAM in stacked layers of trained BAMs. Global stability of generalized BAMs guarantees rapid convergence in pretraining where neural signals feed back between contiguous layers. Bipolar coding of inputs further improves pretraining performance.