Noise-enhanced clustering and competitive learning algorithms

On the Post Hoc Explainability of Optimized Self-Organizing Reservoir Network for Action Recognition

This work proposes a novel unsupervised self-organizing network, called the Self-Organizing Convolutional Echo State Network (SO-ConvESN), for learning node centroids and interconnectivity maps compatible with the deterministic initialization of Echo State Network (ESN) input and reservoir weights, in the context of human action recognition (HAR). To ensure stability and echo state property in the reservoir, Recurrent Plots (RPs) and Recurrence Quantification Analysis (RQA) techniques are exploited for explainability and characterization of the reservoir dynamics and hence tuning ESN hyperparameters. The optimized self-organizing reservoirs are cascaded with a Convolutional Neural Network (CNN) to ensure that the activation of internal echo state representations (ESRs) echoes similar topological qualities and temporal features of the input time-series, and the CNN efficiently learns the dynamics and multiscale temporal features from the ESRs for action recognition. The hyperparameter optimization (HPO) algorithms are additionally adopted to optimize the CNN stage in SO-ConvESN. Experimental results on the HAR problem using several publicly available 3D-skeleton-based action datasets demonstrate the showcasing of the RPs and RQA technique in examining the explainability of reservoir dynamics for designing stable self-organizing reservoirs and the usefulness of implementing HPOs in SO-ConvESN for the HAR task. The proposed SO-ConvESN exhibits competitive recognition accuracy.

Noise can speed Markov chain Monte Carlo estimation and quantum annealing

Carefully injected noise can speed the average convergence of Markov chain Monte Carlo (MCMC) estimates and simulated annealing optimization. This includes quantum annealing and the MCMC special case of the Metropolis-Hastings algorithm. MCMC seeks the solution to a computational problem as the equilibrium probability density of a reversible Markov chain. The algorithm must cycle through a long burn-in phase until it reaches equilibrium because the Markov samples are statistically correlated. The special injected noise reduces this burn-in period in MCMC. A related theorem shows that it reduces the cooling time in simulated annealing. Simulations showed that optimal noise gave a 76% speed-up in finding the global minimum in the Schwefel optimization benchmark. The noise-boosted simulations found the global minimum in 99.8% of trials compared with only 95.4% of trials in noiseless simulated annealing. Simulations also showed that the noise boost is robust to accelerated cooling schedules and that noise decreased convergence times by more than 32% under aggressive geometric cooling. Molecular dynamics simulations showed that optimal noise gave a 42% speed-up in finding the minimum potential energy configuration of an eight-argon-atom gas system with a Lennard-Jones 12-6 potential. The annealing speed-up also extends to quantum Monte Carlo implementations of quantum annealing. Noise improved ground-state energy estimates in a 1024-spin simulated quantum annealing simulation by 25.6%. The quantum noise flips spins along a Trotter ring. The noisy MCMC algorithm brings each Markov step closer on average to equilibrium if an inequality holds between two expectations. Gaussian or Cauchy jump probabilities reduce the noise-benefit inequality to a simple quadratic inequality. Simulations show that noise-boosted simulated annealing is more likely than noiseless annealing to sample high probability regions of the search space and to accept solutions that increase the search breadth.

A Regularizer Approach for RBF Networks Under the Concurrent Weight Failure Situation

Many existing results on fault-tolerant algorithms focus on the single fault source situation, where a trained network is affected by one kind of weight failure. In fact, a trained network may be affected by multiple kinds of weight failure. This paper first studies how the open weight fault and the multiplicative weight noise degrade the performance of radial basis function (RBF) networks. Afterward, we define the objective function for training fault-tolerant RBF networks. Based on the objective function, we then develop two learning algorithms, one batch mode and one online mode. Besides, the convergent conditions of our online algorithm are investigated. Finally, we develop a formula to estimate the test set error of faulty networks trained from our approach. This formula helps us to optimize some tuning parameters, such as RBF width.