An Energy-Efficient Bayesian Neural Network Implementation Using Stochastic Computing Method

On the Robustness of Bayesian Neural Networks to Adversarial Attacks

Vulnerability to adversarial attacks is one of the principal hurdles to the adoption of deep learning in safety-critical applications. Despite significant efforts, both practical and theoretical, training deep learning models robust to adversarial attacks is still an open problem. In this article, we analyse the geometry of adversarial attacks in the over-parameterized limit for Bayesian neural networks (BNNs). We show that, in the limit, vulnerability to gradient-based attacks arises as a result of degeneracy in the data distribution, i.e., when the data lie on a lower dimensional submanifold of the ambient space. As a direct consequence, we demonstrate that in this limit, BNN posteriors are robust to gradient-based adversarial attacks. Crucially, by relying on the convergence of infinitely-wide BNNs to Gaussian processes (GPs), we prove that, under certain relatively mild assumptions, the expected gradient of the loss with respect to the BNN posterior distribution is vanishing, even when each NN sampled from the BNN posterior does not have vanishing gradients. The experimental results on the MNIST, Fashion MNIST, and a synthetic dataset with BNNs trained with Hamiltonian Monte Carlo and variational inference support this line of arguments, empirically showing that BNNs can display both high accuracy on clean data and robustness to both gradient-based and gradient-free adversarial attacks.

Delicately Reinforced k-Nearest Neighbor Classifier Combined With Expert Knowledge Applied to Abnormity Forecast in Electrolytic Cell

As the profit and safety requirements become higher and higher, it is more and more necessary to realize an advanced intelligent analysis for abnormity forecast of the synthetical balance of material and energy (AF-SBME) on aluminum reduction cells (ARCs). Without loss of generality, AF-SBME belongs to classification problems. Its advanced intelligent analysis can be realized by high-performance data-driven classifiers. However, AF-SBME has some difficulties, including a high requirement for interpretability of data-driven classifiers, a small number, and decreasing-over-time correctness of training samples. In this article, based on a preferable data-driven classifier, which is called a reinforced k -nearest neighbor (R-KNN) classifier, a delicately R-KNN combined with expert knowledge (DR-KNN/CE) is proposed. It improves R-KNN in two ways, including using expert knowledge as external assistance and enhancing self-ability to mine and synthesize data knowledge. The related experiments on AF-SBME, where the relevant data are directly sampled from practical production, have demonstrated that the proposed DR-KNN/CE not only makes an effective improvement for R-KNN, but also has a more advanced performance compared with other existing high-performance data-driven classifiers.