Architectural Parameter-Independent Network Initialization Scheme for Sigmoidal Feedforward ANNs

An analysis of weight initialization methods in connection with different activation functions for feedforward neural networks

The selection of weight initialization in an artificial neural network is one of the key aspects and affects the learning speed, convergence rate and correctness of classification by an artificial neural network. In this paper, we investigate the effects of weight initialization in an artificial neural network. Nguyen-Widrow weight initialization, random initialization, and Xavier initialization method are paired with five different activation functions. This paper deals with a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer. The paired combination of weight initialization methods with activation functions are examined and tested and compared based on their best achieved loss rate in training. This work aims to better understand how weight initialization methods in neural networks, in combination with activation functions, affect the learning speed in comparison after a fixed number of training epochs.

Evaluation of Ideological and Political Education under Deep Learning Neural Network

Under the background of the rapid development of the new generation of information technology, artificial neural networks (ANNs) have made some progress in the evaluation research of various courses in colleges and universities. However, there is little research on the application of ANNs in Ideological and Political Education (IPE) courses. Based on this, this work attempts to introduce the Backpropagation Neural Network (BPNN) into the evaluation system of IPE courses. Firstly, the structure and characteristics of the BPNN are given, and it is optimized by the genetic algorithm based on its characteristics. Secondly, the theoretical framework of IPE course evaluation is established, and the corresponding evaluation model is constructed using the optimized BPNN. Finally, a questionnaire survey is designed to analyze the current situation of IPE evaluation in colleges and universities, and a simulation experiment is set up to test the BPNN evaluation model before and after optimization. The results are as follows. First, there are mainly five evaluation methods for IPE courses in Chinese colleges and universities: 6/4, 3/7, 2/2/6, 5/5, and 3/3/4. Second, the training error value of the BPNN model is in the interval (-2.3, 2.2), and when the number of cycles is 552, the error is infinitely close to zero. Thirdly, the training error value of the optimized BPNN model is in the interval (-0.22, 1.2), and when the number of cycles is 775, the error is infinitely close to zero. Fourthly, the error between the output value of the optimized BPNN model and the expert score value is generally smaller than the error value before optimization. This work aims to provide a theoretical reference for the further application of neural network technology in the evaluation of IPE.