Introspective analysis of convolutional neural networks for improving discrimination performance and feature visualisation

CDTNet: Improved Image Classification Method Using Standard, Dilated and Transposed Convolutions

Convolutional neural networks (CNNs) have achieved great success in image classification tasks. In the process of a convolutional operation, a larger input area can capture more context information. Stacking several convolutional layers can enlarge the receptive field, but this increases the parameters. Most CNN models use pooling layers to extract important features, but the pooling operations cause information loss. Transposed convolution can increase the spatial size of the feature maps to recover the lost low-resolution information. In this study, we used two branches with different dilated rates to obtain different size features. The dilated convolution can capture richer information, and the outputs from the two channels are concatenated together as input for the next block. The small size feature maps of the top blocks are transposed to increase the spatial size of the feature maps to recover low-resolution prediction maps. We evaluated the model on three image classification benchmark datasets (CIFAR-10, SVHN, and FMNIST) with four state-of-the-art models, namely, VGG16, VGG19, ResNeXt, and DenseNet. The experimental results show that CDTNet achieved lower loss, higher accuracy, and faster convergence speed in the training and test stages. The average test accuracy of CDTNet increased by 54.81% at most on SVHN with VGG19 and by 1.28% at least on FMNIST with VGG16, which proves that CDTNet has better performance and strong generalization abilities, as well as fewer parameters.

A Novel Evaluation Strategy to Artificial Neural Network Model Based on Bionics

With the continuous deepening of Artificial Neural Network (ANN) research, ANN model structure and function are improving towards diversification and intelligence. However, the model is more evaluated from the pros and cons of the problem-solving results and the lack of evaluation from the biomimetic aspect of imitating neural networks is not inclusive enough. Hence, a new ANN models evaluation strategy is proposed from the perspective of bionics in response to this problem in the paper. Firstly, four classical neural network models are illustrated: Back Propagation (BP) network, Deep Belief Network (DBN), LeNet5 network, and olfactory bionic model (KIII model), and the neuron transmission mode and equation, network structure, and weight updating principle of the models are analyzed qualitatively. The analysis results show that the KIII model comes closer to the actual biological nervous system compared with other models, and the LeNet5 network simulates the nervous system in depth. Secondly, evaluation indexes of ANN are constructed from the perspective of bionics in this paper: small-world, synchronous, and chaotic characteristics. Finally, the network model is quantitatively analyzed by evaluation indexes from the perspective of bionics. The experimental results show that the DBN network, LeNet5 network, and BP network have synchronous characteristics. And the DBN network and LeNet5 network have certain chaotic characteristics, but there is still a certain distance between the three classical neural networks and actual biological neural networks. The KIII model has certain small-world characteristics in structure, and its network also exhibits synchronization characteristics and chaotic characteristics. Compared with the DBN network, LeNet5 network, and the BP network, the KIII model is closer to the real biological neural network.