Spiking pattern recognition using informative signal of image and unsupervised biologically plausible learning

Digital design of a spatial-pow-STDP learning block with high accuracy utilizing pow CORDIC for large-scale image classifier spatiotemporal SNN

The paramount concern of highly accurate energy-efficient computing in machines with significant cognitive capabilities aims to enhance the accuracy and efficiency of bio-inspired Spiking Neural Networks (SNNs). This paper addresses this main objective by introducing a novel spatial power spike-timing-dependent plasticity (Spatial-Pow-STDP) learning rule as a digital block with high accuracy in a bio-inspired SNN model. Motivated by the demand for precise and accelerated computation that reduces high-cost resources in neural network applications, this paper presents a methodology based on COordinate Rotation DIgital Computer (CORDIC) definitions. The proposed designs of CORDIC algorithms for exponential (Exp CORDIC), natural logarithm (Ln CORDIC), and arbitrary power function (Pow CORDIC) are meticulously detailed and evaluated to ensure optimal acceleration and accuracy, which respectively show average errors near 10-9, 10-6, and 10-5 with 4, 4, and 6 iterations. The engineered architectures for the Exp, Ln, and Pow CORDIC implementations are illustrated and assessed, showcasing the efficiency achieved through high frequency, leading to the introduction of a Spatial-Pow-STDP learning block design based on Pow CORDIC that facilitates efficient and accurate hardware computation with 6.93 × 10-3 average error with 9 iterations. The proposed learning mechanism integrates this structure into a large-scale spatiotemporal SNN consisting of three layers with reduced hyper-parameters, enabling unsupervised training in an event-based paradigm using excitatory and inhibitory synapses. As a result, the application of the developed methodology and equations in the computational SNN model for image classification reveals superior accuracy and convergence speed compared to existing spiking networks by achieving up to 97.5%, 97.6%, 93.4%, and 93% accuracy, respectively, when trained on the MNIST, EMNIST digits, EMNIST letters, and CIFAR10 datasets with 6, 2, 2, and 6 training epochs.

Spiking image processing unit based on neural analog of Boolean logic operations

McCulloch and Pitts hypothesized in 1943 that the brain is entirely composed of logic gates, akin to current computers' IP cores, which led to several neural analogs of Boolean logic. The current study proposes a spiking image processing unit (SIPU) based on spiking frequency gates and coordinate logic operations, as a dynamical model of synapses and spiking neurons. SIPU can imitate DSP functions like edge recognition, picture magnification, noise reduction, etc. but can be extended to cater for more advanced computing tasks. The proposed spiking Boolean logic platform can be used to develop advanced applications without relying on learning or specialized datasets. It could aid in gaining a deeper understanding of complex brain functions and spur new forms of neural analogs.

Recognizing intertwined patterns using a network of spiking pattern recognition platforms

Artificial intelligence computing adapted from biology is a suitable platform for the development of intelligent machines by imitating the functional mechanisms of the nervous system in creating high-level activities such as learning, decision making and cognition in today's systems. Here, the concentration is on improvement the cognitive potential of artificial intelligence network with a bio-inspired structure. In this regard, four spiking pattern recognition platforms for recognizing digits and letters of EMNIST, patterns of YALE, and ORL datasets are proposed. All networks are developed based on a similar structure in the input image coding, model of neurons (pyramidal neurons and interneurons) and synapses (excitatory AMPA and inhibitory GABA currents), and learning procedure. Networks 1-4 are trained on Digits, Letters, faces of YALE and ORL, respectively, with the proposed un-supervised, spatial-temporal, and sparse spike-based learning mechanism based on the biological observation of the brain learning. When the networks have reached the highest recognition accuracy in the relevant patterns, the main goal of the article, which is to achieve high-performance pattern recognition system with higher cognitive ability, is followed. The pattern recognition network that is able to detect the combination of multiple patterns which called intertwined patterns has not been discussed yet. Therefore, by integrating four trained spiking pattern recognition platforms in one system configuration, we are able to recognize intertwined patterns. These results are presented for the first time and could be the pioneer of a new generation of pattern recognition networks with a significant ability in smart machines.

A Parallel Spiking Neural Network Based on Adaptive Lateral Inhibition Mechanism for Objective Recognition

Spiking neural network (SNN) has attracted extensive attention in the field of machine learning because of its biological interpretability and low power consumption. However, the accuracy of pattern recognition cannot completely surpass deep neural networks (DNNs). The main reason is that the inherent nondifferentiability of spiking neurons makes SNN unable to be trained directly by the gradient descent algorithm, and there is also no unified training algorithm for SNN. Inspired by the biological vision system, this paper proposes a parallel convolution SNN structure combined with an adaptive lateral inhibition mechanism. And, a way of dynamically evolving the time constant with the training of SNN is proposed to ensure the diversity of neurons. This paper verifies the effectiveness of the proposed methods on static datasets and neuromorphic datasets and extends it to the recognition of breast tumors. Experimental results show that the SNN has obvious advantages in dynamical datasets. For breast tumors, it is also an edge-based task, because the edge of a medical image contains the most important information in the image. This kind of information can provide great help for the noninvasive and accurate diagnosis of diseases. The Experimental results show that the proposed method is very close to the recognition results of DNNs on static datasets, and its performance on neuromorphic datasets exceeds that of DNNs.

Linear leaky-integrate-and-fire neuron model based spiking neural networks and its mapping relationship to deep neural networks

Spiking neural networks (SNNs) are brain-inspired machine learning algorithms with merits such as biological plausibility and unsupervised learning capability. Previous works have shown that converting Artificial Neural Networks (ANNs) into SNNs is a practical and efficient approach for implementing an SNN. However, the basic principle and theoretical groundwork are lacking for training a non-accuracy-loss SNN. This paper establishes a precise mathematical mapping between the biological parameters of the Linear Leaky-Integrate-and-Fire model (LIF)/SNNs and the parameters of ReLU-AN/Deep Neural Networks (DNNs). Such mapping relationship is analytically proven under certain conditions and demonstrated by simulation and real data experiments. It can serve as the theoretical basis for the potential combination of the respective merits of the two categories of neural networks.

A Layered Spiking Neural System for Classification Problems

Biological brains have a natural capacity for resolving certain classification tasks. Studies on biologically plausible spiking neurons, architectures and mechanisms of artificial neural systems that closely match biological observations while giving high classification performance are gaining momentum. Spiking neural P systems (SN P systems) are a class of membrane computing models and third-generation neural networks that are based on the behavior of biological neural cells and have been used in various engineering applications. Furthermore, SN P systems are characterized by a highly flexible structure that enables the design of a machine learning algorithm by mimicking the structure and behavior of biological cells without the over-simplification present in neural networks. Based on this aspect, this paper proposes a novel type of SN P system, namely, layered SN P system (LSN P system), to solve classification problems by supervised learning. The proposed LSN P system consists of a multi-layer network containing multiple weighted fuzzy SN P systems with adaptive weight adjustment rules. The proposed system employs specific ascending dimension techniques and a selection method of output neurons for classification problems. The experimental results obtained using benchmark datasets from the UCI machine learning repository and MNIST dataset demonstrated the feasibility and effectiveness of the proposed LSN P system. More importantly, the proposed LSN P system presents the first SN P system that demonstrates sufficient performance for use in addressing real-world classification problems.

nMNSD-A Spiking Neuron-Based Classifier That Combines Weight-Adjustment and Delay-Shift

The recent “multi-neuronal spike sequence detector” (MNSD) architecture integrates the weight- and delay-adjustment methods by combining heterosynaptic plasticity with the neurocomputational feature spike latency, representing a new opportunity to understand the mechanisms underlying biological learning. Unfortunately, the range of problems to which this topology can be applied is limited because of the low cardinality of the parallel spike trains that it can process, and the lack of a visualization mechanism to understand its internal operation. We present here the nMNSD structure, which is a generalization of the MNSD to any number of inputs. The mathematical framework of the structure is introduced, together with the “trapezoid method,” that is a reduced method to analyze the recognition mechanism operated by the nMNSD in response to a specific input parallel spike train. We apply the nMNSD to a classification problem previously faced with the classical MNSD from the same authors, showing the new possibilities the nMNSD opens, with associated improvement in classification performances. Finally, we benchmark the nMNSD on the classification of static inputs (MNIST database) obtaining state-of-the-art accuracies together with advantageous aspects in terms of time- and energy-efficiency if compared to similar classification methods.

Design and Implementation of a Spiking Neural Network with Integrate-and-Fire Neuron Model for Pattern Recognition

In contrast to the previous artificial neural networks (ANNs), spiking neural networks (SNNs) work based on temporal coding approaches. In the proposed SNN, the number of neurons, neuron models, encoding method, and learning algorithm design are described in a correct and pellucid fashion. It is also discussed that optimizing the SNN parameters based on physiology, and maximizing the information they pass leads to a more robust network. In this paper, inspired by the "center-surround" structure of the receptive fields in the retina, and the amount of overlap that they have, a robust SNN is implemented. It is based on the Integrate-and-Fire (IF) neuron model and uses the time-to-first-spike coding to train the network by a newly proposed method. The Iris and MNIST datasets were employed to evaluate the performance of the proposed network whose accuracy, with 60 input neurons, was 96.33% on the Iris dataset. The network was trained in only 45 iterations indicating its reasonable convergence rate. For the MNIST dataset, when the gray level of each pixel was considered as input to the network, 600 input neurons were required, and the accuracy of the network was 90.5%. Next, 14 structural features were used as input. Therefore, the number of input neurons decreased to 210, and accuracy increased up to 95%, meaning that an SNN with fewer input neurons and good skill was implemented. Also, the ABIDE1 dataset is applied to the proposed SNN. Of the 184 data, 79 are used for healthy people and 105 for people with autism. One of the characteristics that can differentiate between these two classes is the entropy of the existing data. Therefore, Shannon entropy is used for feature extraction. Applying these values to the proposed SNN, an accuracy of 84.42% was achieved by only 120 iterations, which is a good result compared to the recent results.

Information Transmitted From Bioinspired Neuron-Astrocyte Network Improves Cortical Spiking Network's Pattern Recognition Performance

We trained two spiking neural networks (SNNs), the cortical spiking network (CSN) and the cortical neuron-astrocyte network (CNAN), using a spike-based unsupervised method, on the MNIST and alpha-digit data sets and achieve an accuracy of 96.1% and 77.35%, respectively. We then connected CNAN to CSN by preserving maximum synchronization between them thanks to the concept of prolate spheroidal wave functions (PSWF). As a result, CSN receives additional information from CNAN without retraining. The important outcome is that CSN reaches 70.57% correct classification rate on capital letters without being trained on them. The overall contribution of transfer is 87.47%. We observed that for CSN the classifying neurons that relate to digits 0-9 of the alpha-digit data set are completely supported by the ones that relate to digits 0-9 of the MNIST data set. This means that CSN recognizes the similarity between the digits of the MNIST and alpha-digit data sets and classifies each digit of both data sets in the same class.