The correspondence between deterministic and stochastic digital neurons: analysis and methodology

Cost-effective stochastic MAC circuits for deep neural networks

Stochastic computing (SC) is a promising computing paradigm that can help address both the uncertainties of future process technology and the challenges of efficient hardware realization for deep neural networks (DNNs). However the impreciseness and long latency of SC have rendered previous SC-based DNN architectures less competitive against optimized fixed-point digital implementations, unless inference accuracy is significantly sacrificed. In this paper we propose a new SC-MAC (multiply-and-accumulate) algorithm, which is a key building block for SC-based DNNs, that is orders of magnitude more efficient and accurate than previous SC-MACs. We also show how our new SC-MAC can be extended to a vector version and used to accelerate both convolution and fully-connected layers of convolutional neural networks (CNNs) using the same hardware. Our experimental results using CNNs designed for MNIST and CIFAR-10 datasets demonstrate that not only is our SC-based CNNs more accurate and 40∼490× more energy-efficient for convolution layers than conventional SC-based ones, but ours can also achieve lower area-delay product and lower energy compared with precision-optimized fixed-point implementations without sacrificing accuracy. We also demonstrate the feasibility of our SC-based CNNs through FPGA prototypes.

Complete Neuron-Astrocyte Interaction Model: Digital Multiplierless Design and Networking Mechanism

Glial cells, also known as neuroglia or glia, are non-neuronal cells providing support and protection for neurons in the central nervous system (CNS). They also act as supportive cells in the brain. Among a variety of glial cells, the star-shaped glial cells, i.e., astrocytes, are the largest cell population in the brain. The important role of astrocyte such as neuronal synchronization, synaptic information regulation, feedback to neural activity and extracellular regulation make the astrocytes play a vital role in brain disease. This paper presents a modified complete neuron-astrocyte interaction model that is more suitable for efficient and large scale biological neural network realization on digital platforms. Simulation results show that the modified complete interaction model can reproduce biological-like behavior of the original neuron-astrocyte mechanism. The modified interaction model is investigated in terms of digital realization feasibility and cost targeting a low cost hardware implementation. Networking behavior of this interaction is investigated and compared between two cases: i) the neuron spiking mechanism without astrocyte effects, and ii) the effect of astrocyte in regulating the neurons behavior and synaptic transmission via controlling the LTP and LTD processes. Hardware implementation on FPGA shows that the modified model mimics the main mechanism of neuron-astrocyte communication with higher performance and considerably lower hardware overhead cost compared with the original interaction model.

A Digital Realization of Astrocyte and Neural Glial Interactions

The implementation of biological neural networks is a key objective of the neuromorphic research field. Astrocytes are the largest cell population in the brain. With the discovery of calcium wave propagation through astrocyte networks, now it is more evident that neuronal networks alone may not explain functionality of the strongest natural computer, the brain. Models of cortical function must now account for astrocyte activities as well as their relationships with neurons in encoding and manipulation of sensory information. From an engineering viewpoint, astrocytes provide feedback to both presynaptic and postsynaptic neurons to regulate their signaling behaviors. This paper presents a modified neural glial interaction model that allows a convenient digital implementation. This model can reproduce relevant biological astrocyte behaviors, which provide appropriate feedback control in regulating neuronal activities in the central nervous system (CNS). Accordingly, we investigate the feasibility of a digital implementation for a single astrocyte constructed by connecting a two coupled FitzHugh Nagumo (FHN) neuron model to an implementation of the proposed astrocyte model using neuron-astrocyte interactions. Hardware synthesis, physical implementation on FPGA, and theoretical analysis confirm that the proposed neuron astrocyte model, with significantly low hardware cost, can mimic biological behavior such as the regulation of postsynaptic neuron activity and the synaptic transmission mechanisms.

Digital implementation of a biological astrocyte model and its application

This paper presents a modified astrocyte model that allows a convenient digital implementation. This model is aimed at reproducing relevant biological astrocyte behaviors, which provide appropriate feedback control in regulating neuronal activities in the central nervous system. Accordingly, we investigate the feasibility of a digital implementation for a single astrocyte and a biological neuronal network model constructed by connecting two limit-cycle Hopf oscillators to an implementation of the proposed astrocyte model using oscillator-astrocyte interactions with weak coupling. Hardware synthesis, physical implementation on field-programmable gate array, and theoretical analysis confirm that the proposed astrocyte model, with considerably low hardware overhead, can mimic biological astrocyte model behaviors, resulting in desynchronization of the two coupled limit-cycle oscillators.