A library of analog operators based on the hodgkin-huxley formalism for the design of tunable, real-time, silicon neurons

Two-Dimensional Materials for Brain-Inspired Computing Hardware

Recent breakthroughs in brain-inspired computing promise to address a wide range of problems from security to healthcare. However, the current strategy of implementing artificial intelligence algorithms using conventional silicon hardware is leading to unsustainable energy consumption. Neuromorphic hardware based on electronic devices mimicking biological systems is emerging as a low-energy alternative, although further progress requires materials that can mimic biological function while maintaining scalability and speed. As a result of their diverse unique properties, atomically thin two-dimensional (2D) materials are promising building blocks for next-generation electronics including nonvolatile memory, in-memory and neuromorphic computing, and flexible edge-computing systems. Furthermore, 2D materials achieve biorealistic synaptic and neuronal responses that extend beyond conventional logic and memory systems. Here, we provide a comprehensive review of the growth, fabrication, and integration of 2D materials and van der Waals heterojunctions for neuromorphic electronic and optoelectronic devices, circuits, and systems. For each case, the relationship between physical properties and device responses is emphasized followed by a critical comparison of technologies for different applications. We conclude with a forward-looking perspective on the key remaining challenges and opportunities for neuromorphic applications that leverage the fundamental properties of 2D materials and heterojunctions.

Memristive Hodgkin-Huxley Spiking Neuron Model for Reproducing Neuron Behaviors

The Hodgkin-Huxley (HH) spiking neuron model reproduces the dynamic characteristics of the neuron by mimicking the action potential, ionic channels, and spiking behaviors. The memristor is a nonlinear device with variable resistance. In this paper, the memristor is introduced to the HH spiking model, and the memristive Hodgkin-Huxley spiking neuron model (MHH) is presented. We experimentally compare the HH spiking model and the MHH spiking model by applying different stimuli. First, the individual current pulse is injected into the HH and MHH spiking models. The comparison between action potentials, current densities, and conductances is carried out. Second, the reverse single pulse stimulus and a series of pulse stimuli are applied to the two models. The effects of current density and action time on the production of the action potential are analyzed. Finally, the sinusoidal current stimulus acts on the two models. The various spiking behaviors are realized by adjusting the frequency of the sinusoidal stimulus. We experimentally demonstrate that the MHH spiking model generates more action potential than the HH spiking model and takes a short time to change the memductance. The reverse stimulus cannot activate the action potential in both models. The MHH spiking model performs smoother waveforms and a faster speed to return to the resting potential. The larger the external stimulus, the faster action potential generated, and the more noticeable change in conductances. Meanwhile, the MHH spiking model shows the various spiking patterns of neurons.

Biologically Relevant Dynamical Behaviors Realized in an Ultra-Compact Neuron Model

We demonstrate a variety of biologically relevant dynamical behaviors building on a recently introduced ultra-compact neuron (UCN) model. We provide the detailed circuits which all share a common basic block that realizes the leaky-integrate-and-fire (LIF) spiking behavior. All circuits have a small number of active components and the basic block has only three, two transistors and a silicon controlled rectifier (SCR). We also demonstrate that numerical simulations can faithfully represent the variety of spiking behavior and can be used for further exploration of dynamical behaviors. Taking Izhikevich’s set of biologically relevant behaviors as a reference, our work demonstrates that a circuit of a LIF neuron model can be used as a basis to implement a large variety of relevant spiking patterns. These behaviors may be useful to construct neural networks that can capture complex brain dynamics or may also be useful for artificial intelligence applications. Our UCN model can therefore be considered the electronic circuit counterpart of Izhikevich’s (2003) mathematical neuron model, sharing its two seemingly contradicting features, extreme simplicity and rich dynamical behavior.

A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores

Here, we describe a multicompartment neuron circuit based on the adaptive-exponential I&F (AdEx) model, developed for the second-generation BrainScaleS hardware. Based on an existing modular leaky integrate-and-fire (LIF) architecture designed in 65-nm CMOS, the circuit features exponential spike generation, neuronal adaptation, intercompartmental connections as well as a conductance-based reset. The design reproduces a diverse set of firing patterns observed in cortical pyramidal neurons. Further, it enables the emulation of sodium and calcium spikes, as well as N-methyl-D-aspartate plateau potentials known from apical and thin dendrites. We characterize the AdEx circuit extensions and exemplify how the interplay between passive and nonlinear active signal processing enhances the computational capabilities of single (but structured) on-chip neurons.

Hodgkin-Huxley Neuron and FPAA Dynamics

We present the experimental silicon results on the dynamics of a Hodgkin-Huxley neuron implemented on a reconfigurable platform. The circuit has been inspired by the similarity between biology and silicon, by modeling ion channels and their time constants. Another significant motivation behind this paper is to make the system available to circuit designers as well as users in the neuroscience community. The open-source tool infrastructure and a remote system ease the accessibility of our system to a number of users. We demonstrate the reproducibility of the results by replicating the dynamics across different boards along with responses from different inputs and with different parameters. The reconfigurability enables one to make use of a single primary design to obtain a variety of results. The measurements are taken from the system compiled on a field programmable analog array fabricated on a 350-nm process.

Scalable excitatory synaptic circuit design using floating gate based leaky integrators

We propose a scalable synaptic circuit realizing spike timing dependent plasticity (STDP)-compatible with randomly spiking neurons. The feasible working of the circuit was examined by circuit simulation using the BSIM 4.6.0 model. A distinguishable feature of the circuit is the use of floating-gate integrators that provide the compact implementation of biologically plausible relaxation time scale. This relaxation occurs on the basis of charge tunneling that mainly relies upon area-independent tunnel barrier properties (e.g. barrier width and height) rather than capacitance. The circuit simulations feature (i) weight-dependent STDP that spontaneously limits the synaptic weight growth, (ii) competitive synaptic adaptation within both unsupervised and supervised frameworks with randomly spiking neurons. The estimated power consumption is merely 34 pW, perhaps meeting one of the most crucial principles (power-efficiency) of neuromorphic engineering. Finally, a means of fine-tuning the STDP behavior is provided.

Neuromorphic Implementation of Attractor Dynamics in a Two-Variable Winner-Take-All Circuit with NMDARs: A Simulation Study

Neural networks configured with winner-take-all (WTA) competition and N-methyl-D-aspartate receptor (NMDAR)-mediated synaptic dynamics are endowed with various dynamic characteristics of attractors underlying many cognitive functions. This paper presents a novel method for neuromorphic implementation of a two-variable WTA circuit with NMDARs aimed at implementing decision-making, working memory and hysteresis in visual perceptions. The method proposed is a dynamical system approach of circuit synthesis based on a biophysically plausible WTA model. Notably, slow and non-linear temporal dynamics of NMDAR-mediated synapses was generated. Circuit simulations in Cadence reproduced ramping neural activities observed in electrophysiological recordings in experiments of decision-making, the sustained activities observed in the prefrontal cortex during working memory, and classical hysteresis behavior during visual discrimination tasks. Furthermore, theoretical analysis of the dynamical system approach illuminated the underlying mechanisms of decision-making, memory capacity and hysteresis loops. The consistence between the circuit simulations and theoretical analysis demonstrated that the WTA circuit with NMDARs was able to capture the attractor dynamics underlying these cognitive functions. Their physical implementations as elementary modules are promising for assembly into integrated neuromorphic cognitive systems.

Complete Neuron-Astrocyte Interaction Model: Digital Multiplierless Design and Networking Mechanism

Glial cells, also known as neuroglia or glia, are non-neuronal cells providing support and protection for neurons in the central nervous system (CNS). They also act as supportive cells in the brain. Among a variety of glial cells, the star-shaped glial cells, i.e., astrocytes, are the largest cell population in the brain. The important role of astrocyte such as neuronal synchronization, synaptic information regulation, feedback to neural activity and extracellular regulation make the astrocytes play a vital role in brain disease. This paper presents a modified complete neuron-astrocyte interaction model that is more suitable for efficient and large scale biological neural network realization on digital platforms. Simulation results show that the modified complete interaction model can reproduce biological-like behavior of the original neuron-astrocyte mechanism. The modified interaction model is investigated in terms of digital realization feasibility and cost targeting a low cost hardware implementation. Networking behavior of this interaction is investigated and compared between two cases: i) the neuron spiking mechanism without astrocyte effects, and ii) the effect of astrocyte in regulating the neurons behavior and synaptic transmission via controlling the LTP and LTD processes. Hardware implementation on FPGA shows that the modified model mimics the main mechanism of neuron-astrocyte communication with higher performance and considerably lower hardware overhead cost compared with the original interaction model.

Qualitative-Modeling-Based Silicon Neurons and Their Networks

The ionic conductance models of neuronal cells can finely reproduce a wide variety of complex neuronal activities. However, the complexity of these models has prompted the development of qualitative neuron models. They are described by differential equations with a reduced number of variables and their low-dimensional polynomials, which retain the core mathematical structures. Such simple models form the foundation of a bottom-up approach in computational and theoretical neuroscience. We proposed a qualitative-modeling-based approach for designing silicon neuron circuits, in which the mathematical structures in the polynomial-based qualitative models are reproduced by differential equations with silicon-native expressions. This approach can realize low-power-consuming circuits that can be configured to realize various classes of neuronal cells. In this article, our qualitative-modeling-based silicon neuron circuits for analog and digital implementations are quickly reviewed. One of our CMOS analog silicon neuron circuits can realize a variety of neuronal activities with a power consumption less than 72 nW. The square-wave bursting mode of this circuit is explained. Another circuit can realize Class I and II neuronal activities with about 3 nW. Our digital silicon neuron circuit can also realize these classes. An auto-associative memory realized on an all-to-all connected network of these silicon neurons is also reviewed, in which the neuron class plays important roles in its performance.

A Digital Realization of Astrocyte and Neural Glial Interactions

The implementation of biological neural networks is a key objective of the neuromorphic research field. Astrocytes are the largest cell population in the brain. With the discovery of calcium wave propagation through astrocyte networks, now it is more evident that neuronal networks alone may not explain functionality of the strongest natural computer, the brain. Models of cortical function must now account for astrocyte activities as well as their relationships with neurons in encoding and manipulation of sensory information. From an engineering viewpoint, astrocytes provide feedback to both presynaptic and postsynaptic neurons to regulate their signaling behaviors. This paper presents a modified neural glial interaction model that allows a convenient digital implementation. This model can reproduce relevant biological astrocyte behaviors, which provide appropriate feedback control in regulating neuronal activities in the central nervous system (CNS). Accordingly, we investigate the feasibility of a digital implementation for a single astrocyte constructed by connecting a two coupled FitzHugh Nagumo (FHN) neuron model to an implementation of the proposed astrocyte model using neuron-astrocyte interactions. Hardware synthesis, physical implementation on FPGA, and theoretical analysis confirm that the proposed neuron astrocyte model, with significantly low hardware cost, can mimic biological behavior such as the regulation of postsynaptic neuron activity and the synaptic transmission mechanisms.

An astrocyte neuromorphic circuit that influences neuronal phase synchrony

Neuromorphic circuits are designed and simulated to emulate the role of astrocytes in phase synchronization of neuronal activity. We emulate, to a first order, the ability of slow inward currents (SICs) evoked by the astrocyte, acting on extrasynaptic N-methyl-D-aspartate receptors (NMDAR) of adjacent neurons, as a mechanism for phase synchronization. We run a simulation test incorporating two small networks of neurons interacting with astrocytic microdomains. These microdomains are designed using a resistive and capacitive ladder network and their interactions occur through pass transistors. Upon enough synaptic activity, the astrocytic microdomains interact with each other, generating SIC events on synapses of adjacent neurons. Since the amplitude of SICs is several orders of magnitude larger compared to synaptic currents, a SIC event drastically enhances the excitatory postsynaptic potential (EPSP) on adjacent neurons simultaneously. This causes neurons to fire synchronously in phase. Phase synchrony holds for a duration of time proportional to the time constant of the SIC decay. Once the SIC decay has completed, the neurons are able to go back to their natural phase difference, inducing desynchronization of their firing of spikes. This paper incorporates some biological aspects observed by recent experiments showing astrocytic influence on neuronal synchronization, and intends to offer a circuit view on the hypothesis of astrocytic role on synchronous activity that could potentially lead to the binding of neuronal information.

Spin-transfer torque magnetic memory as a stochastic memristive synapse for neuromorphic systems

Spin-transfer torque magnetic memory (STT-MRAM) is currently under intense academic and industrial development, since it features non-volatility, high write and read speed and high endurance. In this work, we show that when used in a non-conventional regime, it can additionally act as a stochastic memristive device, appropriate to implement a "synaptic" function. We introduce basic concepts relating to spin-transfer torque magnetic tunnel junction (STT-MTJ, the STT-MRAM cell) behavior and its possible use to implement learning-capable synapses. Three programming regimes (low, intermediate and high current) are identified and compared. System-level simulations on a task of vehicle counting highlight the potential of the technology for learning systems. Monte Carlo simulations show its robustness to device variations. The simulations also allow comparing system operation when the different programming regimes of STT-MTJs are used. In comparison to the high and low current regimes, the intermediate current regime allows minimization of energy consumption, while retaining a high robustness to device variations. These results open the way for unexplored applications of STT-MTJs in robust, low power, cognitive-type systems.

Finding a roadmap to achieve large neuromorphic hardware systems

Neuromorphic systems are gaining increasing importance in an era where CMOS digital computing techniques are reaching physical limits. These silicon systems mimic extremely energy efficient neural computing structures, potentially both for solving engineering applications as well as understanding neural computation. Toward this end, the authors provide a glimpse at what the technology evolution roadmap looks like for these systems so that Neuromorphic engineers may gain the same benefit of anticipation and foresight that IC designers gained from Moore's law many years ago. Scaling of energy efficiency, performance, and size will be discussed as well as how the implementation and application space of Neuromorphic systems are expected to evolve over time.

Design of silicon brains in the nano-CMOS era: spiking neurons, learning synapses and neural architecture optimization

We present a design framework for neuromorphic architectures in the nano-CMOS era. Our approach to the design of spiking neurons and STDP learning circuits relies on parallel computational structures where neurons are abstracted as digital arithmetic logic units and communication processors. Using this approach, we have developed arrays of silicon neurons that scale to millions of neurons in a single state-of-the-art Field Programmable Gate Array (FPGA). We demonstrate the validity of the design methodology through the implementation of cortical development in a circuit of spiking neurons, STDP synapses, and neural architecture optimization.

Active processing of spatio-temporal input patterns in silicon dendrites

Capturing the functionality of active dendritic processing into abstract mathematical models will help us to understand the role of complex biophysical neurons in neuronal computation and to build future useful neuromorphic analog Very Large Scale Integrated (aVLSI) neuronal devices. Previous work based on an aVLSI multi-compartmental neuron model demonstrates that the compartmental response in the presence of either of two widely studied classes of active mechanisms, is a nonlinear sigmoidal function of the degree of either input temporal synchrony OR input clustering level. Using the same silicon model, this work expounds the interaction between both active mechanisms in a compartment receiving input patterns of varying temporal AND spatial clustering structure and demonstrates that this compartmental response can be captured by a combined sigmoid and radial-basis function over both input dimensions. This paper further shows that the response to input spatio-temporal patterns in a one-dimensional multi-compartmental dendrite, can be described by a radial-basis like function of the degree of temporal synchrony between the inter-compartmental inputs.

A learning-enabled neuron array IC based upon transistor channel models of biological phenomena

We present a single-chip array of 100 biologically-based electronic neuron models interconnected to each other and the outside environment through 30,000 synapses. The chip was fabricated in a standard 350 nm CMOS IC process. Our approach used dense circuit models of synaptic behavior, including biological computation and learning, as well as transistor channel models. We use Address-Event Representation (AER) spike communication for inputs and outputs to this IC. We present the IC architecture and infrastructure, including IC chip, configuration tools, and testing platform. We present measurement of small network of neurons, measurement of STDP neuron dynamics, and measurement from a compiled spiking neuron WTA topology, all compiled into this IC.

Parameter estimation of a spiking silicon neuron

Spiking neuron models are used in a multitude of tasks ranging from understanding neural behavior at its most basic level to neuroprosthetics. Parameter estimation of a single neuron model, such that the model's output matches that of a biological neuron is an extremely important task. Hand tuning of parameters to obtain such behaviors is a difficult and time consuming process. This is further complicated when the neuron is instantiated in silicon (an attractive medium in which to implement these models) as fabrication imperfections make the task of parameter configuration more complex. In this paper we show two methods to automate the configuration of a silicon (hardware) neuron's parameters. First, we show how a Maximum Likelihood method can be applied to a leaky integrate and fire silicon neuron with spike induced currents to fit the neuron's output to desired spike times. We then show how a distance based method which approximates the negative log likelihood of the lognormal distribution can also be used to tune the neuron's parameters. We conclude that the distance based method is better suited for parameter configuration of silicon neurons due to its superior optimization speed.

Tunable neuromimetic integrated system for emulating cortical neuron models

Nowadays, many software solutions are currently available for simulating neuron models. Less conventional than software-based systems, hardware-based solutions generally combine digital and analog forms of computation. In previous work, we designed several neuromimetic chips, including the Galway chip that we used for this paper. These silicon neurons are based on the Hodgkin–Huxley formalism and they are optimized for reproducing a large variety of neuron behaviors thanks to tunable parameters. Due to process variation and device mismatch in analog chips, we use a full-custom fitting method in voltage-clamp mode to tune our neuromimetic integrated circuits. By comparing them with experimental electrophysiological data of these cells, we show that the circuits can reproduce the main firing features of cortical cell types. In this paper, we present the experimental measurements of our system which mimic the four most prominent biological cells: fast spiking, regular spiking, intrinsically bursting, and low-threshold spiking neurons into analog neuromimetic integrated circuit dedicated to cortical neuron simulations. This hardware and software platform will allow to improve the hybrid technique, also called “dynamic-clamp,” that consists of connecting artificial and biological neurons to study the function of neuronal circuits.

Biophysical Neural Spiking, Bursting, and Excitability Dynamics in Reconfigurable Analog VLSI

We study a range of neural dynamics under variations in biophysical parameters underlying extended Morris-Lecar and Hodgkin-Huxley models in three gating variables. The extended models are implemented in NeuroDyn, a four neuron, twelve synapse continuous-time analog VLSI programmable neural emulation platform with generalized channel kinetics and biophysical membrane dynamics. The dynamics exhibit a wide range of time scales extending beyond 100 ms neglected in typical silicon models of tonic spiking neurons. Circuit simulations and measurements show transition from tonic spiking to tonic bursting dynamics through variation of a single conductance parameter governing calcium recovery. We similarly demonstrate transition from graded to all-or-none neural excitability in the onset of spiking dynamics through the variation of channel kinetic parameters governing the speed of potassium activation. Other combinations of variations in conductance and channel kinetic parameters give rise to phasic spiking and spike frequency adaptation dynamics. The NeuroDyn chip consumes 1.29 mW and occupies 3 mm × 3 mm in 0.5 μm CMOS, supporting emerging developments in neuromorphic silicon-neuron interfaces.

Automated Parameter Estimation of the Hodgkin-Huxley Model Using the Differential Evolution Algorithm: Application to Neuromimetic Analog Integrated Circuits

We propose a new estimation method for the characterization of the Hodgkin-Huxley formalism. This method is an alternative technique to the classical estimation methods associated with voltage clamp measurements. It uses voltage clamp type recordings, but is based on the differential evolution algorithm. The parameters of an ionic channel are estimated simultaneously, such that the usual approximations of classical methods are avoided and all the parameters of the model, including the time constant, can be correctly optimized. In a second step, this new estimation technique is applied to the automated tuning of neuromimetic analog integrated circuits designed by our research group. We present a tuning example of a fast spiking neuron, which reproduces the frequency-current characteristics of the reference data, as well as the membrane voltage behavior. The final goal of this tuning is to interconnect neuromimetic chips as neural networks, with specific cellular properties, for future theoretical studies in neuroscience.

Neuromorphic silicon neuron circuits

Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain–machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin–Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.

Real-time simulation of biologically realistic stochastic neurons in VLSI

Neuronal variability has been thought to play an important role in the brain. As the variability mainly comes from the uncertainty in biophysical mechanisms, stochastic neuron models have been proposed for studying how neurons compute with noise. However, most papers are limited to simulating stochastic neurons in a digital computer. The speed and the efficiency are thus limited especially when a large neuronal network is of concern. This brief explores the feasibility of simulating the stochastic behavior of biological neurons in a very large scale integrated (VLSI) system, which implements a programmable and configurable Hodgkin-Huxley model. By simply injecting noise to the VLSI neuron, various stochastic behaviors observed in biological neurons are reproduced realistically in VLSI. The noise-induced variability is further shown to enhance the signal modulation of a neuron. These results point toward the development of analog VLSI systems for exploring the stochastic behaviors of biological neuronal networks in large scale.