1
|
Baruzzi V, Indiveri G, Sabatini SP. Recurrent models of orientation selectivity enable robust early-vision processing in mixed-signal neuromorphic hardware. Nat Commun 2025; 16:243. [PMID: 39747257 PMCID: PMC11696034 DOI: 10.1038/s41467-024-55749-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 12/24/2024] [Indexed: 01/04/2025] Open
Abstract
Mixed signal analog/digital neuromorphic circuits represent an ideal medium for reproducing bio-physically realistic dynamics of biological neural systems in real-time. However, similar to their biological counterparts, these circuits have limited resolution and are affected by a high degree of variability. By developing a recurrent spiking neural network model of the retinocortical visual pathway, we show how such noisy and heterogeneous computing substrate can produce linear receptive fields tuned to visual stimuli with specific orientations and spatial frequencies. Compared to strictly feed-forward schemes, the model generates highly structured Gabor-like receptive fields of any phase symmetry, making optimal use of the hardware resources available in terms of synaptic connections and neuron numbers. Experimental results validate the approach, demonstrating how principles of neural computation can lead to robust sensory processing electronic systems, even when they are affected by high degree of heterogeneity, e.g., due to the use of analog circuits or memristive devices.
Collapse
Affiliation(s)
- Valentina Baruzzi
- Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Via Opera Pia 13, I-16145, Genoa, Italy
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Silvio P Sabatini
- Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Via Opera Pia 13, I-16145, Genoa, Italy.
| |
Collapse
|
2
|
Date P, Kulkarni S, Young A, Schuman C, Potok T, Vetter J. Encoding integers and rationals on neuromorphic computers using virtual neuron. Sci Rep 2023; 13:10975. [PMID: 37414838 PMCID: PMC10326008 DOI: 10.1038/s41598-023-35005-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 05/11/2023] [Indexed: 07/08/2023] Open
Abstract
Neuromorphic computers emulate the human brain while being extremely power efficient for computing tasks. In fact, they are poised to be critical for energy-efficient computing in the future. Neuromorphic computers are primarily used in spiking neural network-based machine learning applications. However, they are known to be Turing-complete, and in theory can perform all general-purpose computation. One of the biggest bottlenecks in realizing general-purpose computations on neuromorphic computers today is the inability to efficiently encode data on the neuromorphic computers. To fully realize the potential of neuromorphic computers for energy-efficient general-purpose computing, efficient mechanisms must be devised for encoding numbers. Current encoding mechanisms (e.g., binning, rate-based encoding, and time-based encoding) have limited applicability and are not suited for general-purpose computation. In this paper, we present the virtual neuron abstraction as a mechanism for encoding and adding integers and rational numbers by using spiking neural network primitives. We evaluate the performance of the virtual neuron on physical and simulated neuromorphic hardware. We estimate that the virtual neuron could perform an addition operation using just 23 nJ of energy on average with a mixed-signal, memristor-based neuromorphic processor. We also demonstrate the utility of the virtual neuron by using it in some of the μ-recursive functions, which are the building blocks of general-purpose computation.
Collapse
Affiliation(s)
- Prasanna Date
- Oak Ridge National Laboratory, Oak Ridge, TN, 37830, USA.
| | | | - Aaron Young
- Oak Ridge National Laboratory, Oak Ridge, TN, 37830, USA
| | | | - Thomas Potok
- Oak Ridge National Laboratory, Oak Ridge, TN, 37830, USA
| | - Jeffrey Vetter
- Oak Ridge National Laboratory, Oak Ridge, TN, 37830, USA
| |
Collapse
|
3
|
Ou W, Xiao S, Zhu C, Han W, Zhang Q. An overview of brain-like computing: Architecture, applications, and future trends. Front Neurorobot 2022; 16:1041108. [PMID: 36506817 PMCID: PMC9730831 DOI: 10.3389/fnbot.2022.1041108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2022] [Accepted: 10/31/2022] [Indexed: 11/25/2022] Open
Abstract
With the development of technology, Moore's law will come to an end, and scientists are trying to find a new way out in brain-like computing. But we still know very little about how the brain works. At the present stage of research, brain-like models are all structured to mimic the brain in order to achieve some of the brain's functions, and then continue to improve the theories and models. This article summarizes the important progress and status of brain-like computing, summarizes the generally accepted and feasible brain-like computing models, introduces, analyzes, and compares the more mature brain-like computing chips, outlines the attempts and challenges of brain-like computing applications at this stage, and looks forward to the future development of brain-like computing. It is hoped that the summarized results will help relevant researchers and practitioners to quickly grasp the research progress in the field of brain-like computing and acquire the application methods and related knowledge in this field.
Collapse
Affiliation(s)
- Wei Ou
- The School of Cyberspace Security, Hainan University, Hainan, China
- Henan Key Laboratory of Network Cryptography Technology, Zhengzhou, China
| | - Shitao Xiao
- The School of Computer Science and Technology, Hainan, China
| | - Chengyu Zhu
- The School of Cyberspace Security, Hainan University, Hainan, China
| | - Wenbao Han
- The School of Cyberspace Security, Hainan University, Hainan, China
| | - Qionglu Zhang
- State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
4
|
Wang Z, Hong Q, Wang X. A Memristive Circuit Implementation of Eyes State Detection in Fatigue Driving Based on Biological Long Short-Term Memory Rule. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2021; 18:2218-2229. [PMID: 32086217 DOI: 10.1109/tcbb.2020.2974944] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Biological long short-term memory (B-LSTM) can effectively help human process all kinds of received information. In this work, a memristive B-LSTM circuit which mimics a conversion from short-term memory to long-term memory is proposed. That is, the stronger the signal, the more profound the memory and the higher the output. On this basis, an image binarization circuit using adaptive row threshold algorithm is proposed. It can make the image remain a deep impression on the strong pixel information and effectively filter the relatively weak pixel information. In combination with the function of image binarization, a memristive circuit for eyes state detection is proposed by adding corresponding horizontal projection calculation, subtraction calculation and judgement open or closed eyes modules. The proposed circuit can detect whether there is a blink between two adjacent facial images, which uses the characteristics of memristor to detect the difference of horizontal projection between two images. Due to the use of memristor, the proposed circuit can realize in-memory computing, which fundamentally avoids the problem of storage wall and shorten the execution time. Finally, an expectation application in fatigue driving based on the proposed method is demonstrated, which indicates the practicability of the circuit design in this work.
Collapse
|
5
|
Hussain S, Basu A. Multiclass Classification by Adaptive Network of Dendritic Neurons with Binary Synapses Using Structural Plasticity. Front Neurosci 2016; 10:113. [PMID: 27065782 PMCID: PMC4814530 DOI: 10.3389/fnins.2016.00113] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2015] [Accepted: 03/07/2016] [Indexed: 11/28/2022] Open
Abstract
The development of power-efficient neuromorphic devices presents the challenge of designing spike pattern classification algorithms which can be implemented on low-precision hardware and can also achieve state-of-the-art performance. In our pursuit of meeting this challenge, we present a pattern classification model which uses a sparse connection matrix and exploits the mechanism of nonlinear dendritic processing to achieve high classification accuracy. A rate-based structural learning rule for multiclass classification is proposed which modifies a connectivity matrix of binary synaptic connections by choosing the best "k" out of "d" inputs to make connections on every dendritic branch (k < < d). Because learning only modifies connectivity, the model is well suited for implementation in neuromorphic systems using address-event representation (AER). We develop an ensemble method which combines several dendritic classifiers to achieve enhanced generalization over individual classifiers. We have two major findings: (1) Our results demonstrate that an ensemble created with classifiers comprising moderate number of dendrites performs better than both ensembles of perceptrons and of complex dendritic trees. (2) In order to determine the moderate number of dendrites required for a specific classification problem, a two-step solution is proposed. First, an adaptive approach is proposed which scales the relative size of the dendritic trees of neurons for each class. It works by progressively adding dendrites with fixed number of synapses to the network, thereby allocating synaptic resources as per the complexity of the given problem. As a second step, theoretical capacity calculations are used to convert each neuronal dendritic tree to its optimal topology where dendrites of each class are assigned different number of synapses. The performance of the model is evaluated on classification of handwritten digits from the benchmark MNIST dataset and compared with other spike classifiers. We show that our system can achieve classification accuracy within 1 - 2% of other reported spike-based classifiers while using much less synaptic resources (only 7%) compared to that used by other methods. Further, an ensemble classifier created with adaptively learned sizes can attain accuracy of 96.4% which is at par with the best reported performance of spike-based classifiers. Moreover, the proposed method achieves this by using about 20% of the synapses used by other spike algorithms. We also present results of applying our algorithm to classify the MNIST-DVS dataset collected from a real spike-based image sensor and show results comparable to the best reported ones (88.1% accuracy). For VLSI implementations, we show that the reduced synaptic memory can save upto 4X area compared to conventional crossbar topologies. Finally, we also present a biologically realistic spike-based version for calculating the correlations required by the structural learning rule and demonstrate the correspondence between the rate-based and spike-based methods of learning.
Collapse
Affiliation(s)
| | - Arindam Basu
- School of Electrical and Electronic Engineering, Nanyang Technological UniversitySingapore, Singapore
| |
Collapse
|
6
|
Hussain S, Liu SC, Basu A. Hardware-amenable structural learning for spike-based pattern classification using a simple model of active dendrites. Neural Comput 2015; 27:845-97. [PMID: 25734494 DOI: 10.1162/neco_a_00713] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
This letter presents a spike-based model that employs neurons with functionally distinct dendritic compartments for classifying high-dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron the capacity to perform a large number of input-output mappings. The model uses sparse synaptic connectivity, where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin-enhancing learning algorithm inspired by the mechanism of structural plasticity in biological neurons. The learning algorithm groups correlated synaptic inputs on the same dendritic branch. Since the learning results in modified connection patterns, it can be incorporated into current event-based neuromorphic systems with little overhead. This work also presents a branch-specific spike-based version of this structural plasticity rule. The proposed model is evaluated on benchmark binary classification problems, and its performance is compared against that achieved using support vector machine and extreme learning machine techniques. Our proposed method attains comparable performance while using 10% to 50% less in computational resource than the other reported techniques.
Collapse
Affiliation(s)
- Shaista Hussain
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798
| | | | | |
Collapse
|
7
|
Gupta P, Markan CM. An adaptable neuromorphic model of orientation selectivity based on floating gate dynamics. Front Neurosci 2014; 8:54. [PMID: 24765062 PMCID: PMC3980111 DOI: 10.3389/fnins.2014.00054] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2013] [Accepted: 03/09/2014] [Indexed: 11/21/2022] Open
Abstract
The biggest challenge that the neuromorphic community faces today is to build systems that can be considered truly cognitive. Adaptation and self-organization are the two basic principles that underlie any cognitive function that the brain performs. If we can replicate this behavior in hardware, we move a step closer to our goal of having cognitive neuromorphic systems. Adaptive feature selectivity is a mechanism by which nature optimizes resources so as to have greater acuity for more abundant features. Developing neuromorphic feature maps can help design generic machines that can emulate this adaptive behavior. Most neuromorphic models that have attempted to build self-organizing systems, follow the approach of modeling abstract theoretical frameworks in hardware. While this is good from a modeling and analysis perspective, it may not lead to the most efficient hardware. On the other hand, exploiting hardware dynamics to build adaptive systems rather than forcing the hardware to behave like mathematical equations, seems to be a more robust methodology when it comes to developing actual hardware for real world applications. In this paper we use a novel time-staggered Winner Take All circuit, that exploits the adaptation dynamics of floating gate transistors, to model an adaptive cortical cell that demonstrates Orientation Selectivity, a well-known biological phenomenon observed in the visual cortex. The cell performs competitive learning, refining its weights in response to input patterns resembling different oriented bars, becoming selective to a particular oriented pattern. Different analysis performed on the cell such as orientation tuning, application of abnormal inputs, response to spatial frequency and periodic patterns reveal close similarity between our cell and its biological counterpart. Embedded in a RC grid, these cells interact diffusively exhibiting cluster formation, making way for adaptively building orientation selective maps in silicon.
Collapse
Affiliation(s)
- Priti Gupta
- VLSI Design Technology Lab, Department of Physics and Computer Science, Dayalbagh Educational Institute Agra, Uttar Pradesh, India
| | - C M Markan
- VLSI Design Technology Lab, Department of Physics and Computer Science, Dayalbagh Educational Institute Agra, Uttar Pradesh, India
| |
Collapse
|
8
|
Camuñas-Mesa LA, Serrano-Gotarredona T, Ieng SH, Benosman RB, Linares-Barranco B. On the use of orientation filters for 3D reconstruction in event-driven stereo vision. Front Neurosci 2014; 8:48. [PMID: 24744694 PMCID: PMC3978326 DOI: 10.3389/fnins.2014.00048] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2013] [Accepted: 02/23/2014] [Indexed: 11/13/2022] Open
Abstract
The recently developed Dynamic Vision Sensors (DVS) sense visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, therefore increasing the number of constraints applied to the matching algorithm. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.
Collapse
Affiliation(s)
- Luis A Camuñas-Mesa
- Instituto de Microelectrónica de Sevilla (IMSE-CNM), CSIC y Universidad de Sevilla Sevilla, Spain
| | | | - Sio H Ieng
- UMR_S968 Inserm/UPMC/CNRS 7210, Institut de la Vision, Université de Pierre et Marie Curie Paris, France
| | - Ryad B Benosman
- UMR_S968 Inserm/UPMC/CNRS 7210, Institut de la Vision, Université de Pierre et Marie Curie Paris, France
| | - Bernabe Linares-Barranco
- Instituto de Microelectrónica de Sevilla (IMSE-CNM), CSIC y Universidad de Sevilla Sevilla, Spain
| |
Collapse
|
9
|
Markan CM, Gupta P, Bansal M. An adaptive neuromorphic model of ocular dominance map using floating gate 'synapse'. Neural Netw 2013; 45:117-33. [PMID: 23648171 DOI: 10.1016/j.neunet.2013.04.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2012] [Revised: 04/02/2013] [Accepted: 04/04/2013] [Indexed: 11/28/2022]
Abstract
A novel analogue CMOS design of a cortical cell, that computes weighted sum of inputs, is presented. The cell's feedback regime exploits the adaptation dynamics of floating gate pFET 'synapse' to perform competitive learning amongst input weights as time-staggered winner take all. A learning rate parameter regulates adaptation time and a bias enforces resource limitation by restricting the number of input branches and winners in a competition. When learning ends, the cell's response favours one input pattern over others to exhibit feature selectivity. Embedded in a 2-D RC grid, these feature selective cells are capable of performing a symmetry breaking pattern formation, observed in some reaction-diffusion models of cortical feature map formation, e.g. ocular dominance. Close similarity with biological networks in terms of adaptability and long term memory indicates that the cell's design is ideally suited for analogue VLSI implementation of Self-Organizing Feature Map (SOFM) models of cortical feature maps.
Collapse
Affiliation(s)
- C M Markan
- Department of Physics & Computer Science, Dayalbagh Educational Institute (Deemed University), Dayalbagh, Agra-282005, India.
| | | | | |
Collapse
|
10
|
Benjamin BV, Arthur JV, Gao P, Merolla P, Boahen K. A superposable silicon synapse with programmable reversal potential. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2012:771-4. [PMID: 23366006 DOI: 10.1109/embc.2012.6346045] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
We present a novel log-domain silicon synapse designed for subthreshold analog operation that emulates common synaptic interactions found in biology. Our circuit models the dynamic gating of ion-channel conductances by emulating the processes of neurotransmitter release-reuptake and receptor binding-unbinding in a superposable fashion: Only a single circuit is required to model the entire population of synapses (of a given type) that a biological neuron receives. Unlike previous designs, which are strictly excitatory or inhibitory, our silicon synapse implements-for the first time in the log-domain-a programmable reversal potential (i.e., driving force). To demonstrate our design's scalability, we fabricated in 180nm CMOS an array of 64K silicon neurons, each with four independent superposable synapse circuits occupying 11.0×21.5 µm(2) apiece. After verifying that these synapses have the predicted effect on the neurons' spike rate, we explored a recurrent network where the synapses' reversal potentials are set near the neurons' threshold, acting as shunts. These shunting synapses synchronized neuronal spiking more robustly than nonshunting synapses, confirming that reversal potentials can have important network-level implications.
Collapse
Affiliation(s)
- Ben V Benjamin
- Electrical Engineering and P. Gao and K. Boahen are with Bioengineering, Stanford University, Stanford, CA, USA.
| | | | | | | | | |
Collapse
|
11
|
Basu A, Shuo S, Zhou H, Hiot Lim M, Huang GB. Silicon spiking neurons for hardware implementation of extreme learning machines. Neurocomputing 2013. [DOI: 10.1016/j.neucom.2012.01.042] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
12
|
Zamarreno-Ramos C, Linares-Barranco A, Serrano-Gotarredona T, Linares-Barranco B. Multicasting mesh AER: a scalable assembly approach for reconfigurable neuromorphic structured AER systems. Application to ConvNets. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2013; 7:82-102. [PMID: 23853282 DOI: 10.1109/tbcas.2012.2195725] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
This paper presents a modular, scalable approach to assembling hierarchically structured neuromorphic Address Event Representation (AER) systems. The method consists of arranging modules in a 2D mesh, each communicating bidirectionally with all four neighbors. Address events include a module label. Each module includes an AER router which decides how to route address events. Two routing approaches have been proposed, analyzed and tested, using either destination or source module labels. Our analyses reveal that depending on traffic conditions and network topologies either one or the other approach may result in better performance. Experimental results are given after testing the approach using high-end Virtex-6 FPGAs. The approach is proposed for both single and multiple FPGAs, in which case a special bidirectional parallel-serial AER link with flow control is exploited, using the FPGA Rocket-I/O interfaces. Extensive test results are provided exploiting convolution modules of 64 × 64 pixels with kernels with sizes up to 11 × 11, which process real sensory data from a Dynamic Vision Sensor (DVS) retina. One single Virtex-6 FPGA can hold up to 64 of these convolution modules, which is equivalent to a neural network with 262 × 10(3) neurons and almost 32 million synapses.
Collapse
|
13
|
Zamarreño-Ramos C, Serrano-Gotarredona T, Linares-Barranco B. A 0.35 μm sub-ns wake-up time ON-OFF switchable LVDS driver-receiver chip I/O pad pair for rate-dependent power saving in AER bit-serial links. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2012; 6:486-497. [PMID: 23853235 DOI: 10.1109/tbcas.2012.2186136] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
This paper presents a low power switchable current mode driver/receiver I/O pair for high speed serial transmission of asynchronous address event representation (AER) information. The sparse nature of AER packets (also called events) allows driver/receiver bias currents to be switched off to save power. The on/off times must be lower than the bit time to minimize the latency introduced by the switching mechanism. Using this technique, the link power consumption can be scaled down with the event rate without compromising the maximum system throughput. The proposed technique has been implemented on a typical push/pull low voltage differential signaling (LVDS) circuit, but it can easily be extended to other widely used current mode standards, such as current mode logic (CML) or low-voltage positive emitter-coupled logic (LVPECL). A proof of concept prototype has been fabricated in 0.35 μm CMOS incorporating the proposed driver/receiver pair along with a previously reported switchable serializer/deserializer scheme. At a 500 Mbps bit rate, the maximum event rate is 11 Mevent/s for 32-bit events. In this situation, current consumption is 7.5 mA and 9.6 mA for the driver and receiver, respectively, while differential voltage amplitude is ±300 mV. However, if event rate is lower than 20-30 Kevent/s, current consumption has a floor of 270 μA for the driver and 570 μA for the receiver. The measured ON/OFF switching times are in the order of 1 ns. The serial link could be operated at up to 710 Mbps bit rate, resulting in a maximum 32-bit event rate of 15 Mevent/s . This is the same peak event rate as that obtained with the same SerDes circuits and a non-switched driver/receiver pair.
Collapse
|
14
|
Asynchronous frameless event-based optical flow. Neural Netw 2012; 27:32-7. [DOI: 10.1016/j.neunet.2011.11.001] [Citation(s) in RCA: 115] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2011] [Revised: 10/03/2011] [Accepted: 11/02/2011] [Indexed: 11/19/2022]
|
15
|
MENG YICONG, TSANG ERICKC, LAM STANLEYYM, SHI BERTRAME. THE HKUST MULTIMAP SYSTEM FOR ACTIVE VISION. INT J HUM ROBOT 2011. [DOI: 10.1142/s0219843609001838] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
This paper describes the HKUST MultiMap system, a hardware system we have developed to support the computation and integration of large numbers of neurally inspired feature maps at frame rates. The computed maps could serve as the basis for a generic image representation to enable robots to perform a variety of different tasks. In order to support the computation of feature representations with increasing complexity, the system can split the processing among a scalable number of processors, each on a different printed circuit board. We describe the hardware design of this board, as well as the communication protocol between different boards. Our experimental results on a four board system validate the performance of the proposed communication protocol. As an example of the applications enabled by this system, we describe a robotic binocular tracking system built by using it.
Collapse
Affiliation(s)
- YICONG MENG
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
| | - ERIC K. C. TSANG
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
| | - STANLEY Y. M. LAM
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
| | - BERTRAM E. SHI
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
| |
Collapse
|
16
|
Neftci E, Chicca E, Indiveri G, Douglas R. A Systematic Method for Configuring VLSI Networks of Spiking Neurons. Neural Comput 2011; 23:2457-97. [DOI: 10.1162/neco_a_00182] [Citation(s) in RCA: 46] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.
Collapse
Affiliation(s)
- Emre Neftci
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Elisabetta Chicca
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Giacomo Indiveri
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Rodney Douglas
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| |
Collapse
|
17
|
Benosman R, Ieng SH, Rogister P, Posch C. Asynchronous event-based hebbian epipolar geometry. ACTA ACUST UNITED AC 2011; 22:1723-34. [PMID: 21954205 DOI: 10.1109/tnn.2011.2167239] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Epipolar geometry, the cornerstone of perspective stereo vision, has been studied extensively since the advent of computer vision. Establishing such a geometric constraint is of primary importance, as it allows the recovery of the 3-D structure of scenes. Estimating the epipolar constraints of nonperspective stereo is difficult, they can no longer be defined because of the complexity of the sensor geometry. This paper will show that these limitations are, to some extent, a consequence of the static image frames commonly used in vision. The conventional frame-based approach suffers from a lack of the dynamics present in natural scenes. We introduce the use of neuromorphic event-based--rather than frame-based--vision sensors for perspective stereo vision. This type of sensor uses the dimension of time as the main conveyor of information. In this paper, we present a model for asynchronous event-based vision, which is then used to derive a general new concept of epipolar geometry linked to the temporal activation of pixels. Practical experiments demonstrate the validity of the approach, solving the problem of estimating the fundamental matrix applied, in a first stage, to classic perspective vision and then to more general cameras. Furthermore, this paper shows that the properties of event-based vision sensors allow the exploration of not-yet-defined geometric relationships, finally, we provide a definition of general epipolar geometry deployable to almost any visual sensor.
Collapse
Affiliation(s)
- Ryad Benosman
- Vision Institute and the Institute of Intelligent Systems and Robotics, University Pierre and Marie Curie, Paris 75252 Cedex 05, France.
| | | | | | | |
Collapse
|
18
|
Ohno T, Hasegawa T, Tsuruoka T, Terabe K, Gimzewski JK, Aono M. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. NATURE MATERIALS 2011; 10:591-5. [PMID: 21706012 DOI: 10.1038/nmat3054] [Citation(s) in RCA: 609] [Impact Index Per Article: 43.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2011] [Accepted: 05/24/2011] [Indexed: 05/22/2023]
Abstract
Memory is believed to occur in the human brain as a result of two types of synaptic plasticity: short-term plasticity (STP) and long-term potentiation (LTP; refs 1-4). In neuromorphic engineering, emulation of known neural behaviour has proven to be difficult to implement in software because of the highly complex interconnected nature of thought processes. Here we report the discovery of a Ag(2)S inorganic synapse, which emulates the synaptic functions of both STP and LTP characteristics through the use of input pulse repetition time. The structure known as an atomic switch, operating at critical voltages, stores information as STP with a spontaneous decay of conductance level in response to intermittent input stimuli, whereas frequent stimulation results in a transition to LTP. The Ag(2)S inorganic synapse has interesting characteristics with analogies to an individual biological synapse, and achieves dynamic memorization in a single device without the need of external preprogramming. A psychological model related to the process of memorizing and forgetting is also demonstrated using the inorganic synapses. Our Ag(2)S element indicates a breakthrough in mimicking synaptic behaviour essential for the further creation of artificial neural systems that emulate characteristics of human memory.
Collapse
Affiliation(s)
- Takeo Ohno
- International Center for Materials Nanoarchitectonics (MANA), National Institute for Materials Science (NIMS), 1-1 Namiki, Tsukuba, Ibaraki 305-0044, Japan.
| | | | | | | | | | | |
Collapse
|
19
|
|
20
|
Wang Y, Liu SC. Multilayer processing of spatiotemporal spike patterns in a neuron with active dendrites. Neural Comput 2010; 22:2086-112. [PMID: 20337538 DOI: 10.1162/neco.2010.06-09-1030] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
With the advent of new experimental evidence showing that dendrites play an active role in processing a neuron's inputs, we revisit the question of a suitable abstraction for the computing function of a neuron in processing spatiotemporal input patterns. Although the integrative role of a neuron in relation to the spatial clustering of synaptic inputs can be described by a two-layer neural network, no corresponding abstraction has yet been described for how a neuron processes temporal input patterns on the dendrites. We address this void using a real-time aVLSI (analog very-large-scale-integrated) dendritic compartmental model, which incorporates two widely studied classes of regenerative event mechanisms: one is mediated by voltage-gated ion channels and the other by transmitter-gated NMDA channels. From this model, we find that the response of a dendritic compartment can be described as a nonlinear sigmoidal function of both the degree of input temporal synchrony and the synaptic input spatial clustering. We propose that a neuron with active dendrites can be modeled as a multilayer network that selectively amplifies responses to relevant spatiotemporal input spike patterns.
Collapse
Affiliation(s)
- Yingxue Wang
- Institute of Neuroinformatics, University of Zürich and ETH Zürich, Zürich, Switzerland.
| | | |
Collapse
|
21
|
Neuromorphic sensory systems. Curr Opin Neurobiol 2010; 20:288-95. [DOI: 10.1016/j.conb.2010.03.007] [Citation(s) in RCA: 210] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2010] [Revised: 03/22/2010] [Accepted: 03/24/2010] [Indexed: 11/17/2022]
|
22
|
Bamford SA, Murray AF, Willshaw DJ. Large developing receptive fields using a distributed and locally reprogrammable address-event receiver. ACTA ACUST UNITED AC 2010; 21:286-304. [PMID: 20071258 DOI: 10.1109/tnn.2009.2036912] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
A distributed and locally reprogrammable address-event receiver has been designed, in which incoming address-events are monitored simultaneously by all synapses, allowing for arbitrarily large axonal fan-out without reducing channel capacity. Synapses can change the address of their presynaptic neuron, allowing the distributed implementation of a biologically realistic learning rule, with both synapse formation and elimination (synaptic rewiring). Probabilistic synapse formation leads to topographic map development, made possible by a cross-chip current-mode calculation of Euclidean distance. As well as synaptic plasticity in rewiring, synapses change weights using a competitive Hebbian learning rule (spike-timing-dependent plasticity). The weight plasticity allows receptive fields to be modified based on spatio-temporal correlations in the inputs, and the rewiring plasticity allows these modifications to become embedded in the network topology.
Collapse
Affiliation(s)
- Simeon A Bamford
- Institute of Integrated Micro and Nano Systems, Neuroinformatics Doctoral Training Centre, University of Edinburgh, Edinburgh, UK.
| | | | | |
Collapse
|
23
|
Abstract
The winner-take-all (WTA) computation in networks of recurrently connected neurons is an important decision element of many models of cortical processing. However, analytical studies of the WTA performance in recurrent networks have generally addressed rate-based models. Very few have addressed networks of spiking neurons, which are relevant for understanding the biological networks themselves and also for the development of neuromorphic electronic neurons that commmunicate by action potential like address-events. Here, we make steps in that direction by using a simplified Markov model of the spiking network to examine analytically the ability of a spike-based WTA network to discriminate the statistics of inputs ranging from stationary regular to nonstationary Poisson events. Our work extends previous theoretical results showing that a WTA recurrent network receiving regular spike inputs can select the correct winner within one interspike interval. We show first for the case of spike rate inputs that input discrimination and the effects of self-excitation and inhibition on this discrimination are consistent with results obtained from the standard rate-based WTA models. We also extend this discrimination analysis of spiking WTAs to nonstationary inputs with time-varying spike rates resembling statistics of real-world sensory stimuli. We conclude that spiking WTAs are consistent with their continuous counterparts for steady-state inputs, but they also exhibit high discrimination performance with nonstationary inputs.
Collapse
Affiliation(s)
- Matthias Oster
- Institute of Neuroinformatics, Uni-ETH Zurich, CH-8057 Zurich, Switzerland.
| | | | | |
Collapse
|
24
|
Mitra S, Fusi S, Indiveri G. Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2009; 3:32-42. [PMID: 23853161 DOI: 10.1109/tbcas.2008.2005781] [Citation(s) in RCA: 48] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
Real-time classification of patterns of spike trains is a difficult computational problem that both natural and artificial networks of spiking neurons are confronted with. The solution to this problem not only could contribute to understanding the fundamental mechanisms of computation used in the biological brain, but could also lead to efficient hardware implementations of a wide range of applications ranging from autonomous sensory-motor systems to brain-machine interfaces. Here we demonstrate real-time classification of complex patterns of mean firing rates, using a VLSI network of spiking neurons and dynamic synapses which implement a robust spike-driven plasticity mechanism. The learning rule implemented is a supervised one: a teacher signal provides the output neuron with an extra input spike-train during training, in parallel to the spike-trains that represent the input pattern. The teacher signal simply indicates if the neuron should respond to the input pattern with a high rate or with a low one. The learning mechanism modifies the synaptic weights only as long as the current generated by all the stimulated plastic synapses does not match the output desired by the teacher, as in the perceptron learning rule. We describe the implementation of this learning mechanism and present experimental data that demonstrate how the VLSI neural network can learn to classify patterns of neural activities, also in the case in which they are highly correlated.
Collapse
|
25
|
Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition. Cognit Comput 2009. [DOI: 10.1007/s12559-008-9003-6] [Citation(s) in RCA: 40] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
26
|
|
27
|
Shimonomura K, Yagi T. Neuromorphic VLSI vision system for real-time texture segregation. Neural Netw 2008; 21:1197-204. [PMID: 18723317 DOI: 10.1016/j.neunet.2008.07.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2007] [Revised: 07/14/2008] [Accepted: 07/28/2008] [Indexed: 10/21/2022]
Abstract
The visual system of the brain can perceive an external scene in real-time with extremely low power dissipation, although the response speed of an individual neuron is considerably lower than that of semiconductor devices. The neurons in the visual pathway generate their receptive fields using a parallel and hierarchical architecture. This architecture of the visual cortex is interesting and important for designing a novel perception system from an engineering perspective. The aim of this study is to develop a vision system hardware, which is designed inspired by a hierarchical visual processing in V1, for real time texture segregation. The system consists of a silicon retina, orientation chip, and field programmable gate array (FPGA) circuit. The silicon retina emulates the neural circuits of the vertebrate retina and exhibits a Laplacian-Gaussian-like receptive field. The orientation chip selectively aggregates multiple pixels of the silicon retina in order to produce Gabor-like receptive fields that are tuned to various orientations by mimicking the feed-forward model proposed by Hubel and Wiesel. The FPGA circuit receives the output of the orientation chip and computes the responses of the complex cells. Using this system, the neural images of simple cells were computed in real-time for various orientations and spatial frequencies. Using the orientation-selective outputs obtained from the multi-chip system, a real-time texture segregation was conducted based on a computational model inspired by psychophysics and neurophysiology. The texture image was filtered by the two orthogonally oriented receptive fields of the multi-chip system and the filtered images were combined to segregate the area of different texture orientation with the aid of FPGA. The present system is also useful for the investigation of the functions of the higher-order cells that can be obtained by combining the simple and complex cells.
Collapse
Affiliation(s)
- Kazuhiro Shimonomura
- The Center for Advanced Medical Engineering and Informatics, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan
| | | |
Collapse
|
28
|
Serrano-Gotarredona R, Serrano-Gotarredona T, Acosta-Jimenez A, Serrano-Gotarredona C, Perez-Carrasco J, Linares-Barranco B, Linares-Barranco A, Jimenez-Moreno G, Civit-Ballcels A. On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing. ACTA ACUST UNITED AC 2008. [DOI: 10.1109/tnn.2008.2000163] [Citation(s) in RCA: 54] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
29
|
Meng Y, Shi BE. Adaptive gain control for spike-based map communication in a neuromorphic vision system. IEEE TRANSACTIONS ON NEURAL NETWORKS 2008; 19:1010-21. [PMID: 18541501 DOI: 10.1109/tnn.2007.915113] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
To support large numbers of model neurons, neuromorphic vision systems are increasingly adopting a distributed architecture, where different arrays of neurons are located on different chips or processors. Spike-based protocols are used to communicate activity between processors. The spike activity in the arrays depends on the input statistics as well as internal parameters such as time constants and gains. In this paper, we investigate strategies for automatically adapting these parameters to maintain a constant firing rate in response to changes in the input statistics. We find that under the constraint of maintaining a fixed firing rate, a strategy based upon updating the gain alone performs as well as an optimal strategy where both the gain and the time constant are allowed to vary. We discuss how to choose the time constant and propose an adaptive gain control mechanism whose operation is robust to changes in the input statistics. Our experimental results on a mobile robotic platform validate the analysis and efficacy of the proposed strategy.
Collapse
Affiliation(s)
- Yicong Meng
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong.
| | | |
Collapse
|
30
|
Shimonomura K, Kushima T, Yagi T. Binocular robot vision emulating disparity computation in the primary visual cortex. Neural Netw 2008; 21:331-40. [DOI: 10.1016/j.neunet.2007.12.033] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2007] [Revised: 12/01/2007] [Accepted: 12/11/2007] [Indexed: 12/01/2022]
|
31
|
|
32
|
Vogelstein RJ, Mallik U, Culurciello E, Cauwenberghs G, Etienne-Cummings R. A multichip neuromorphic system for spike-based visual information processing. Neural Comput 2007; 19:2281-300. [PMID: 17650061 DOI: 10.1162/neco.2007.19.9.2281] [Citation(s) in RCA: 56] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
We present a multichip, mixed-signal VLSI system for spike-based vision processing. The system consists of an 80 x 60 pixel neuromorphic retina and a 4800 neuron silicon cortex with 4,194,304 synapses. Its functionality is illustrated with experimental data on multiple components of an attention-based hierarchical model of cortical object recognition, including feature coding, salience detection, and foveation. This model exploits arbitrary and reconfigurable connectivity between cells in the multichip architecture, achieved by asynchronously routing neural spike events within and between chips according to a memory-based look-up table. Synaptic parameters, including conductance and reversal potential, are also stored in memory and are used to dynamically configure synapse circuits within the silicon neurons.
Collapse
Affiliation(s)
- R Jacob Vogelstein
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21205, USA.
| | | | | | | | | |
Collapse
|
33
|
Costas-Santos J, Serrano-Gotarredona T, Serrano-Gotarredona R, Linares-Barranco B. A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2007.900179] [Citation(s) in RCA: 83] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
34
|
Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, Del Giudice P, Douglas RJ, Indiveri G. A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2007.893509] [Citation(s) in RCA: 93] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
35
|
Merolla PA, Arthur JV, Shi BE, Boahen KA. Expandable Networks for Neuromorphic Chips. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2006.887474] [Citation(s) in RCA: 76] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
36
|
Serrano-Gotarredona R, Serrano-Gotarredona T, Acosta-Jimenez A, Linares-Barranco B. A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems. ACTA ACUST UNITED AC 2006. [DOI: 10.1109/tcsi.2006.883843] [Citation(s) in RCA: 78] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|