1
|
Cotteret M, Greatorex H, Ziegler M, Chicca E. Vector Symbolic Finite State Machines in Attractor Neural Networks. Neural Comput 2024; 36:549-595. [PMID: 38457766 DOI: 10.1162/neco_a_01638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 10/19/2023] [Indexed: 03/10/2024]
Abstract
Hopfield attractor networks are robust distributed models of human memory, but they lack a general mechanism for effecting state-dependent attractor transitions in response to input. We propose construction rules such that an attractor network may implement an arbitrary finite state machine (FSM), where states and stimuli are represented by high-dimensional random vectors and all state transitions are enacted by the attractor network's dynamics. Numerical simulations show the capacity of the model, in terms of the maximum size of implementable FSM, to be linear in the size of the attractor network for dense bipolar state vectors and approximately quadratic for sparse binary state vectors. We show that the model is robust to imprecise and noisy weights, and so a prime candidate for implementation with high-density but unreliable devices. By endowing attractor networks with the ability to emulate arbitrary FSMs, we propose a plausible path by which FSMs could exist as a distributed computational primitive in biological neural networks.
Collapse
Affiliation(s)
- Madison Cotteret
- Micro- and Nanoelectronic Systems, Institute of Micro- and Nanotechnologies (IMN) MacroNano, Technische Universität Ilmenau, 98693 Ilmenau, Germany
- Bio-Inspired Circuits and Systems Lab, Zernike Institute for Advanced Materials, and Groningen Cognitive Systems and Materials Center, University of Groningen, 9747 AG Groningen, Netherlands
| | - Hugh Greatorex
- Bio-Inspired Circuits and Systems Lab, Zernike Institute for Advanced Materials, and Groningen Cognitive Systems and Materials Center, University of Groningen, 9747 AG Groningen, Netherlands
| | - Martin Ziegler
- Micro- and Nanoelectronic Systems, Institute of Micro- and Nanotechnologies (IMN) MacroNano, Technische Universität Ilmenau, 98693 Ilmenau, Germany
| | - Elisabetta Chicca
- Bio-Inspired Circuits and Systems Lab, Zernike Institute for Advanced Materials, and Groningen Cognitive Systems and Materials Center, University of Groningen, 9747 AG Groningen, Netherlands
| |
Collapse
|
2
|
He Y, van der Ven S, Liaw HP, Shi C, Russo P, Gourdouparis M, Konijnenburg M, Traferro S, Timmermans M, Lopez CM, Harpe P, Cantatore E, Chicca E, Liu YH. An Event-based Neural Compressive Telemetry with >11× Loss-less Data Reduction for High-bandwidth Intracortical Brain Computer Interfaces. IEEE Trans Biomed Circuits Syst 2024; PP:1-10. [PMID: 38498746 DOI: 10.1109/tbcas.2024.3378973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/20/2024]
Abstract
Intracortical brain-computer interfaces offer superior spatial and temporal resolutions, but face challenges as the increasing number of recording channels introduces high amounts of data to be transferred. This requires power-hungry data serialization and telemetry, leading to potential tissue damage risks. To address this challenge, this paper introduces an event-based neural compressive telemetry (NCT) consisting of 8 channel-rotating Δ-ADCs, an event-driven serializer supporting a proposed ternary address event representation protocol, and an event-based LVDS driver. Leveraging a high sparsity of extracellular spikes and high spatial correlation of the high-density recordings, the proposed NCT achieves a compression ratio of >11.4×, while consumes only 1 μW per channel, which is 127× more efficient than state of the art. The NCT well preserves the spike waveform fidelity, and has a low normalized RMS error <23% even with a spike amplitude down to only 31 μV.
Collapse
|
3
|
Schoepe T, Janotte E, Milde MB, Bertrand OJN, Egelhaaf M, Chicca E. Finding the gap: neuromorphic motion-vision in dense environments. Nat Commun 2024; 15:817. [PMID: 38280859 PMCID: PMC10821932 DOI: 10.1038/s41467-024-45063-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 01/15/2024] [Indexed: 01/29/2024] Open
Abstract
Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.
Collapse
Affiliation(s)
- Thorben Schoepe
- Peter Grünberg Institut 15, Forschungszentrum Jülich, Aachen, Germany.
- Faculty of Technology and Cognitive Interaction Technology Center of Excellence (CITEC), Bielefeld University, Bielefeld, Germany.
- Bio-Inspired Circuits and Systems (BICS) Lab. Zernike Institute for Advanced Materials (Zernike Inst Adv Mat), University of Groningen, Groningen, Netherlands.
- CogniGron (Groningen Cognitive Systems and Materials Center), University of Groningen, Groningen, Netherlands.
| | - Ella Janotte
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub facility, Genoa, Italy
| | - Moritz B Milde
- International Centre for Neuromorphic Systems, MARCS Institute, Western Sydney University, Penrith, Australia
| | | | - Martin Egelhaaf
- Neurobiology, Faculty of Biology, Bielefeld University, Bielefeld, Germany
| | - Elisabetta Chicca
- Faculty of Technology and Cognitive Interaction Technology Center of Excellence (CITEC), Bielefeld University, Bielefeld, Germany
- Bio-Inspired Circuits and Systems (BICS) Lab. Zernike Institute for Advanced Materials (Zernike Inst Adv Mat), University of Groningen, Groningen, Netherlands
- CogniGron (Groningen Cognitive Systems and Materials Center), University of Groningen, Groningen, Netherlands
| |
Collapse
|
4
|
He Y, Corradi F, Shi C, van der Ven S, Timmermans M, Stuijt J, Detterer P, Harpe P, Lindeboom L, Hermeling E, Langereis G, Chicca E, Liu YH. An Implantable Neuromorphic Sensing System Featuring Near-sensor Computation and Send-on-Delta Transmission for Wireless Neural Sensing of Peripheral Nerves. IEEE J Solid-State Circuits 2022; 57:3058-3070. [PMID: 36741239 PMCID: PMC7614138 DOI: 10.1109/jssc.2022.3193846] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
This paper presents a bio-inspired event-driven neuromorphic sensing system (NSS) capable of performing on-chip feature extraction and "send-on-delta" pulse-based transmission, targeting peripheral-nerve neural recording applications. The proposed NSS employs event-based sampling which, by leveraging the sparse nature of electroneurogram (ENG) signals, achieves a data compression ratio of >125×, while maintaining a low normalized RMS error of 4% after reconstruction. The proposed NSS consists of three sub-circuits. A clockless level-crossing (LC) ADC with background offset calibration has been employed to reduce the data rate, while maintaining a high signal to quantization noise ratio. A fully synthesized spiking neural network (SNN) extracts temporal features of compound action potential signals consumes only 13 μW. An event-driven pulse-based body channel communication (Pulse-BCC) with serialized address-event representation encoding (AER) schemes minimizes transmission energy and form factor. The prototype is fabricated in 40-nm CMOS occupying a 0.32-mm2 active area and consumes in total 28.2 μW and 50 μW power in feature extraction and full diagnosis mode, respectively. The presented NSS also extracts temporal features of compound action potential signals with 10-μs precision.
Collapse
|
5
|
Gutierrez-Galan D, Schoepe T, Dominguez-Morales JP, Jimenez-Fernandez A, Chicca E, Linares-Barranco A. An Event-Based Digital Time Difference Encoder Model Implementation for Neuromorphic Systems. IEEE Trans Neural Netw Learn Syst 2022; 33:1959-1973. [PMID: 34495850 DOI: 10.1109/tnnls.2021.3108047] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption.
Collapse
|
6
|
Sengupta D, Mastella M, Chicca E, Kottapalli AGP. Skin-Inspired Flexible and Stretchable Electrospun Carbon Nanofiber Sensors for Neuromorphic Sensing. ACS Appl Electron Mater 2022; 4:308-315. [PMID: 35098136 PMCID: PMC8793024 DOI: 10.1021/acsaelm.1c01010] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Accepted: 12/20/2021] [Indexed: 06/14/2023]
Abstract
During the past few decades, a significant amount of research effort has been dedicated toward developing skin-inspired sensors for real-time human motion monitoring and next-generation robotic devices. Although several flexible and wearable sensors have been developed in the past, the need of the hour is developing accurate, reliable, sophisticated, facile yet inexpensive flexible sensors coupled with neuromorphic systems or spiking neural networks to encode tactile information without the need for complex digital architectures, thus achieving true skin-like sensing with limited resources. In this work, we propose an approach entailing carbon nanofiber-polydimethylsiloxane composite-based piezoresistive sensors, coupled with spiking neural networks, to mimic skin-like sensing. The strain and pressure sensors have been combined with appropriately designed neural networks to encode analog voltages to spikes to recreate bioinspired tactile sensing and proprioception. To further validate the proprioceptive capability of the system, a gesture tracking smart glove, combined with a spiking neural network, was demonstrated. Wearable and flexible sensors with accompanying neural networks such as the ones proposed in this work will pave the way for a future generation of skin-mimetic sensors for advanced prosthetic devices, apparel integrable smart sensors for human motion monitoring, and human-machine interfaces.
Collapse
Affiliation(s)
- Debarun Sengupta
- Department
of Advanced Production Engineering (APE), Engineering and Technology
Institute Groningen (ENTEG), University
of Groningen, Groningen 9747 AG, The Netherlands
| | - Michele Mastella
- Groningen
Cognitive Systems and Materials Center (CogniGron), University of Groningen, Groningen 9747 AG, The Netherlands
- Bio-Inspired
Circuits and Systems (BICS) Laboratory, Zernike Institute for Advanced
Materials (Zernike Inst Adv Mat), University
of Groningen, Nijenborgh
4, Groningen NL-9747 AG, Netherlands
| | - Elisabetta Chicca
- Groningen
Cognitive Systems and Materials Center (CogniGron), University of Groningen, Groningen 9747 AG, The Netherlands
- Bio-Inspired
Circuits and Systems (BICS) Laboratory, Zernike Institute for Advanced
Materials (Zernike Inst Adv Mat), University
of Groningen, Nijenborgh
4, Groningen NL-9747 AG, Netherlands
| | - Ajay Giri Prakash Kottapalli
- Department
of Advanced Production Engineering (APE), Engineering and Technology
Institute Groningen (ENTEG), University
of Groningen, Groningen 9747 AG, The Netherlands
- MIT
Sea Grant College Program, Massachusetts
Institute of Technology (MIT), 77 Massachusetts Avenue, NW98-151, Cambridge, Massachusetts 02139, United States
| |
Collapse
|
7
|
Linares-Barranco A, Perez-Peña F, Jimenez-Fernandez A, Chicca E. ED-BioRob: A Neuromorphic Robotic Arm With FPGA-Based Infrastructure for Bio-Inspired Spiking Motor Controllers. Front Neurorobot 2020; 14:590163. [PMID: 33328951 PMCID: PMC7735321 DOI: 10.3389/fnbot.2020.590163] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 10/27/2020] [Indexed: 11/13/2022] Open
Abstract
Compared to classic robotics, biological nervous systems respond to stimuli in a fast and efficient way regarding the body motor actions. Decision making, once the sensory information arrives to the brain, is in the order of ms, while the whole process from sensing to movement requires tens of ms. Classic robotic systems usually require complex computational abilities. Key differences between biological systems and robotic machines lie in the way information is coded and transmitted. A neuron is the "basic" element that constitutes biological nervous systems. Neurons communicate in an event-driven way through small currents or ionic pulses (spikes). When neurons are arranged in networks, they allow not only for the processing of sensory information, but also for the actuation over the muscles in the same spiking manner. This paper presents the application of a classic motor control model (proportional-integral-derivative) developed with the biological spike processing principle, including the motor actuation with time enlarged spikes instead of the classic pulse-width-modulation. This closed-loop control model, called spike-based PID controller (sPID), was improved and adapted for a dual FPGA-based system to control the four joints of a bioinspired light robot (BioRob X5), called event-driven BioRob (ED-BioRob). The use of spiking signals allowed the system to achieve a current consumption bellow 1A for the entire 4 DoF working at the same time. Furthermore, the robot joints commands can be received from a population of silicon-neurons running on the Dynap-SE platform. Thus, our proposal aims to bridge the gap between a general purpose processing analog neuromorphic hardware and the spiking actuation of a robotic platform.
Collapse
Affiliation(s)
- Alejandro Linares-Barranco
- Robotics and Technology of Computers Lab (ETSII-EPS), Universidad de Sevilla, Sevilla, Spain.,Smart Computer Systems Researh and Engineering Lab (SCORE), Research Institute of Computer Engineering (I3US), Universidad de Sevilla, Sevilla, Spain
| | | | - Angel Jimenez-Fernandez
- Robotics and Technology of Computers Lab (ETSII-EPS), Universidad de Sevilla, Sevilla, Spain.,Smart Computer Systems Researh and Engineering Lab (SCORE), Research Institute of Computer Engineering (I3US), Universidad de Sevilla, Sevilla, Spain
| | - Elisabetta Chicca
- Faculty of Technology and Cognitive Interaction Technology Center of Excellence (CITEC) - Bielefeld University, Bielefeld, Germany.,Bio-Inspired Circuits and Systems Lab (BICS), Zernike Institute for Advanced Materials, University of Groningen, Groningen, Netherlands.,Groningen Cognitive Systems and Materials Center (CogniGron), University of Groningen, Groningen, Netherlands
| |
Collapse
|
8
|
D'Angelo G, Janotte E, Schoepe T, O'Keeffe J, Milde MB, Chicca E, Bartolozzi C. Event-Based Eccentric Motion Detection Exploiting Time Difference Encoding. Front Neurosci 2020; 14:451. [PMID: 32457575 PMCID: PMC7227134 DOI: 10.3389/fnins.2020.00451] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2019] [Accepted: 04/14/2020] [Indexed: 11/13/2022] Open
Abstract
Attentional selectivity tends to follow events considered as interesting stimuli. Indeed, the motion of visual stimuli present in the environment attract our attention and allow us to react and interact with our surroundings. Extracting relevant motion information from the environment presents a challenge with regards to the high information content of the visual input. In this work we propose a novel integration between an eccentric down-sampling of the visual field, taking inspiration from the varying size of receptive fields (RFs) in the mammalian retina, and the Spiking Elementary Motion Detector (sEMD) model. We characterize the system functionality with simulated data and real world data collected with bio-inspired event driven cameras, successfully implementing motion detection along the four cardinal directions and diagonally.
Collapse
Affiliation(s)
- Giulia D'Angelo
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub Facility, Genoa, Italy
| | - Ella Janotte
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - Thorben Schoepe
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - James O'Keeffe
- Biosciences Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Moritz B Milde
- International Centre for Neuromorphic Systems, The MARCS Institute, Western Sydney University, Sydney, NSW, Australia
| | - Elisabetta Chicca
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| | - Chiara Bartolozzi
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub Facility, Genoa, Italy
| |
Collapse
|
9
|
Kugele A, Pfeil T, Pfeiffer M, Chicca E. Efficient Processing of Spatio-Temporal Data Streams With Spiking Neural Networks. Front Neurosci 2020; 14:439. [PMID: 32431592 PMCID: PMC7214871 DOI: 10.3389/fnins.2020.00439] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Accepted: 04/09/2020] [Indexed: 11/15/2022] Open
Abstract
Spiking neural networks (SNNs) are potentially highly efficient models for inference on fully parallel neuromorphic hardware, but existing training methods that convert conventional artificial neural networks (ANNs) into SNNs are unable to exploit these advantages. Although ANN-to-SNN conversion has achieved state-of-the-art accuracy for static image classification tasks, the following subtle but important difference in the way SNNs and ANNs integrate information over time makes the direct application of conversion techniques for sequence processing tasks challenging. Whereas all connections in SNNs have a certain propagation delay larger than zero, ANNs assign different roles to feed-forward connections, which immediately update all neurons within the same time step, and recurrent connections, which have to be rolled out in time and are typically assigned a delay of one time step. Here, we present a novel method to obtain highly accurate SNNs for sequence processing by modifying the ANN training before conversion, such that delays induced by ANN rollouts match the propagation delays in the targeted SNN implementation. Our method builds on the recently introduced framework of streaming rollouts, which aims for fully parallel model execution of ANNs and inherently allows for temporal integration by merging paths of different delays between input and output of the network. The resulting networks achieve state-of-the-art accuracy for multiple event-based benchmark datasets, including N-MNIST, CIFAR10-DVS, N-CARS, and DvsGesture, and through the use of spatio-temporal shortcut connections yield low-latency approximate network responses that improve over time as more of the input sequence is processed. In addition, our converted SNNs are consistently more energy-efficient than their corresponding ANNs.
Collapse
Affiliation(s)
- Alexander Kugele
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
- Bosch Center for Artificial Intelligence, Renningen, Germany
| | - Thomas Pfeil
- Bosch Center for Artificial Intelligence, Renningen, Germany
| | | | - Elisabetta Chicca
- Faculty of Technology and Center of Cognitive Interaction Technology (CITEC), Bielefeld University, Bielefeld, Germany
| |
Collapse
|
10
|
Thakur CS, Molin JL, Cauwenberghs G, Indiveri G, Kumar K, Qiao N, Schemmel J, Wang R, Chicca E, Hasler JO, Seo JS, Yu S, Cao Y, van Schaik A, Etienne-Cummings R. Corrigendum: Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain. Front Neurosci 2019; 12:991. [PMID: 30666180 PMCID: PMC6330659 DOI: 10.3389/fnins.2018.00991] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2018] [Accepted: 12/10/2018] [Indexed: 11/13/2022] Open
Affiliation(s)
- Chetan Singh Thakur
- Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore, India
| | - Jamal Lottier Molin
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Gert Cauwenberghs
- Department of Bioengineering and Institute for Neural Computation, University of California, San Diego, La Jolla, CA, United States
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Kundan Kumar
- Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore, India
| | - Ning Qiao
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Johannes Schemmel
- Kirchhoff Institute for Physics, University of Heidelberg, Heidelberg, Germany
| | - Runchun Wang
- The MARCS Institute, Western Sydney University, Kingswood, NSW, Australia
| | - Elisabetta Chicca
- Cognitive Interaction Technology - Center of Excellence, Bielefeld University, Bielefeld, Germany
| | - Jennifer Olson Hasler
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | - Jae-Sun Seo
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - Shimeng Yu
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - Yu Cao
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - André van Schaik
- The MARCS Institute, Western Sydney University, Kingswood, NSW, Australia
| | - Ralph Etienne-Cummings
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
| |
Collapse
|
11
|
Thakur CS, Molin JL, Cauwenberghs G, Indiveri G, Kumar K, Qiao N, Schemmel J, Wang R, Chicca E, Olson Hasler J, Seo JS, Yu S, Cao Y, van Schaik A, Etienne-Cummings R. Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain. Front Neurosci 2018; 12:891. [PMID: 30559644 PMCID: PMC6287454 DOI: 10.3389/fnins.2018.00891] [Citation(s) in RCA: 59] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2018] [Accepted: 11/14/2018] [Indexed: 11/16/2022] Open
Abstract
Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design systems capable of brain-like computation. Numerous large-scale neuromorphic projects have emerged recently. This interdisciplinary field was listed among the top 10 technology breakthroughs of 2014 by the MIT Technology Review and among the top 10 emerging technologies of 2015 by the World Economic Forum. NE has two-way goals: one, a scientific goal to understand the computational properties of biological neural systems by using models implemented in integrated circuits (ICs); second, an engineering goal to exploit the known properties of biological systems to design and implement efficient devices for engineering applications. Building hardware neural emulators can be extremely useful for simulating large-scale neural models to explain how intelligent behavior arises in the brain. The principal advantages of neuromorphic emulators are that they are highly energy efficient, parallel and distributed, and require a small silicon area. Thus, compared to conventional CPUs, these neuromorphic emulators are beneficial in many engineering applications such as for the porting of deep learning algorithms for various recognitions tasks. In this review article, we describe some of the most significant neuromorphic spiking emulators, compare the different architectures and approaches used by them, illustrate their advantages and drawbacks, and highlight the capabilities that each can deliver to neural modelers. This article focuses on the discussion of large-scale emulators and is a continuation of a previous review of various neural and synapse circuits (Indiveri et al., 2011). We also explore applications where these emulators have been used and discuss some of their promising future applications.
Collapse
Affiliation(s)
- Chetan Singh Thakur
- Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore, India
| | - Jamal Lottier Molin
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Gert Cauwenberghs
- Department of Bioengineering and Institute for Neural Computation, University of California, San Diego, La Jolla, CA, United States
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Kundan Kumar
- Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore, India
| | - Ning Qiao
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Johannes Schemmel
- Kirchhoff Institute for Physics, University of Heidelberg, Heidelberg, Germany
| | - Runchun Wang
- The MARCS Institute, Western Sydney University, Kingswood, NSW, Australia
| | - Elisabetta Chicca
- Cognitive Interaction Technology – Center of Excellence, Bielefeld University, Bielefeld, Germany
| | - Jennifer Olson Hasler
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | - Jae-sun Seo
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - Shimeng Yu
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - Yu Cao
- School of Electrical, Computer and Engineering, Arizona State University, Tempe, AZ, United States
| | - André van Schaik
- The MARCS Institute, Western Sydney University, Kingswood, NSW, Australia
| | - Ralph Etienne-Cummings
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
| |
Collapse
|
12
|
Mulaosmanovic H, Chicca E, Bertele M, Mikolajick T, Slesazeck S. Mimicking biological neurons with a nanoscale ferroelectric transistor. Nanoscale 2018; 10:21755-21763. [PMID: 30431045 DOI: 10.1039/c8nr07135g] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Neuron is the basic computing unit in brain-inspired neural networks. Although a multitude of excellent artificial neurons realized with conventional transistors have been proposed, they might not be energy and area efficient in large-scale networks. The recent discovery of ferroelectricity in hafnium oxide (HfO2) and the related switching phenomena at the nanoscale might provide a solution. This study employs the newly reported accumulative polarization reversal in nanoscale HfO2-based ferroelectric field-effect transistors (FeFETs) to implement two key neuronal dynamics: the integration of action potentials and the subsequent firing according to the biologically plausible all-or-nothing law. We show that by carefully shaping electrical excitations based on the particular nucleation-limited switching kinetics of the ferroelectric layer further neuronal behaviors can be emulated, such as firing activity tuning, arbitrary refractory period and the leaky effect. Finally, we discuss the advantages of an FeFET-based neuron, highlighting its transferability to advanced scaling technologies and the beneficial impact it may have in reducing the complexity of neuromorphic circuits.
Collapse
|
13
|
Abstract
Apparent motion of the surroundings on an agent's retina can be used to navigate through cluttered environments, avoid collisions with obstacles, or track targets of interest. The pattern of apparent motion of objects, (i.e., the optic flow), contains spatial information about the surrounding environment. For a small, fast-moving agent, as used in search and rescue missions, it is crucial to estimate the distance to close-by objects to avoid collisions quickly. This estimation cannot be done by conventional methods, such as frame-based optic flow estimation, given the size, power, and latency constraints of the necessary hardware. A practical alternative makes use of event-based vision sensors. Contrary to the frame-based approach, they produce so-called events only when there are changes in the visual scene. We propose a novel asynchronous circuit, the spiking elementary motion detector (sEMD), composed of a single silicon neuron and synapse, to detect elementary motion from an event-based vision sensor. The sEMD encodes the time an object's image needs to travel across the retina into a burst of spikes. The number of spikes within the burst is proportional to the speed of events across the retina. A fast but imprecise estimate of the time-to-travel can already be obtained from the first two spikes of a burst and refined by subsequent interspike intervals. The latter encoding scheme is possible due to an adaptive nonlinear synaptic efficacy scaling. We show that the sEMD can be used to compute a collision avoidance direction in the context of robotic navigation in a cluttered outdoor environment and compared the collision avoidance direction to a frame-based algorithm. The proposed computational principle constitutes a generic spiking temporal correlation detector that can be applied to other sensory modalities (e.g., sound localization), and it provides a novel perspective to gating information in spiking neural networks.
Collapse
Affiliation(s)
- M B Milde
- Institute of Neuroinformatics, University of Zurich, and ETH Zurich, 8057 Zurich, Switzerland
| | - O J N Bertrand
- Neurobiology, Faculty of Biology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - H Ramachandran
- Faculty of Technology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - M Egelhaaf
- Neurobiology, Faculty of Biology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - E Chicca
- Faculty of Technology, Bielefeld University, 33615 Bielefeld, Germany, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| |
Collapse
|
14
|
Engelmann J, Walther T, Grant K, Chicca E, Gómez-Sena L. Modeling latency code processing in the electric sense: from the biological template to its VLSI implementation. Bioinspir Biomim 2016; 11:055007. [PMID: 27623047 DOI: 10.1088/1748-3190/11/5/055007] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Understanding the coding of sensory information under the temporal constraints of natural behavior is not yet well resolved. There is a growing consensus that spike timing or latency coding can maximally exploit the timing of neural events to make fast computing elements and that such mechanisms are essential to information processing functions in the brain. The electric sense of mormyrid fish provides a convenient biological model where this coding scheme can be studied. The sensory input is a physically ordered spatial pattern of current densities, which is coded in the precise timing of primary afferent spikes. The neural circuits of the processing pathway are well known and the system exhibits the best known illustration of corollary discharge, which provides the reference to decoding the sensory afferent latency pattern. A theoretical model has been constructed from available electrophysiological and neuroanatomical data to integrate the principal traits of the neural processing structure and to study sensory interaction with motor-command-driven corollary discharge signals. This has been used to explore neural coding strategies at successive stages in the network and to examine the simulated network capacity to reproduce output neuron responses. The model shows that the network has the ability to resolve primary afferent spike timing differences in the sub-millisecond range, and that this depends on the coincidence of sensory and corollary discharge-driven gating signals. In the integrative and output stages of the network, corollary discharge sets up a proactive background filter, providing temporally structured excitation and inhibition within the network whose balance is then modulated locally by sensory input. This complements the initial gating mechanism and contributes to amplification of the input pattern of latencies, conferring network hyperacuity. These mechanisms give the system a robust capacity to extract behaviorally meaningful features of the electric image with high sensitivity over a broad working range. Since the network largely depends on spike timing, we finally discuss its suitability for implementation in robotic applications based on neuromorphic hardware.
Collapse
Affiliation(s)
- Jacob Engelmann
- Bielefeld University, Faculty of Biology/CITEC, AG Active Sensing, Universitätsstraße 25, 33615 Bielefeld, Germany
| | | | | | | | | |
Collapse
|
15
|
Affiliation(s)
- Christian G Mayr
- Chair of Highly-Parallel VLSI-Systems and Neuromorphic Circuits, Technische Universität Dresden Dresden, Germany
| | - Sadique Sheik
- BioCircuits Institute, University of California, San Diego San Diego, CA, USA
| | | | - Elisabetta Chicca
- Cognitive Interaction Technology - Center of Excellence, Faculty of Technology, Bielefeld University Bielefeld, Germany
| |
Collapse
|
16
|
Thomas A, Niehörster S, Fabretti S, Shepheard N, Kuschel O, Küpper K, Wollschläger J, Krzysteczko P, Chicca E. Tunnel junction based memristors as artificial synapses. Front Neurosci 2015; 9:241. [PMID: 26217173 PMCID: PMC4493388 DOI: 10.3389/fnins.2015.00241] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2014] [Accepted: 06/24/2015] [Indexed: 11/30/2022] Open
Abstract
We prepared magnesia, tantalum oxide, and barium titanate based tunnel junction structures and investigated their memristive properties. The low amplitudes of the resistance change in these types of junctions are the major obstacle for their use. Here, we increased the amplitude of the resistance change from 10% up to 100%. Utilizing the memristive properties, we looked into the use of the junction structures as artificial synapses. We observed analogs of long-term potentiation, long-term depression and spike-time dependent plasticity in these simple two terminal devices. Finally, we suggest a possible pathway of these devices toward their integration in neuromorphic systems for storing analog synaptic weights and supporting the implementation of biologically plausible learning mechanisms.
Collapse
Affiliation(s)
- Andy Thomas
- Thin Films and Physics of Nanostructures, Bielefeld UniversityBielefeld, Germany
- IFW Dresden, Institute for Metallic MaterialsDresden, Germany
| | - Stefan Niehörster
- Thin Films and Physics of Nanostructures, Bielefeld UniversityBielefeld, Germany
| | - Savio Fabretti
- Thin Films and Physics of Nanostructures, Bielefeld UniversityBielefeld, Germany
| | - Norman Shepheard
- Thin Films and Physics of Nanostructures, Bielefeld UniversityBielefeld, Germany
- Cognitive Interaction Technology Center of Excellence and Faculty of Technology, Bielefeld UniversityBielefeld, Germany
| | - Olga Kuschel
- Fachbereich Physik and Center of Physics and Chemistry of New Materials, Osnabrück UniversityOsnabrück, Germany
| | - Karsten Küpper
- Fachbereich Physik and Center of Physics and Chemistry of New Materials, Osnabrück UniversityOsnabrück, Germany
| | - Joachim Wollschläger
- Fachbereich Physik and Center of Physics and Chemistry of New Materials, Osnabrück UniversityOsnabrück, Germany
| | - Patryk Krzysteczko
- Thin Films and Physics of Nanostructures, Bielefeld UniversityBielefeld, Germany
- Physikalisch Technische BundesanstaltBraunschweig, Germany
| | - Elisabetta Chicca
- Cognitive Interaction Technology Center of Excellence and Faculty of Technology, Bielefeld UniversityBielefeld, Germany
| |
Collapse
|
17
|
|
18
|
Coath M, Sheik S, Chicca E, Indiveri G, Denham SL, Wennekers T. A robust sound perception model suitable for neuromorphic implementation. Front Neurosci 2014; 7:278. [PMID: 24478621 PMCID: PMC3894459 DOI: 10.3389/fnins.2013.00278] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2013] [Accepted: 12/30/2013] [Indexed: 11/30/2022] Open
Abstract
We have recently demonstrated the emergence of dynamic feature sensitivity through exposure to formative stimuli in a real-time neuromorphic system implementing a hybrid analog/digital network of spiking neurons. This network, inspired by models of auditory processing in mammals, includes several mutually connected layers with distance-dependent transmission delays and learning in the form of spike timing dependent plasticity, which effects stimulus-driven changes in the network connectivity. Here we present results that demonstrate that the network is robust to a range of variations in the stimulus pattern, such as are found in naturalistic stimuli and neural responses. This robustness is a property critical to the development of realistic, electronic neuromorphic systems. We analyze the variability of the response of the network to “noisy” stimuli which allows us to characterize the acuity in information-theoretic terms. This provides an objective basis for the quantitative comparison of networks, their connectivity patterns, and learning strategies, which can inform future design decisions. We also show, using stimuli derived from speech samples, that the principles are robust to other challenges, such as variable presentation rate, that would have to be met by systems deployed in the real world. Finally we demonstrate the potential applicability of the approach to real sounds.
Collapse
Affiliation(s)
- Martin Coath
- Cognition Institute, Plymouth University Plymouth, UK ; Faculty of Health and Human Sciences, School of Psychology, Plymouth University Plymouth, UK
| | - Sadique Sheik
- Institute of Neuroinformatics, University of Zurich and ETH Zurich Zurich, Switzerland
| | - Elisabetta Chicca
- Faculty of Technology, Cognitive Interaction Technology - Center of Excellence, Bielefeld University Bielefeld, Germany
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich Zurich, Switzerland
| | - Susan L Denham
- Cognition Institute, Plymouth University Plymouth, UK ; Faculty of Health and Human Sciences, School of Psychology, Plymouth University Plymouth, UK
| | - Thomas Wennekers
- Cognition Institute, Plymouth University Plymouth, UK ; Faculty of Science and Environment, School of Computing and Mathematics, Plymouth University Plymouth, UK
| |
Collapse
|
19
|
Sheik S, Coath M, Indiveri G, Denham SL, Wennekers T, Chicca E. Emergent Auditory Feature Tuning in a Real-Time Neuromorphic VLSI System. Front Neurosci 2012; 6:17. [PMID: 22347163 PMCID: PMC3272652 DOI: 10.3389/fnins.2012.00017] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2011] [Accepted: 01/19/2012] [Indexed: 11/29/2022] Open
Abstract
Many sounds of ecological importance, such as communication calls, are characterized by time-varying spectra. However, most neuromorphic auditory models to date have focused on distinguishing mainly static patterns, under the assumption that dynamic patterns can be learned as sequences of static ones. In contrast, the emergence of dynamic feature sensitivity through exposure to formative stimuli has been recently modeled in a network of spiking neurons based on the thalamo-cortical architecture. The proposed network models the effect of lateral and recurrent connections between cortical layers, distance-dependent axonal transmission delays, and learning in the form of Spike Timing Dependent Plasticity (STDP), which effects stimulus-driven changes in the pattern of network connectivity. In this paper we demonstrate how these principles can be efficiently implemented in neuromorphic hardware. In doing so we address two principle problems in the design of neuromorphic systems: real-time event-based asynchronous communication in multi-chip systems, and the realization in hybrid analog/digital VLSI technology of neural computational principles that we propose underlie plasticity in neural processing of dynamic stimuli. The result is a hardware neural network that learns in real-time and shows preferential responses, after exposure, to stimuli exhibiting particular spectro-temporal patterns. The availability of hardware on which the model can be implemented, makes this a significant step toward the development of adaptive, neurobiologically plausible, spike-based, artificial sensory systems.
Collapse
Affiliation(s)
- Sadique Sheik
- Institute of Neuroinformatics, University of Zurich and ETH Zurich Zurich, Switzerland
| | | | | | | | | | | |
Collapse
|
20
|
Abstract
An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.
Collapse
Affiliation(s)
- Emre Neftci
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Elisabetta Chicca
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Giacomo Indiveri
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| | - Rodney Douglas
- Institute of Neuroinformatics, ETH, and University of Zurich, Zurich 8057, Switzerland
| |
Collapse
|
21
|
Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, Del Giudice P, Douglas RJ, Indiveri G. A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2007.893509] [Citation(s) in RCA: 93] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
22
|
Abstract
We present a mixed-mode analog/digital VLSI device comprising an array of leaky integrate-and-fire (I&F) neurons, adaptive synapses with spike-timing dependent plasticity, and an asynchronous event based communication infrastructure that allows the user to (re)configure networks of spiking neurons with arbitrary topologies. The asynchronous communication protocol used by the silicon neurons to transmit spikes (events) off-chip and the silicon synapses to receive spikes from the outside is based on the "address-event representation" (AER). We describe the analog circuits designed to implement the silicon neurons and synapses and present experimental data showing the neuron's response properties and the synapses characteristics, in response to AER input spike trains. Our results indicate that these circuits can be used in massively parallel VLSI networks of I&F neurons to simulate real-time complex spike-based learning algorithms.
Collapse
Affiliation(s)
- Giacomo Indiveri
- Institute of Neuroinformatics, Swiss Federal Institute of Technology, Zurich CH-8057, Switzerland.
| | | | | |
Collapse
|
23
|
Chicca E, Badoni D, Dante V, D'Andreagiovanni M, Salina G, Carota L, Fusi S, Del Giudice P. A vlsi recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory. ACTA ACUST UNITED AC 2003; 14:1297-307. [DOI: 10.1109/tnn.2003.816367] [Citation(s) in RCA: 135] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|