1
|
Dai Z, Fu Q, Peng J, Li H. SLoN: a spiking looming perception network exploiting neural encoding and processing in ON/OFF channels. Front Neurosci 2024; 18:1291053. [PMID: 38510466 PMCID: PMC10950957 DOI: 10.3389/fnins.2024.1291053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 02/14/2024] [Indexed: 03/22/2024] Open
Abstract
Looming perception, the ability to sense approaching objects, is crucial for the survival of humans and animals. After hundreds of millions of years of evolutionary development, biological entities have evolved efficient and robust looming perception visual systems. However, current artificial vision systems fall short of such capabilities. In this study, we propose a novel spiking neural network for looming perception that mimics biological vision to communicate motion information through action potentials or spikes, providing a more realistic approach than previous artificial neural networks based on sum-then-activate operations. The proposed spiking looming perception network (SLoN) comprises three core components. Neural encoding, known as phase coding, transforms video signals into spike trains, introducing the concept of phase delay to depict the spatial-temporal competition between phasic excitatory and inhibitory signals shaping looming selectivity. To align with biological substrates where visual signals are bifurcated into parallel ON/OFF channels encoding brightness increments and decrements separately to achieve specific selectivity to ON/OFF-contrast stimuli, we implement eccentric down-sampling at the entrance of ON/OFF channels, mimicking the foveal region of the mammalian receptive field with higher acuity to motion, computationally modeled with a leaky integrate-and-fire (LIF) neuronal network. The SLoN model is deliberately tested under various visual collision scenarios, ranging from synthetic to real-world stimuli. A notable achievement is that the SLoN selectively spikes for looming features concealed in visual streams against other categories of movements, including translating, receding, grating, and near misses, demonstrating robust selectivity in line with biological principles. Additionally, the efficacy of the ON/OFF channels, the phase coding with delay, and the eccentric visual processing are further investigated to demonstrate their effectiveness in looming perception. The cornerstone of this study rests upon showcasing a new paradigm for looming perception that is more biologically plausible in light of biological motion perception.
Collapse
|
2
|
Fu Q, Li Z, Peng J. Harmonizing motion and contrast vision for robust looming detection. ARRAY 2023. [DOI: 10.1016/j.array.2022.100272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
|
3
|
Wang Y, Li H, Zheng Y, Peng J. A directionally selective collision-sensing visual neural network based on fractional-order differential operator. Front Neurorobot 2023; 17:1149675. [PMID: 37152416 PMCID: PMC10160397 DOI: 10.3389/fnbot.2023.1149675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 03/30/2023] [Indexed: 05/09/2023] Open
Abstract
In this paper, we propose a directionally selective fractional-order lobular giant motion detector (LGMD) visual neural network. Unlike most collision-sensing network models based on LGMDs, our model can not only sense collision threats but also obtain the motion direction of the collision object. Firstly, this paper simulates the membrane potential response of neurons using the fractional-order differential operator to generate reliable collision response spikes. Then, a new correlation mechanism is proposed to obtain the motion direction of objects. Specifically, this paper performs correlation operation on the signals extracted from two pixels, utilizing the temporal delay of the signals to obtain their position relationship. In this way, the response characteristics of direction-selective neurons can be characterized. Finally, ON/OFF visual channels are introduced to encode increases and decreases in brightness, respectively, thereby modeling the bipolar response of special neurons. Extensive experimental results show that the proposed visual neural system conforms to the response characteristics of biological LGMD and direction-selective neurons, and that the performance of the system is stable and reliable.
Collapse
|
4
|
Wen C, Horiuchi TK. The Curved Openspace Algorithm and a Spike-Latency Model for Sonar-Based Obstacle Avoidance. Front Neurorobot 2022; 16:850013. [PMID: 35721278 PMCID: PMC9198458 DOI: 10.3389/fnbot.2022.850013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Accepted: 05/04/2022] [Indexed: 11/13/2022] Open
Abstract
The rapid control of a sonar-guided vehicle to pursue a goal while avoiding obstacles has been a persistent research topic for decades. Taking into account the limited field-of-view of practical sonar systems and vehicle kinematics, we propose a neural model for obstacle avoidance that maps the 2-D sensory space into a 1-D motor space and evaluates motor actions while combining obstacles and goal information. A two-stage winner-take-all (WTA) mechanism is used to select the final steering action. To avoid excessive scanning of the environment, an attentional system is proposed to control the directions of sonar pings for efficient, task-driven, sensory data collection. A mobile robot was used to test the proposed model navigating through a cluttered environment using a narrow field-of-view sonar system. We further propose a spiking neural model using spike-timing representations, a spike-latency memory, and a “race-to-first-spike” WTA circuit.
Collapse
Affiliation(s)
- Chenxi Wen
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, United States
| | - Timothy K. Horiuchi
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, United States
- Neuroscience and Cognitive Sciences Program, University of Maryland, College Park, MD, United States
- *Correspondence: Timothy K. Horiuchi
| |
Collapse
|
5
|
Xing D, Li J, Zhang T, Xu B. A Brain-Inspired Approach for Collision-Free Movement Planning in the Small Operational Space. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:2094-2105. [PMID: 34520379 DOI: 10.1109/tnnls.2021.3111051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
In a small operational space, e.g., mesoscale or microscale, we need to control movements carefully because of fragile objects. This article proposes a novel structure based on spiking neural networks to imitate the joint function of multiple brain regions in visual guiding in the small operational space and offers two channels to achieve collision-free movements. For the state sensation, we simulate the primary visual cortex to directly extract features from multiple input images and the high-level visual cortex to obtain the object distance, which is indirectly measurable, in the Cartesian coordinates. Our approach emulates the prefrontal cortex from two aspects: multiple liquid state machines to predict distances of the next several steps based on the preceding trajectory and a block-based excitation-inhibition feedforward network to plan movements considering the target and prediction. Responding to "too close" states needs rich temporal information, and we leverage a cerebellar network for the subconscious reaction. From the viewpoint of the inner pathway, they also form two channels. One channel starts from state extraction to attraction movement planning, both in the camera coordinates, behaving visual-servo control. The other is the collision-avoidance channel, which calculates distances, predicts trajectories, and reacts to the repulsion, all in the Cartesian coordinates. We provide appropriate supervised signals for coarse training and apply reinforcement learning to modify synapses in accordance with reality. Simulation and experiment results validate the proposed method.
Collapse
|
6
|
Abstract
The design of robots that interact autonomously with the environment and exhibit complex behaviours is an open challenge that can benefit from understanding what makes living beings fit to act in the world. Neuromorphic engineering studies neural computational principles to develop technologies that can provide a computing substrate for building compact and low-power processing systems. We discuss why endowing robots with neuromorphic technologies - from perception to motor control - represents a promising approach for the creation of robots which can seamlessly integrate in society. We present initial attempts in this direction, highlight open challenges, and propose actions required to overcome current limitations.
Collapse
Affiliation(s)
- Chiara Bartolozzi
- Event-Driven Perception for Robotics, Istituto Italiano di Tecnologia, via San Quirico 19D, 16163, Genova, Italy.
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Winterthurerstr. 190, 8057, Zurich, Switzerland
| | - Elisa Donati
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Winterthurerstr. 190, 8057, Zurich, Switzerland
| |
Collapse
|
7
|
Milde MB, Afshar S, Xu Y, Marcireau A, Joubert D, Ramesh B, Bethi Y, Ralph NO, El Arja S, Dennler N, van Schaik A, Cohen G. Neuromorphic Engineering Needs Closed-Loop Benchmarks. Front Neurosci 2022; 16:813555. [PMID: 35237122 PMCID: PMC8884247 DOI: 10.3389/fnins.2022.813555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 01/24/2022] [Indexed: 12/02/2022] Open
Abstract
Neuromorphic engineering aims to build (autonomous) systems by mimicking biological systems. It is motivated by the observation that biological organisms—from algae to primates—excel in sensing their environment, reacting promptly to their perils and opportunities. Furthermore, they do so more resiliently than our most advanced machines, at a fraction of the power consumption. It follows that the performance of neuromorphic systems should be evaluated in terms of real-time operation, power consumption, and resiliency to real-world perturbations and noise using task-relevant evaluation metrics. Yet, following in the footsteps of conventional machine learning, most neuromorphic benchmarks rely on recorded datasets that foster sensing accuracy as the primary measure for performance. Sensing accuracy is but an arbitrary proxy for the actual system's goal—taking a good decision in a timely manner. Moreover, static datasets hinder our ability to study and compare closed-loop sensing and control strategies that are central to survival for biological organisms. This article makes the case for a renewed focus on closed-loop benchmarks involving real-world tasks. Such benchmarks will be crucial in developing and progressing neuromorphic Intelligence. The shift towards dynamic real-world benchmarking tasks should usher in richer, more resilient, and robust artificially intelligent systems in the future.
Collapse
|
8
|
Luan H, Fu Q, Zhang Y, Hua M, Chen S, Yue S. A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice. Front Neurosci 2022; 15:787256. [PMID: 35126038 PMCID: PMC8814358 DOI: 10.3389/fnins.2021.787256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 12/23/2021] [Indexed: 11/13/2022] Open
Abstract
Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.
Collapse
Affiliation(s)
- Hao Luan
- School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China
| | - Qinbing Fu
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- Computational Intelligence Laboratory (CIL), School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Yicheng Zhang
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Mu Hua
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
| | - Shengyong Chen
- School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China
| | - Shigang Yue
- Machine Life and Intelligence Research Centre, School of Mathematics and Information Science, Guangzhou University, Guangzhou, China
- Computational Intelligence Laboratory (CIL), School of Computer Science, University of Lincoln, Lincoln, United Kingdom
- *Correspondence: Shigang Yue
| |
Collapse
|
9
|
Krauhausen I, Koutsouras DA, Melianas A, Keene ST, Lieberth K, Ledanseur H, Sheelamanthula R, Giovannitti A, Torricelli F, Mcculloch I, Blom PWM, Salleo A, van de Burgt Y, Gkoupidenis P. Organic neuromorphic electronics for sensorimotor integration and learning in robotics. SCIENCE ADVANCES 2021; 7:eabl5068. [PMID: 34890232 PMCID: PMC8664264 DOI: 10.1126/sciadv.abl5068] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
In living organisms, sensory and motor processes are distributed, locally merged, and capable of forming dynamic sensorimotor associations. We introduce a simple and efficient organic neuromorphic circuit for local sensorimotor merging and processing on a robot that is placed in a maze. While the robot is exposed to external environmental stimuli, visuomotor associations are formed on the adaptable neuromorphic circuit. With this on-chip sensorimotor integration, the robot learns to follow a path to the exit of a maze, while being guided by visually indicated paths. The ease of processability of organic neuromorphic electronics and their unconventional form factors, in combination with education-purpose robotics, showcase a promising approach of an affordable, versatile, and readily accessible platform for exploring, designing, and evaluating behavioral intelligence through decentralized sensorimotor integration.
Collapse
Affiliation(s)
- Imke Krauhausen
- Max Planck Institute for Polymer Research, Mainz, Germany
- Microsystems, Institute for Complex Molecular Systems, Eindhoven University of Technology, Eindhoven, Netherlands
| | | | - Armantas Melianas
- Department of Materials Science and Engineering, Stanford University, Stanford, CA 94305, USA
- Exponent, 149 Commonwealth Dr, Menlo Park, CA 94025, USA
| | - Scott T. Keene
- Department of Engineering, University of Cambridge, Cambridge, UK
| | | | | | - Rajendar Sheelamanthula
- KAUST Solar Center, Physical Science and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
| | - Alexander Giovannitti
- Department of Materials Science and Engineering, Stanford University, Stanford, CA 94305, USA
| | - Fabrizio Torricelli
- Department of Information Engineering, University of Brescia, 25123 Brescia, Italy
| | - Iain Mcculloch
- KAUST Solar Center, Physical Science and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
- Department of Chemistry, University of Oxford, Oxford, UK
| | | | - Alberto Salleo
- Department of Materials Science and Engineering, Stanford University, Stanford, CA 94305, USA
- Corresponding author. (A.S.); (Y.v.d.B); (P.G.)
| | - Yoeri van de Burgt
- Microsystems, Institute for Complex Molecular Systems, Eindhoven University of Technology, Eindhoven, Netherlands
- Corresponding author. (A.S.); (Y.v.d.B); (P.G.)
| | - Paschalis Gkoupidenis
- Max Planck Institute for Polymer Research, Mainz, Germany
- Corresponding author. (A.S.); (Y.v.d.B); (P.G.)
| |
Collapse
|
10
|
Auge D, Hille J, Mueller E, Knoll A. A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks. Neural Process Lett 2021. [DOI: 10.1007/s11063-021-10562-2] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
AbstractBiologically inspired spiking neural networks are increasingly popular in the field of artificial intelligence due to their ability to solve complex problems while being power efficient. They do so by leveraging the timing of discrete spikes as main information carrier. Though, industrial applications are still lacking, partially because the question of how to encode incoming data into discrete spike events cannot be uniformly answered. In this paper, we summarise the signal encoding schemes presented in the literature and propose a uniform nomenclature to prevent the vague usage of ambiguous definitions. Therefore we survey both, the theoretical foundations as well as applications of the encoding schemes. This work provides a foundation in spiking signal encoding and gives an overview over different application-oriented implementations which utilise the schemes.
Collapse
|
11
|
Bhardwaj A, Khyam MO, Müller R. Biomimetic detection of dynamic signatures in foliage echoes. BIOINSPIRATION & BIOMIMETICS 2021; 16:046026. [PMID: 33862609 DOI: 10.1088/1748-3190/abf910] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 04/16/2021] [Indexed: 06/12/2023]
Abstract
Certain bat species (familyRhinolophidae) dynamically deform their emission baffles (noseleaves) and reception baffles (pinnae) during echolocation. Prior research using numerical models, laboratory characterizations, and experiments with simple targets have suggested that this dynamics may manifest itself in time-variant echo signatures. Since the pronounced random nature of echoes from natural targets such as foliage has not been reflected in these experiments, we have collected a large number (>55 000) of foliage echoes outdoors with a sonar head that mimics the dynamic periphery in bats. The echo data was processed with a custom auditory processing model to create spike-based echo representations. Deep-learning classifiers were able to estimate the dynamic state of the periphery, i.e., static or dynamic, based on single echoes with accuracies of up to 80%. This suggests that the effects of the peripheral dynamics are present in the bat brains and could hence be used by the animals. The best classification performances were obtained for data obtained within a spatially confined area. Hence, if the bat brains suffer from the same generalization issues, they would have to have a way to adapt their neural echo processing to such local fluctuations to exploit the dynamic effects successfully.
Collapse
Affiliation(s)
- Ananya Bhardwaj
- Department of Mechanical Engineering, Virginia Tech, Blacksburg 24060, United States of America
| | - M Omar Khyam
- Department of Mechanical Engineering, Virginia Tech, Blacksburg 24060, United States of America
| | - Rolf Müller
- Department of Mechanical Engineering, Virginia Tech, Blacksburg 24060, United States of America
| |
Collapse
|
12
|
Frenkel C, Lefebvre M, Bol D. Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks. Front Neurosci 2021; 15:629892. [PMID: 33642986 PMCID: PMC7902857 DOI: 10.3389/fnins.2021.629892] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 01/06/2021] [Indexed: 11/13/2022] Open
Abstract
While the backpropagation of error algorithm enables deep neural network training, it implies (i) bidirectional synaptic weight transport and (ii) update locking until the forward and backward passes are completed. Not only do these constraints preclude biological plausibility, but they also hinder the development of low-cost adaptive smart sensors at the edge, as they severely constrain memory accesses and entail buffering overhead. In this work, we show that the one-hot-encoded labels provided in supervised classification problems, denoted as targets, can be viewed as a proxy for the error sign. Therefore, their fixed random projections enable a layerwise feedforward training of the hidden layers, thus solving the weight transport and update locking problems while relaxing the computational and memory requirements. Based on these observations, we propose the direct random target projection (DRTP) algorithm and demonstrate that it provides a tradeoff between accuracy and computational cost that is suitable for adaptive edge computing devices.
Collapse
Affiliation(s)
- Charlotte Frenkel
- Institute of Neuroinformatics, University of Zürich and ETH Zürich, Zurich, Switzerland.,ICTEAM Institute, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Martin Lefebvre
- ICTEAM Institute, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - David Bol
- ICTEAM Institute, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| |
Collapse
|
13
|
de Croon GCHE, De Wagter C, Seidl T. Enhancing optical-flow-based control by learning visual appearance cues for flying robots. NAT MACH INTELL 2021. [DOI: 10.1038/s42256-020-00279-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
14
|
Lele A, Fang Y, Ting J, Raychowdhury A. An End-to-end Spiking Neural Network Platform for Edge Robotics: From Event-Cameras to Central Pattern Generation. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3097675] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
15
|
Choi S, Yang J, Wang G. Emerging Memristive Artificial Synapses and Neurons for Energy-Efficient Neuromorphic Computing. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2020; 32:e2004659. [PMID: 33006204 DOI: 10.1002/adma.202004659] [Citation(s) in RCA: 77] [Impact Index Per Article: 19.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2020] [Revised: 08/12/2020] [Indexed: 06/11/2023]
Abstract
Memristors have recently attracted significant interest due to their applicability as promising building blocks of neuromorphic computing and electronic systems. The dynamic reconfiguration of memristors, which is based on the history of applied electrical stimuli, can mimic both essential analog synaptic and neuronal functionalities. These can be utilized as the node and terminal devices in an artificial neural network. Consequently, the ability to understand, control, and utilize fundamental switching principles and various types of device architectures of the memristor is necessary for achieving memristor-based neuromorphic hardware systems. Herein, a wide range of memristors and memristive-related devices for artificial synapses and neurons is highlighted. The device structures, switching principles, and the applications of essential synaptic and neuronal functionalities are sequentially presented. Moreover, recent advances in memristive artificial neural networks and their hardware implementations are introduced along with an overview of the various learning algorithms. Finally, the main challenges of the memristive synapses and neurons toward high-performance and energy-efficient neuromorphic computing are briefly discussed. This progress report aims to be an insightful guide for the research on memristors and neuromorphic-based computing.
Collapse
Affiliation(s)
- Sanghyeon Choi
- KU-KIST Graduate School of Converging Science and Technology, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea
| | - Jehyeon Yang
- KU-KIST Graduate School of Converging Science and Technology, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea
| | - Gunuk Wang
- KU-KIST Graduate School of Converging Science and Technology, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea
| |
Collapse
|
16
|
Zou Z, Zhao R, Wu Y, Yang Z, Tian L, Wu S, Wang G, Yu Y, Zhao Q, Chen M, Pei J, Chen F, Zhang Y, Song S, Zhao M, Shi L. A hybrid and scalable brain-inspired robotic platform. Sci Rep 2020; 10:18160. [PMID: 33097742 PMCID: PMC7584638 DOI: 10.1038/s41598-020-73366-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Accepted: 08/31/2020] [Indexed: 01/04/2023] Open
Abstract
Recent years have witnessed tremendous progress of intelligent robots brought about by mimicking human intelligence. However, current robots are still far from being able to handle multiple tasks in a dynamic environment as efficiently as humans. To cope with complexity and variability, further progress toward scalability and adaptability are essential for intelligent robots. Here, we report a brain-inspired robotic platform implemented by an unmanned bicycle that exhibits scalability of network scale, quantity and diversity to handle the changing needs of different scenarios. The platform adopts rich coding schemes and a trainable and scalable neural state machine, enabling flexible cooperation of hybrid networks. In addition, an embedded system is developed using a cross-paradigm neuromorphic chip to facilitate the implementation of diverse neural networks in spike or non-spike form. The platform achieved various real-time tasks concurrently in different real-world scenarios, providing a new pathway to enhance robots' intelligence.
Collapse
Affiliation(s)
- Zhe Zou
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Rong Zhao
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Yujie Wu
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Zheyu Yang
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Lei Tian
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Shuang Wu
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Guanrui Wang
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Yongchao Yu
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Qi Zhao
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Mingwang Chen
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Jing Pei
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
| | - Feng Chen
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Youhui Zhang
- Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China
| | - Sen Song
- Department of Biomedical Engineering, Tsinghua University, Beijing, 100084, China
| | - Mingguo Zhao
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China.
- Department of Automation, Tsinghua University, Beijing, 100084, China.
| | - Luping Shi
- Center for Brain-Inspired Computing Research (CBICR), Beijing Innovation Center for Future Chip, Optical Memory National Engineering Research Center, and Department of Precision Instrument, Tsinghua University, Beijing, 100084, China.
| |
Collapse
|
17
|
Kassanos P. Analog-digital computing let robots go through the motions. Sci Robot 2020; 5:5/47/eabe6818. [PMID: 33087484 DOI: 10.1126/scirobotics.abe6818] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Accepted: 09/28/2020] [Indexed: 12/22/2022]
Abstract
A hybrid analog-digital computing platform using memristors enables faster, more energy-efficient, and adaptive real-time robot control.
Collapse
Affiliation(s)
- Panagiotis Kassanos
- Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, London SW7 2AZ, UK.
| |
Collapse
|
18
|
Chen K, Hwu T, Kashyap HJ, Krichmar JL, Stewart K, Xing J, Zou X. Neurorobots as a Means Toward Neuroethology and Explainable AI. Front Neurorobot 2020; 14:570308. [PMID: 33192435 PMCID: PMC7604467 DOI: 10.3389/fnbot.2020.570308] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2020] [Accepted: 08/25/2020] [Indexed: 12/18/2022] Open
Abstract
Understanding why deep neural networks and machine learning algorithms act as they do is a difficult endeavor. Neuroscientists are faced with similar problems. One way biologists address this issue is by closely observing behavior while recording neurons or manipulating brain circuits. This has been called neuroethology. In a similar way, neurorobotics can be used to explain how neural network activity leads to behavior. In real world settings, neurorobots have been shown to perform behaviors analogous to animals. Moreover, a neuroroboticist has total control over the network, and by analyzing different neural groups or studying the effect of network perturbations (e.g., simulated lesions), they may be able to explain how the robot's behavior arises from artificial brain activity. In this paper, we review neurorobot experiments by focusing on how the robot's behavior leads to a qualitative and quantitative explanation of neural activity, and vice versa, that is, how neural activity leads to behavior. We suggest that using neurorobots as a form of computational neuroethology can be a powerful methodology for understanding neuroscience, as well as for artificial intelligence and machine learning.
Collapse
Affiliation(s)
- Kexin Chen
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
| | - Tiffany Hwu
- HRL Laboratories (formerly Hughes Research Laboratory), LLC, Malibu, CA, United States
| | - Hirak J Kashyap
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| | - Jeffrey L Krichmar
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States.,Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| | - Kenneth Stewart
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| | - Jinwei Xing
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
| | - Xinyun Zou
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| |
Collapse
|
19
|
Aznar F, Pujol M, Rizo R. Obtaining fault tolerance avoidance behavior using deep reinforcement learning. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2018.11.090] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
20
|
Bing Z, Baumann I, Jiang Z, Huang K, Cai C, Knoll A. Supervised Learning in SNN via Reward-Modulated Spike-Timing-Dependent Plasticity for a Target Reaching Vehicle. Front Neurorobot 2019; 13:18. [PMID: 31130854 PMCID: PMC6509616 DOI: 10.3389/fnbot.2019.00018] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2018] [Accepted: 04/15/2019] [Indexed: 11/16/2022] Open
Abstract
Spiking neural networks (SNNs) offer many advantages over traditional artificial neural networks (ANNs) such as biological plausibility, fast information processing, and energy efficiency. Although SNNs have been used to solve a variety of control tasks using the Spike-Timing-Dependent Plasticity (STDP) learning rule, existing solutions usually involve hard-coded network architectures solving specific tasks rather than solving different kinds of tasks generally. This results in neglecting one of the biggest advantages of ANNs, i.e., being general-purpose and easy-to-use due to their simple network architecture, which usually consists of an input layer, one or multiple hidden layers and an output layer. This paper addresses the problem by introducing an end-to-end learning approach of spiking neural networks constructed with one hidden layer and reward-modulated Spike-Timing-Dependent Plasticity (R-STDP) synapses in an all-to-all fashion. We use the supervised reward-modulated Spike-Timing-Dependent-Plasticity learning rule to train two different SNN-based sub-controllers to replicate a desired obstacle avoiding and goal approaching behavior, provided by pre-generated datasets. Together they make up a target-reaching controller, which is used to control a simulated mobile robot to reach a target area while avoiding obstacles in its path. We demonstrate the performance and effectiveness of our trained SNNs to achieve target reaching tasks in different unknown scenarios.
Collapse
Affiliation(s)
- Zhenshan Bing
- Chair of Robotics, Artificial Intelligence and Embedded Systems, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Ivan Baumann
- Chair of Robotics, Artificial Intelligence and Embedded Systems, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Zhuangyi Jiang
- Chair of Robotics, Artificial Intelligence and Embedded Systems, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Kai Huang
- Department of Data and Computer Science, Sun Yat-Sen University, Guangzhou, China.,Peng Cheng Laboratory, Shenzhen, China
| | - Caixia Cai
- Chair of Robotics, Artificial Intelligence and Embedded Systems, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Alois Knoll
- Chair of Robotics, Artificial Intelligence and Embedded Systems, Department of Informatics, Technical University of Munich, Munich, Germany
| |
Collapse
|
21
|
Frenkel C, Lefebvre M, Legat JD, Bol D. A 0.086-mm 2 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:145-158. [PMID: 30418919 DOI: 10.1109/tbcas.2018.2880425] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Shifting computing architectures from von Neumann to event-based spiking neural networks (SNNs) uncovers new opportunities for low-power processing of sensory data in applications such as vision or sensorimotor control. Exploring roads toward cognitive SNNs requires the design of compact, low-power and versatile experimentation platforms with the key requirement of online learning in order to adapt and learn new features in uncontrolled environments. However, embedding online learning in SNNs is currently hindered by high incurred complexity and area overheads. In this paper, we present ODIN, a 0.086-mm 2 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm FDSOI CMOS achieving a minimum energy per synaptic operation (SOP) of 12.7 pJ. It leverages an efficient implementation of the spike-driven synaptic plasticity (SDSP) learning rule for high-density embedded online learning with only 0.68 μm 2 per 4-bit synapse. Neurons can be independently configured as a standard leaky integrate-and-fire model or as a custom phenomenological model that emulates the 20 Izhikevich behaviors found in biological spiking neurons. Using a single presentation of 6k 16 × 16 MNIST training images to a single-layer fully-connected 10-neuron network with on-chip SDSP-based learning, ODIN achieves a classification accuracy of 84.5%, while consuming only 15 nJ/inference at 0.55 V using rank order coding. ODIN thus enables further developments toward cognitive neuromorphic devices for low-power, adaptive and low-cost processing.
Collapse
|
22
|
Fu Q, Wang H, Hu C, Yue S. Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review. ARTIFICIAL LIFE 2019; 25:263-311. [PMID: 31397604 DOI: 10.1162/artl_a_00297] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.
Collapse
Affiliation(s)
- Qinbing Fu
- Guangzhou University, School of Mechanical and Electrical Engineering; Machine Life and Intelligence Research Centre
- University of Lincoln, Computational Intelligence Lab, School of Computer Science; Lincoln Centre for Autonomous Systems.
| | - Hongxin Wang
- University of Lincoln, Computational Intelligence Lab, School of Computer Science; Lincoln Centre for Autonomous Systems.
| | - Cheng Hu
- Guangzhou University, School of Mechanical and Electrical Engineering; Machine Life and Intelligence Research Centre
- University of Lincoln, Computational Intelligence Lab, School of Computer Science; Lincoln Centre for Autonomous Systems.
| | - Shigang Yue
- Guangzhou University, School of Mechanical and Electrical Engineering; Machine Life and Intelligence Research Centre
- University of Lincoln, Computational Intelligence Lab, School of Computer Science; Lincoln Centre for Autonomous Systems.
| |
Collapse
|
23
|
Kreiser R, Aathmani D, Qiao N, Indiveri G, Sandamirskaya Y. Organizing Sequential Memory in a Neuromorphic Device Using Dynamic Neural Fields. Front Neurosci 2018; 12:717. [PMID: 30524218 PMCID: PMC6262404 DOI: 10.3389/fnins.2018.00717] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2018] [Accepted: 09/19/2018] [Indexed: 11/26/2022] Open
Abstract
Neuromorphic Very Large Scale Integration (VLSI) devices emulate the activation dynamics of biological neuronal networks using either mixed-signal analog/digital or purely digital electronic circuits. Using analog circuits in silicon to physically emulate the functionality of biological neurons and synapses enables faithful modeling of neural and synaptic dynamics at ultra low power consumption in real-time, and thus may serve as computational substrate for a new generation of efficient neural controllers for artificial intelligent systems. Although one of the main advantages of neural networks is their ability to perform on-line learning, only a small number of neuromorphic hardware devices implement this feature on-chip. In this work, we use a reconfigurable on-line learning spiking (ROLLS) neuromorphic processor chip to build a neuronal architecture for sequence learning. The proposed neuronal architecture uses the attractor properties of winner-takes-all (WTA) dynamics to cope with mismatch and noise in the ROLLS analog computing elements, and it uses its on-chip plasticity features to store sequences of states. We demonstrate, with a proof-of-concept feasibility study how this architecture can store, replay, and update sequences of states, induced by external inputs. Controlled by the attractor dynamics and an explicit destabilizing signal, the items in a sequence can last for varying amounts of time and thus reliable sequence learning and replay can be robustly implemented in a real sensorimotor system.
Collapse
Affiliation(s)
- Raphaela Kreiser
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Dora Aathmani
- The School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | - Ning Qiao
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Giacomo Indiveri
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Yulia Sandamirskaya
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
| |
Collapse
|
24
|
Knight JC, Nowotny T. GPUs Outperform Current HPC and Neuromorphic Solutions in Terms of Speed and Energy When Simulating a Highly-Connected Cortical Model. Front Neurosci 2018; 12:941. [PMID: 30618570 PMCID: PMC6299048 DOI: 10.3389/fnins.2018.00941] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2018] [Accepted: 11/29/2018] [Indexed: 11/15/2022] Open
Abstract
While neuromorphic systems may be the ultimate platform for deploying spiking neural networks (SNNs), their distributed nature and optimization for specific types of models makes them unwieldy tools for developing them. Instead, SNN models tend to be developed and simulated on computers or clusters of computers with standard von Neumann CPU architectures. Over the last decade, as well as becoming a common fixture in many workstations, NVIDIA GPU accelerators have entered the High Performance Computing field and are now used in 50 % of the Top 10 super computing sites worldwide. In this paper we use our GeNN code generator to re-implement two neo-cortex-inspired, circuit-scale, point neuron network models on GPU hardware. We verify the correctness of our GPU simulations against prior results obtained with NEST running on traditional HPC hardware and compare the performance with respect to speed and energy consumption against published data from CPU-based HPC and neuromorphic hardware. A full-scale model of a cortical column can be simulated at speeds approaching 0.5× real-time using a single NVIDIA Tesla V100 accelerator-faster than is currently possible using a CPU based cluster or the SpiNNaker neuromorphic system. In addition, we find that, across a range of GPU systems, the energy to solution as well as the energy per synaptic event of the microcircuit simulation is as much as 14× lower than either on SpiNNaker or in CPU-based simulations. Besides performance in terms of speed and energy consumption of the simulation, efficient initialization of models is also a crucial concern, particularly in a research context where repeated runs and parameter-space exploration are required. Therefore, we also introduce in this paper some of the novel parallel initialization methods implemented in the latest version of GeNN and demonstrate how they can enable further speed and energy advantages.
Collapse
Affiliation(s)
- James C. Knight
- Centre for Computational Neuroscience and Robotics, School of Engineering and Informatics, University of Sussex, Brighton, United Kingdom
| | | |
Collapse
|
25
|
Detorakis G, Sheik S, Augustine C, Paul S, Pedroni BU, Dutt N, Krichmar J, Cauwenberghs G, Neftci E. Neural and Synaptic Array Transceiver: A Brain-Inspired Computing Framework for Embedded Learning. Front Neurosci 2018; 12:583. [PMID: 30210274 PMCID: PMC6123384 DOI: 10.3389/fnins.2018.00583] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2018] [Accepted: 08/03/2018] [Indexed: 11/13/2022] Open
Abstract
Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromorphic hardware. However, neuromorphic implementations of embedded learning at large scales that are both flexible and efficient have been hindered by a lack of a suitable algorithmic framework. As a result, most neuromorphic hardware are trained off-line on large clusters of dedicated processors or GPUs and transferred post hoc to the device. We address this by introducing the neural and synaptic array transceiver (NSAT), a neuromorphic computational framework facilitating flexible and efficient embedded learning by matching algorithmic requirements and neural and synaptic dynamics. NSAT supports event-driven supervised, unsupervised and reinforcement learning algorithms including deep learning. We demonstrate the NSAT in a wide range of tasks, including the simulation of Mihalas-Niebur neuron, dynamic neural fields, event-driven random back-propagation for event-based deep learning, event-based contrastive divergence for unsupervised learning, and voltage-based learning rules for sequence learning. We anticipate that this contribution will establish the foundation for a new generation of devices enabling adaptive mobile systems, wearable devices, and robots with data-driven autonomy.
Collapse
Affiliation(s)
- Georgios Detorakis
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
| | - Sadique Sheik
- Biocircuits Institute, University of California, San Diego, La Jolla, CA, United States
| | - Charles Augustine
- Intel Corporation-Circuit Research Lab, Hillsboro, OR, United States
| | - Somnath Paul
- Intel Corporation-Circuit Research Lab, Hillsboro, OR, United States
| | - Bruno U. Pedroni
- Department of Bioengineering and Institute for Neural Computation, University of California, San Diego, La Jolla, CA, United States
| | - Nikil Dutt
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| | - Jeffrey Krichmar
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| | - Gert Cauwenberghs
- Department of Bioengineering and Institute for Neural Computation, University of California, San Diego, La Jolla, CA, United States
| | - Emre Neftci
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States
- Department of Computer Science, University of California, Irvine, Irvine, CA, United States
| |
Collapse
|
26
|
Milde MB, Bertrand OJN, Ramachandran H, Egelhaaf M, Chicca E. Spiking Elementary Motion Detector in Neuromorphic Systems. Neural Comput 2018; 30:2384-2417. [PMID: 30021082 DOI: 10.1162/neco_a_01112] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Apparent motion of the surroundings on an agent's retina can be used to navigate through cluttered environments, avoid collisions with obstacles, or track targets of interest. The pattern of apparent motion of objects, (i.e., the optic flow), contains spatial information about the surrounding environment. For a small, fast-moving agent, as used in search and rescue missions, it is crucial to estimate the distance to close-by objects to avoid collisions quickly. This estimation cannot be done by conventional methods, such as frame-based optic flow estimation, given the size, power, and latency constraints of the necessary hardware. A practical alternative makes use of event-based vision sensors. Contrary to the frame-based approach, they produce so-called events only when there are changes in the visual scene. We propose a novel asynchronous circuit, the spiking elementary motion detector (sEMD), composed of a single silicon neuron and synapse, to detect elementary motion from an event-based vision sensor. The sEMD encodes the time an object's image needs to travel across the retina into a burst of spikes. The number of spikes within the burst is proportional to the speed of events across the retina. A fast but imprecise estimate of the time-to-travel can already be obtained from the first two spikes of a burst and refined by subsequent interspike intervals. The latter encoding scheme is possible due to an adaptive nonlinear synaptic efficacy scaling. We show that the sEMD can be used to compute a collision avoidance direction in the context of robotic navigation in a cluttered outdoor environment and compared the collision avoidance direction to a frame-based algorithm. The proposed computational principle constitutes a generic spiking temporal correlation detector that can be applied to other sensory modalities (e.g., sound localization), and it provides a novel perspective to gating information in spiking neural networks.
Collapse
Affiliation(s)
- M B Milde
- Institute of Neuroinformatics, University of Zurich, and ETH Zurich, 8057 Zurich, Switzerland
| | - O J N Bertrand
- Neurobiology, Faculty of Biology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - H Ramachandran
- Faculty of Technology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - M Egelhaaf
- Neurobiology, Faculty of Biology, Bielefeld University, 33615 Bielefeld, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| | - E Chicca
- Faculty of Technology, Bielefeld University, 33615 Bielefeld, Germany, and Cognitive Interaction Technology, Center of Excellence, Bielefeld University, 33501 Bielefeld, Germany
| |
Collapse
|
27
|
Low-Cost Servomotor Driver for PFM Control. SENSORS 2017; 18:s18010093. [PMID: 29301221 PMCID: PMC5795573 DOI: 10.3390/s18010093] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/20/2017] [Revised: 12/16/2017] [Accepted: 12/28/2017] [Indexed: 11/16/2022]
Abstract
Servomotors have already been around for some decades and they are extremely popular among roboticists due to their simple control technique, reliability and low-cost. They are usually controlled by using Pulse Width Modulation (PWM) and this paper aims to keep the idea of simplicity and low-cost, while introducing a new control technique: Pulse Frequency Modulation (PFM). The objective of this paper is to focus on our development of a low-cost servomotor controller which will allow the research community to use them with PFM. A low-cost commercial servomotor is used as the base system for the development: a small PCB that fits inside the case and allocates all the electronic components to control the motor has been designed to replace the original. The potentiometer is retained as the feedback sensor and a microcontroller is responsible for controlling the position of the motor. The paper compares the performance of a PWM and a PFM controlled servomotor. The comparison shows that the servomotor with our controller achieves a faster mechanism for switching targets and a lower latency. This controller can be used with neuromorphic systems to remove the conversion from events to PWM.
Collapse
|