1
|
Schoepe T, Janotte E, Milde MB, Bertrand OJN, Egelhaaf M, Chicca E. Finding the gap: neuromorphic motion-vision in dense environments. Nat Commun 2024; 15:817. [PMID: 38280859 PMCID: PMC10821932 DOI: 10.1038/s41467-024-45063-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 01/15/2024] [Indexed: 01/29/2024] Open
Abstract
Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.
Collapse
Affiliation(s)
- Thorben Schoepe
- Peter Grünberg Institut 15, Forschungszentrum Jülich, Aachen, Germany.
- Faculty of Technology and Cognitive Interaction Technology Center of Excellence (CITEC), Bielefeld University, Bielefeld, Germany.
- Bio-Inspired Circuits and Systems (BICS) Lab. Zernike Institute for Advanced Materials (Zernike Inst Adv Mat), University of Groningen, Groningen, Netherlands.
- CogniGron (Groningen Cognitive Systems and Materials Center), University of Groningen, Groningen, Netherlands.
| | - Ella Janotte
- Event Driven Perception for Robotics, Italian Institute of Technology, iCub facility, Genoa, Italy
| | - Moritz B Milde
- International Centre for Neuromorphic Systems, MARCS Institute, Western Sydney University, Penrith, Australia
| | | | - Martin Egelhaaf
- Neurobiology, Faculty of Biology, Bielefeld University, Bielefeld, Germany
| | - Elisabetta Chicca
- Faculty of Technology and Cognitive Interaction Technology Center of Excellence (CITEC), Bielefeld University, Bielefeld, Germany
- Bio-Inspired Circuits and Systems (BICS) Lab. Zernike Institute for Advanced Materials (Zernike Inst Adv Mat), University of Groningen, Groningen, Netherlands
- CogniGron (Groningen Cognitive Systems and Materials Center), University of Groningen, Groningen, Netherlands
| |
Collapse
|
2
|
Moro F, Hardy E, Fain B, Dalgaty T, Clémençon P, De Prà A, Esmanhotto E, Castellani N, Blard F, Gardien F, Mesquida T, Rummens F, Esseni D, Casas J, Indiveri G, Payvand M, Vianello E. Neuromorphic object localization using resistive memories and ultrasonic transducers. Nat Commun 2022; 13:3506. [PMID: 35717413 PMCID: PMC9206646 DOI: 10.1038/s41467-022-31157-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 06/03/2022] [Indexed: 11/25/2022] Open
Abstract
Real-world sensory-processing applications require compact, low-latency, and low-power computing systems. Enabled by their in-memory event-driven computing abilities, hybrid memristive-Complementary Metal-Oxide Semiconductor neuromorphic architectures provide an ideal hardware substrate for such tasks. To demonstrate the full potential of such systems, we propose and experimentally demonstrate an end-to-end sensory processing solution for a real-world object localization application. Drawing inspiration from the barn owl's neuroanatomy, we developed a bio-inspired, event-driven object localization system that couples state-of-the-art piezoelectric micromachined ultrasound transducer sensors to a neuromorphic resistive memories-based computational map. We present measurement results from the fabricated system comprising resistive memories-based coincidence detectors, delay line circuits, and a full-custom ultrasound sensor. We use these experimental results to calibrate our system-level simulations. These simulations are then used to estimate the angular resolution and energy efficiency of the object localization model. The results reveal the potential of our approach, evaluated in orders of magnitude greater energy efficiency than a microcontroller performing the same task.
Collapse
Affiliation(s)
- Filippo Moro
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France.
| | - Emmanuel Hardy
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
| | - Bruno Fain
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
| | - Thomas Dalgaty
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
- CEA, LIST, Université Grenoble Alpes, 38054, Grenoble, France
| | - Paul Clémençon
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
- Insect Biology Research Institute, Université de Tours, 37020, Tours, France
| | - Alessio De Prà
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
- DPIA, Università degli Studi di Udine, 33100, Udine, Italy
| | | | | | - François Blard
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France
| | | | - Thomas Mesquida
- CEA, LIST, Université Grenoble Alpes, 38054, Grenoble, France
| | | | - David Esseni
- DPIA, Università degli Studi di Udine, 33100, Udine, Italy
| | - Jérôme Casas
- Insect Biology Research Institute, Université de Tours, 37020, Tours, France
| | - Giacomo Indiveri
- Institute for Neuroinformatics, University of Zürich and ETH Zürich, 8057, Zürich, Switzerland
| | - Melika Payvand
- Institute for Neuroinformatics, University of Zürich and ETH Zürich, 8057, Zürich, Switzerland
| | - Elisa Vianello
- CEA, LETI, Université Grenoble Alpes, 38054, Grenoble, France.
| |
Collapse
|