1
|
Song X, Guo T, Ma S, Zhou F, Tian J, Liu Z, Liu J, Li H, Chen Y, Chai X, Li L. Spatially Selective Retinal Ganglion Cell Activation Using Low Invasive Extraocular Temporal Interference Stimulation. Int J Neural Syst 2025; 35:2450066. [PMID: 39318031 DOI: 10.1142/s0129065724500667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/26/2024]
Abstract
Conventional retinal implants involve complex surgical procedures and require invasive implantation. Temporal Interference Stimulation (TIS) has achieved noninvasive and focused stimulation of deep brain regions by delivering high-frequency currents with small frequency differences on multiple electrodes. In this study, we conducted in silico investigations to evaluate extraocular TIS's potential as a novel visual restoration approach. Different from the previously published retinal TIS model, the new model of extraocular TIS incorporated a biophysically detailed retinal ganglion cell (RGC) population, enabling a more accurate simulation of retinal outputs under electrical stimulation. Using this improved model, we made the following major discoveries: (1) the maximum value of TIS envelope electric potential ([Formula: see text] showed a strong correlation with TIS-induced RGC activation; (2) the preferred stimulating/return electrode (SE/RE) locations to achieve focalized TIS were predicted; (3) the performance of extraocular TIS was better than same-frequency sinusoidal stimulation (SSS) in terms of lower RGC threshold and more focused RGC activation; (4) the optimal stimulation parameters to achieve lower threshold and focused activation were identified; and (5) spatial selectivity of TIS could be improved by integrating current steering strategy and reducing electrode size. This study provides insights into the feasibility and effectiveness of a low-invasive stimulation approach in enhancing vision restoration.
Collapse
Affiliation(s)
- Xiaoyu Song
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Tianruo Guo
- Graduate School of Biomedical Engineering, The University of New South Wales, Sydney, NSW 2052, Australia
| | - Saidong Ma
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Feng Zhou
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Jiaxin Tian
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Zhengyang Liu
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Jiao Liu
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Heng Li
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Yao Chen
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Xinyu Chai
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| | - Liming Li
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| |
Collapse
|
2
|
Wysmolek PM, Kiessler FD, Salbaum KA, Shelton ER, Sonntag SM, Serwane F. A minimal-complexity light-sheet microscope maps network activity in 3D neuronal systems. Sci Rep 2022; 12:20420. [PMID: 36443413 PMCID: PMC9705530 DOI: 10.1038/s41598-022-24350-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 11/14/2022] [Indexed: 11/29/2022] Open
Abstract
In vitro systems mimicking brain regions, brain organoids, are revolutionizing the neuroscience field. However, characterization of their electrical activity has remained a challenge as it requires readout at millisecond timescale in 3D at single-neuron resolution. While custom-built microscopes used with genetically encoded sensors are now opening this door, a full 3D characterization of organoid neural activity has not been performed yet, limited by the combined complexity of the optical and the biological system. Here, we introduce an accessible minimalistic light-sheet microscope to the neuroscience community. Designed as an add-on to a standard inverted microscope it can be assembled within one day. In contrast to existing simplistic setups, our platform is suited to record volumetric calcium traces. We successfully extracted 4D calcium traces at high temporal resolution by using a lightweight piezo stage to allow for 5 Hz volumetric scanning combined with a processing pipeline for true 3D neuronal trace segmentation. As a proof of principle, we created a 3D connectivity map of a stem cell derived neuron spheroid by imaging its activity. Our fast, low complexity setup empowers researchers to study the formation of neuronal networks in vitro for fundamental and neurodegeneration research.
Collapse
Affiliation(s)
- Paulina M. Wysmolek
- grid.414703.50000 0001 2202 0959Max Planck Institute for Medical Research, Heidelberg, Germany
| | - Filippo D. Kiessler
- grid.5252.00000 0004 1936 973XFaculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Katja A. Salbaum
- grid.5252.00000 0004 1936 973XFaculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany ,Graduate School of Systemic Neuroscience (GSN), Munich, Germany
| | - Elijah R. Shelton
- grid.5252.00000 0004 1936 973XFaculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Selina M. Sonntag
- grid.5252.00000 0004 1936 973XFaculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Friedhelm Serwane
- grid.5252.00000 0004 1936 973XFaculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany ,Graduate School of Systemic Neuroscience (GSN), Munich, Germany ,grid.452617.3Munich Cluster for Systems Neurology (SyNergy), Munich, Germany
| |
Collapse
|
3
|
Wang J, Qian L, Wang S, Shi L, Wang Z. Directional Preference in Avian Midbrain Saliency Computing Nucleus Reflects a Well-Designed Receptive Field Structure. Animals (Basel) 2022; 12:1143. [PMID: 35565569 PMCID: PMC9105111 DOI: 10.3390/ani12091143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 04/18/2022] [Accepted: 04/26/2022] [Indexed: 11/20/2022] Open
Abstract
Neurons responding sensitively to motions in several rather than all directions have been identified in many sensory systems. Although this directional preference has been demonstrated by previous studies to exist in the isthmi pars magnocellularis (Imc) of pigeon (Columba livia), which plays a key role in the midbrain saliency computing network, the dynamic response characteristics and the physiological basis underlying this phenomenon are unclear. Herein, dots moving in 16 directions and a biologically plausible computational model were used. We found that pigeon Imc's significant responses for objects moving in preferred directions benefit the long response duration and high instantaneous firing rate. Furthermore, the receptive field structures predicted by a computational model, which captures the actual directional tuning curves, agree with the real data collected from population Imc units. These results suggested that directional preference in Imc may be internally prebuilt by elongating the vertical axis of the receptive field, making predators attack from the dorsal-ventral direction and conspecifics flying away in the ventral-dorsal direction, more salient for avians, which is of great ecological and physiological significance for survival.
Collapse
Affiliation(s)
- Jiangtao Wang
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (J.W.); (L.Q.); (S.W.)
| | - Longlong Qian
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (J.W.); (L.Q.); (S.W.)
| | - Songwei Wang
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (J.W.); (L.Q.); (S.W.)
| | - Li Shi
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (J.W.); (L.Q.); (S.W.)
- Department of Automation, Tsinghua University, Beijing 100084, China
| | - Zhizhong Wang
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (J.W.); (L.Q.); (S.W.)
| |
Collapse
|
4
|
Cessac B. Retinal Processing: Insights from Mathematical Modelling. J Imaging 2022; 8:14. [PMID: 35049855 PMCID: PMC8780400 DOI: 10.3390/jimaging8010014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Revised: 01/11/2022] [Accepted: 01/12/2022] [Indexed: 02/04/2023] Open
Abstract
The retina is the entrance of the visual system. Although based on common biophysical principles, the dynamics of retinal neurons are quite different from their cortical counterparts, raising interesting problems for modellers. In this paper, I address some mathematically stated questions in this spirit, discussing, in particular: (1) How could lateral amacrine cell connectivity shape the spatio-temporal spike response of retinal ganglion cells? (2) How could spatio-temporal stimuli correlations and retinal network dynamics shape the spike train correlations at the output of the retina? These questions are addressed, first, introducing a mathematically tractable model of the layered retina, integrating amacrine cells' lateral connectivity and piecewise linear rectification, allowing for computing the retinal ganglion cells receptive field together with the voltage and spike correlations of retinal ganglion cells resulting from the amacrine cells networks. Then, I review some recent results showing how the concept of spatio-temporal Gibbs distributions and linear response theory can be used to characterize the collective spike response to a spatio-temporal stimulus of a set of retinal ganglion cells, coupled via effective interactions corresponding to the amacrine cells network. On these bases, I briefly discuss several potential consequences of these results at the cortical level.
Collapse
Affiliation(s)
- Bruno Cessac
- France INRIA Biovision Team and Neuromod Institute, Université Côte d'Azur, 2004 Route des Lucioles, BP 93, 06902 Valbonne, France
| |
Collapse
|
5
|
Ezra-Tsur E, Amsalem O, Ankri L, Patil P, Segev I, Rivlin-Etzion M. Realistic retinal modeling unravels the differential role of excitation and inhibition to starburst amacrine cells in direction selectivity. PLoS Comput Biol 2021; 17:e1009754. [PMID: 34968385 PMCID: PMC8754344 DOI: 10.1371/journal.pcbi.1009754] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Revised: 01/12/2022] [Accepted: 12/14/2021] [Indexed: 11/19/2022] Open
Abstract
Retinal direction-selectivity originates in starburst amacrine cells (SACs), which display a centrifugal preference, responding with greater depolarization to a stimulus expanding from soma to dendrites than to a collapsing stimulus. Various mechanisms were hypothesized to underlie SAC centrifugal preference, but dissociating them is experimentally challenging and the mechanisms remain debatable. To address this issue, we developed the Retinal Stimulation Modeling Environment (RSME), a multifaceted data-driven retinal model that encompasses detailed neuronal morphology and biophysical properties, retina-tailored connectivity scheme and visual input. Using a genetic algorithm, we demonstrated that spatiotemporally diverse excitatory inputs-sustained in the proximal and transient in the distal processes-are sufficient to generate experimentally validated centrifugal preference in a single SAC. Reversing these input kinetics did not produce any centrifugal-preferring SAC. We then explored the contribution of SAC-SAC inhibitory connections in establishing the centrifugal preference. SAC inhibitory network enhanced the centrifugal preference, but failed to generate it in its absence. Embedding a direction selective ganglion cell (DSGC) in a SAC network showed that the known SAC-DSGC asymmetric connectivity by itself produces direction selectivity. Still, this selectivity is sharpened in a centrifugal-preferring SAC network. Finally, we use RSME to demonstrate the contribution of SAC-SAC inhibitory connections in mediating direction selectivity and recapitulate recent experimental findings. Thus, using RSME, we obtained a mechanistic understanding of SACs' centrifugal preference and its contribution to direction selectivity.
Collapse
Affiliation(s)
- Elishai Ezra-Tsur
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel
- Department of Mathematics and Computer Science, The Open University of Israel, Ra’anana, Israel
| | - Oren Amsalem
- Department of Neurobiology, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Lea Ankri
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel
| | - Pritish Patil
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel
| | - Idan Segev
- Department of Neurobiology, Hebrew University of Jerusalem, Jerusalem, Israel
- Edmond and Lily Safra Center for Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
| | | |
Collapse
|
6
|
Santos-Mayo A, Moratti S, de Echegaray J, Susi G. A Model of the Early Visual System Based on Parallel Spike-Sequence Detection, Showing Orientation Selectivity. BIOLOGY 2021; 10:biology10080801. [PMID: 34440033 PMCID: PMC8389551 DOI: 10.3390/biology10080801] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 08/12/2021] [Accepted: 08/16/2021] [Indexed: 12/22/2022]
Abstract
Simple Summary A computational model of primates’ early visual processing, showing orientation selectivity, is presented. The system importantly integrates two key elements: (1) a neuromorphic spike-decoding structure that considerably resembles the circuitry between layers IV and II/III of the primary visual cortex, both in topology and operation; (2) the plasticity of intrinsic excitability, to embed recent findings about the operation of the same area. The model is proposed as a tool for the analysis and reproduction of the orientation selectivity phenomenon, whose underlying neuronal-level computational mechanisms are today the subject of intense scrutiny. In response to rotated Gabor patches the model is able to exhibit realistic orientation tuning curves and to reproduce responses similar to those found in neurophysiological recordings from the primary visual cortex obtained under the same task, considering different stages of the network. This demonstrates its aptness to capture the mechanisms underlying the evoked response in the primary visual cortex. Our tool is available online, and can be expanded to other experiments using a dedicated software library developed by the authors, to elucidate the computational mechanisms underlying orientation selectivity. Abstract Since the first half of the twentieth century, numerous studies have been conducted on how the visual cortex encodes basic image features. One of the hallmarks of basic feature extraction is the phenomenon of orientation selectivity, of which the underlying neuronal-level computational mechanisms remain partially unclear despite being intensively investigated. In this work we present a reduced visual system model (RVSM) of the first level of scene analysis, involving the retina, the lateral geniculate nucleus and the primary visual cortex (V1), showing orientation selectivity. The detection core of the RVSM is the neuromorphic spike-decoding structure MNSD, which is able to learn and recognize parallel spike sequences and considerably resembles the neuronal microcircuits of V1 in both topology and operation. This structure is equipped with plasticity of intrinsic excitability to embed recent findings about V1 operation. The RVSM, which embeds 81 groups of MNSD arranged in 4 oriented columns, is tested using sets of rotated Gabor patches as input. Finally, synthetic visual evoked activity generated by the RVSM is compared with real neurophysiological signal from V1 area: (1) postsynaptic activity of human subjects obtained by magnetoencephalography and (2) spiking activity of macaques obtained by multi-tetrode arrays. The system is implemented using the NEST simulator. The results attest to a good level of resemblance between the model response and real neurophysiological recordings. As the RVSM is available online, and the model parameters can be customized by the user, we propose it as a tool to elucidate the computational mechanisms underlying orientation selectivity.
Collapse
Affiliation(s)
- Alejandro Santos-Mayo
- Laboratory of Cognitive and Computational Neuroscience, Center for Biomedical Technology, Technical University of Madrid, 28040 Madrid, Spain; (A.S.-M.); (S.M.); (J.d.E.)
- Department of Experimental Psychology, Faculty of Psychology, Complutense University of Madrid, 28040 Madrid, Spain
| | - Stephan Moratti
- Laboratory of Cognitive and Computational Neuroscience, Center for Biomedical Technology, Technical University of Madrid, 28040 Madrid, Spain; (A.S.-M.); (S.M.); (J.d.E.)
- Department of Experimental Psychology, Faculty of Psychology, Complutense University of Madrid, 28040 Madrid, Spain
- Laboratory of Clinical Neuroscience, Center for Biomedical Technology, Technical University of Madrid, 28040 Madrid, Spain
| | - Javier de Echegaray
- Laboratory of Cognitive and Computational Neuroscience, Center for Biomedical Technology, Technical University of Madrid, 28040 Madrid, Spain; (A.S.-M.); (S.M.); (J.d.E.)
- Department of Experimental Psychology, Faculty of Psychology, Complutense University of Madrid, 28040 Madrid, Spain
| | - Gianluca Susi
- Laboratory of Cognitive and Computational Neuroscience, Center for Biomedical Technology, Technical University of Madrid, 28040 Madrid, Spain; (A.S.-M.); (S.M.); (J.d.E.)
- Department of Experimental Psychology, Faculty of Psychology, Complutense University of Madrid, 28040 Madrid, Spain
- Department of Civil Engineering and Computer Science, University of Rome “Tor Vergata”, 00133 Rome, Italy
- Correspondence: ; Tel.: +34-(61)-86893399-79317
| |
Collapse
|
7
|
Edwards DA, Emerick B, Kondic AG, Kiradjiev K, Raymond C, Zyskin M. Mathematical models for the effect of anti-vascular endothelial growth factor on visual acuity. J Math Biol 2020; 81:1397-1428. [PMID: 32968840 DOI: 10.1007/s00285-020-01544-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2020] [Revised: 06/09/2020] [Accepted: 09/13/2020] [Indexed: 11/27/2022]
Abstract
The standard of care treatment for neovascular age-related macular degeneration, delivered as ocular injection, is based on anti-vascular endothelial growth factor (anti-VEGF). The course of treatment may need to be modified quickly for certain patients based on their response. Models that track both the concentration and the response to an anti-VEGF treatment are presented. The specific focus is to assess the existence of analytical solutions for the different types of models. Both an ODE-based model and a map-based model illustrate the dependence of the solution on various biological parameters and allow the measurement of patient-specific parameters from experimental data. A PDE-based model incorporates diffusive effects. The results are consistent with observed values, and could provide a framework for practitioners to understand the effect of the therapy on the progression of the disease in both responsive and non-responsive patients.
Collapse
Affiliation(s)
- David A Edwards
- Department of Mathematical Sciences, University of Delaware, Newark, DE, 19716, USA.
| | - Brooks Emerick
- Department of Mathematics, Kutztown University, Kutztown, PA, 19530, USA
| | | | | | - Christopher Raymond
- Department of Mathematical Sciences, University of Delaware, Newark, DE, 19716, USA
| | - Maxim Zyskin
- Department of Engineering Science, University of Oxford, Oxford, OX1 3PJ, UK
| |
Collapse
|
8
|
Retinal Drug Delivery: Rethinking Outcomes for the Efficient Replication of Retinal Behavior. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10124258] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
The retina is a highly organized structure that is considered to be "an approachable part of the brain." It is attracting the interest of development scientists, as it provides a model neurovascular system. Over the last few years, we have been witnessing significant development in the knowledge of the mechanisms that induce the shape of the retinal vascular system, as well as knowledge of disease processes that lead to retina degeneration. Knowledge and understanding of how our vision works are crucial to creating a hardware-adaptive computational model that can replicate retinal behavior. The neuronal system is nonlinear and very intricate. It is thus instrumental to have a clear view of the neurophysiological and neuroanatomic processes and to take into account the underlying principles that govern the process of hardware transformation to produce an appropriate model that can be mapped to a physical device. The mechanistic and integrated computational models have enormous potential toward helping to understand disease mechanisms and to explain the associations identified in large model-free data sets. The approach used is modulated and based on different models of drug administration, including the geometry of the eye. This work aimed to review the recently used mathematical models to map a directed retinal network.
Collapse
|
9
|
Crespo-Cano R, Cuenca-Asensi S, Fernández E, Martínez-Álvarez A. Metaheuristic Optimisation Algorithms for Tuning a Bioinspired Retinal Model. SENSORS (BASEL, SWITZERLAND) 2019; 19:E4834. [PMID: 31698827 PMCID: PMC6891458 DOI: 10.3390/s19224834] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Revised: 10/31/2019] [Accepted: 11/03/2019] [Indexed: 11/22/2022]
Abstract
A significant challenge in neuroscience is understanding how visual information is encoded in the retina. Such knowledge is extremely important for the purpose of designing bioinspired sensors and artificial retinal systems that will, in so far as may be possible, be capable of mimicking vertebrate retinal behaviour. In this study, we report the tuning of a reliable computational bioinspired retinal model with various algorithms to improve the mimicry of the model. Its main contribution is two-fold. First, given the multi-objective nature of the problem, an automatic multi-objective optimisation strategy is proposed through the use of four biological-based metrics, which are used to adjust the retinal model for accurate prediction of retinal ganglion cell responses. Second, a subset of population-based search heuristics-genetic algorithms (SPEA2, NSGA-II and NSGA-III), particle swarm optimisation (PSO) and differential evolution (DE)-are explored to identify the best algorithm for fine-tuning the retinal model, by comparing performance across a hypervolume metric. Nonparametric statistical tests are used to perform a rigorous comparison between all the metaheuristics. The best results were achieved with the PSO algorithm on the basis of the largest hypervolume that was achieved, well-distributed elements and high numbers on the Pareto front.
Collapse
Affiliation(s)
- Rubén Crespo-Cano
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| | - Sergio Cuenca-Asensi
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| | - Eduardo Fernández
- Institute of Bioengineering, University Miguel Hernández and CIBER BBN, 03202 Elche (Alicante), Spain;
| | - Antonio Martínez-Álvarez
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| |
Collapse
|
10
|
Schroder P, Martinez-Canada P, Amorim A, Fernandes P, Amorim-de-Sousa A, Gonzalez-Meijome JM. A Minimal-Model Approach to Analyze Neuronal Circuit Dynamics from multifocal ERG (mERG). ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2019:2955-2958. [PMID: 31946510 DOI: 10.1109/embc.2019.8856840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The multifocal electroretinogram (mERG) is a valuable non-invasive tool used by clinicians for studying both the local physiology of the normal human retina, as well as retinal function affected by disease. However, the neural basis of the major components of the mERG is not well understood. Computational modeling of neural circuits has lead to important insights into the structure and workings of nervous systems. A better understanding of the different neural factors that contribute to the retina function and progression of the pathology can help in developing an effective clinical intervention. In this paper, we propose a minimal computational model as an analysis tool for inferring the relationship between mERG data of the human retina and its underlying neural activity and interpreting the data in terms of the specific neural features that contribute to it. In this preliminary study, the rigor of this analysis technique was assessed by fitting the model with a genetic algorithm (GA) to collected mERG data from ten healthy patients.
Collapse
|
11
|
Bornet A, Kaiser J, Kroner A, Falotico E, Ambrosano A, Cantero K, Herzog MH, Francis G. Running Large-Scale Simulations on the Neurorobotics Platform to Understand Vision - The Case of Visual Crowding. Front Neurorobot 2019; 13:33. [PMID: 31191291 PMCID: PMC6549494 DOI: 10.3389/fnbot.2019.00033] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Accepted: 05/14/2019] [Indexed: 11/13/2022] Open
Abstract
Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.
Collapse
Affiliation(s)
- Alban Bornet
- Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Jacques Kaiser
- FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Alexander Kroner
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Egidio Falotico
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | | | | | - Michael H. Herzog
- Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Gregory Francis
- Department of Psychological Sciences, Purdue University, West Lafayette, IN, United States
| |
Collapse
|
12
|
Martínez-Cañada P, Morillas C, Pelayo F. A Neuronal Network Model of the Primate Visual System: Color Mechanisms in the Retina, LGN and V1. Int J Neural Syst 2019; 29:1850036. [DOI: 10.1142/s0129065718500363] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Color plays a key role in human vision but the neural machinery that underlies the transformation from stimulus to perception is not well understood. Here, we implemented a two-dimensional network model of the first stages in the primate parvocellular pathway (retina, lateral geniculate nucleus and layer 4C[Formula: see text] in V1) consisting of conductance-based point neurons. Model parameters were tuned based on physiological and anatomical data from the primate foveal and parafoveal vision, the most relevant visual field areas for color vision. We exhaustively benchmarked the model against well-established chromatic and achromatic visual stimuli, showing spatial and temporal responses of the model to disk- and ring-shaped light flashes, spatially uniform squares and sine-wave gratings of varying spatial frequency. The spatiotemporal patterns of parvocellular cells and cortical cells are consistent with their classification into chromatically single-opponent and double-opponent groups, and nonopponent cells selective for luminance stimuli. The model was implemented in the widely used neural simulation tool NEST and released as open source software. The aim of our modeling is to provide a biologically realistic framework within which a broad range of neuronal interactions can be examined at several different levels, with a focus on understanding how color information is processed.
Collapse
Affiliation(s)
- Pablo Martínez-Cañada
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Christian Morillas
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Francisco Pelayo
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| |
Collapse
|
13
|
Carrillo RR, Naveros F, Ros E, Luque NR. A Metric for Evaluating Neural Input Representation in Supervised Learning Networks. Front Neurosci 2019; 12:913. [PMID: 30618549 PMCID: PMC6302114 DOI: 10.3389/fnins.2018.00913] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2017] [Accepted: 11/20/2018] [Indexed: 11/13/2022] Open
Abstract
Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with particular attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity, which constitutes the sole output of the cerebellar cortex. The learning process at this parallel-fiber-to-Purkinje-cell connection makes each Purkinje cell sensitive to a set of specific cerebellar states, which are roughly determined by the granule-cell activity during a certain time window. A Purkinje cell becomes sensitive to each neural input state and, consequently, the network operates as a function able to generate a desired output for each provided input by means of supervised learning. However, not all sets of Purkinje cell responses can be assigned to any set of input states due to the network's own limitations (inherent to the network neurobiological substrate), that is, not all input-output mapping can be learned. A key limiting factor is the representation of the input states through granule-cell activity. The quality of this representation (e.g., in terms of heterogeneity) will determine the capacity of the network to learn a varied set of outputs. Assessing the quality of this representation is interesting when developing and studying models of these networks to identify those neuron or network characteristics that enhance this representation. In this study we present an algorithm for evaluating quantitatively the level of compatibility/interference amongst a set of given cerebellar states according to their representation (granule-cell activation patterns) without the need for actually conducting simulations and network training. The algorithm input consists of a real-number matrix that codifies the activity level of every considered granule-cell in each state. The capability of this representation to generate a varied set of outputs is evaluated geometrically, thus resulting in a real number that assesses the goodness of the representation.
Collapse
Affiliation(s)
- Richard R Carrillo
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain.,Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC-UGR), University of Granada, Granada, Spain
| | - Francisco Naveros
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain.,Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC-UGR), University of Granada, Granada, Spain
| | - Eduardo Ros
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain.,Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC-UGR), University of Granada, Granada, Spain
| | - Niceto R Luque
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain.,Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC-UGR), University of Granada, Granada, Spain.,Aging in Vision and Action, Institut de la Vision, Inserm-UPMC-CNRS, Paris, France
| |
Collapse
|
14
|
Mobarhan MH, Halnes G, Martínez-Cañada P, Hafting T, Fyhn M, Einevoll GT. Firing-rate based network modeling of the dLGN circuit: Effects of cortical feedback on spatiotemporal response properties of relay cells. PLoS Comput Biol 2018; 14:e1006156. [PMID: 29771919 PMCID: PMC5976212 DOI: 10.1371/journal.pcbi.1006156] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2018] [Revised: 05/30/2018] [Accepted: 04/23/2018] [Indexed: 12/01/2022] Open
Abstract
Visually evoked signals in the retina pass through the dorsal geniculate nucleus (dLGN) on the way to the visual cortex. This is however not a simple feedforward flow of information: there is a significant feedback from cortical cells back to both relay cells and interneurons in the dLGN. Despite four decades of experimental and theoretical studies, the functional role of this feedback is still debated. Here we use a firing-rate model, the extended difference-of-Gaussians (eDOG) model, to explore cortical feedback effects on visual responses of dLGN relay cells. For this model the responses are found by direct evaluation of two- or three-dimensional integrals allowing for fast and comprehensive studies of putative effects of different candidate organizations of the cortical feedback. Our analysis identifies a special mixed configuration of excitatory and inhibitory cortical feedback which seems to best account for available experimental data. This configuration consists of (i) a slow (long-delay) and spatially widespread inhibitory feedback, combined with (ii) a fast (short-delayed) and spatially narrow excitatory feedback, where (iii) the excitatory/inhibitory ON-ON connections are accompanied respectively by inhibitory/excitatory OFF-ON connections, i.e. following a phase-reversed arrangement. The recent development of optogenetic and pharmacogenetic methods has provided new tools for more precise manipulation and investigation of the thalamocortical circuit, in particular for mice. Such data will expectedly allow the eDOG model to be better constrained by data from specific animal model systems than has been possible until now for cat. We have therefore made the Python tool pyLGN which allows for easy adaptation of the eDOG model to new situations. On route from the retina to primary visual cortex, visually evoked signals have to pass through the dorsal lateral geniculate nucleus (dLGN). However, this is not an exclusive feedforward flow of information as feedback exists from neurons in the cortex back to both relay cells and interneurons in the dLGN. The functional role of this feedback remains mostly unresolved. Here, we use a firing-rate model, the extended difference-of-Gaussians (eDOG) model, to explore cortical feedback effects on visual responses of dLGN relay cells. Our analysis indicates that a particular mix of excitatory and inhibitory cortical feedback agrees best with available experimental observations. In this configuration ON-center relay cells receive both excitatory and (indirect) inhibitory feedback from ON-center cortical cells (ON-ON feedback) where the excitatory feedback is fast and spatially narrow while the inhibitory feedback is slow and spatially widespread. In addition to the ON-ON feedback, the connections are accompanied by OFF-ON connections following a so-called phase-reversed (push-pull) arrangement. To facilitate further applications of the model, we have made the Python tool pyLGN which allows for easy modification and evaluation of the a priori quite general eDOG model to new situations.
Collapse
Affiliation(s)
- Milad Hobbi Mobarhan
- Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
- Department of Biosciences, University of Oslo, Oslo, Norway
| | - Geir Halnes
- Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
- Department of Mathematical Sciences and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Pablo Martínez-Cañada
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Torkel Hafting
- Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
- Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway
| | - Marianne Fyhn
- Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
- Department of Biosciences, University of Oslo, Oslo, Norway
| | - Gaute T. Einevoll
- Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
- Department of Mathematical Sciences and Technology, Norwegian University of Life Sciences, Ås, Norway
- Department of Physics, University of Oslo, Oslo, Norway
- * E-mail:
| |
Collapse
|
15
|
Eshraghian JK, Baek S, Kim JH, Iannella N, Cho K, Goo YS, Iu HHC, Kang SM, Eshraghian K. Formulation and Implementation of Nonlinear Integral Equations to Model Neural Dynamics Within the Vertebrate Retina. Int J Neural Syst 2018; 28:1850004. [PMID: 29631506 DOI: 10.1142/s0129065718500041] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Existing computational models of the retina often compromise between the biophysical accuracy and a hardware-adaptable methodology of implementation. When compared to the current modes of vision restoration, algorithmic models often contain a greater correlation between stimuli and the affected neural network, but lack physical hardware practicality. Thus, if the present processing methods are adapted to complement very-large-scale circuit design techniques, it is anticipated that it will engender a more feasible approach to the physical construction of the artificial retina. The computational model presented in this research serves to provide a fast and accurate predictive model of the retina, a deeper understanding of neural responses to visual stimulation, and an architecture that can realistically be transformed into a hardware device. Traditionally, implicit (or semi-implicit) ordinary differential equations (OES) have been used for optimal speed and accuracy. We present a novel approach that requires the effective integration of different dynamical time scales within a unified framework of neural responses, where the rod, cone, amacrine, bipolar, and ganglion cells correspond to the implemented pathways. Furthermore, we show that adopting numerical integration can both accelerate retinal pathway simulations by more than 50% when compared with traditional ODE solvers in some cases, and prove to be a more realizable solution for the hardware implementation of predictive retinal models.
Collapse
Affiliation(s)
- Jason K Eshraghian
- 1 School of Electrical, Electronic and Computer Engineering, University of Western Australia, Crawley, Australia
| | - Seungbum Baek
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | - Jun-Ho Kim
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | | | - Kyoungrok Cho
- 2 College of Electrical and Computer Engineering, Chungbuk National University, Cheongju, Republic of Korea
| | - Yong Sook Goo
- 4 Department of Physiology, School of Medicine, Chungbuk National University, Cheongju, Republic of Korea
| | - Herbert H C Iu
- 1 School of Electrical, Electronic and Computer Engineering, University of Western Australia, Crawley, Australia
| | - Sung-Mo Kang
- 5 Department of Electrical Engineering, University of California, Santa Cruz, Santa Cruz, USA
| | | |
Collapse
|
16
|
Huth J, Masquelier T, Arleo A. Convis: A Toolbox to Fit and Simulate Filter-Based Models of Early Visual Processing. Front Neuroinform 2018; 12:9. [PMID: 29563867 PMCID: PMC5845886 DOI: 10.3389/fninf.2018.00009] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2017] [Accepted: 02/21/2018] [Indexed: 11/13/2022] Open
Abstract
We developed Convis, a Python simulation toolbox for large scale neural populations which offers arbitrary receptive fields by 3D convolutions executed on a graphics card. The resulting software proves to be flexible and easily extensible in Python, while building on the PyTorch library (The Pytorch Project, 2017), which was previously used successfully in deep learning applications, for just-in-time optimization and compilation of the model onto CPU or GPU architectures. An alternative implementation based on Theano (Theano Development Team, 2016) is also available, although not fully supported. Through automatic differentiation, any parameter of a specified model can be optimized to approach a desired output which is a significant improvement over e.g., Monte Carlo or particle optimizations without gradients. We show that a number of models including even complex non-linearities such as contrast gain control and spiking mechanisms can be implemented easily. We show in this paper that we can in particular recreate the simulation results of a popular retina simulation software VirtualRetina (Wohrer and Kornprobst, 2009), with the added benefit of providing (1) arbitrary linear filters instead of the product of Gaussian and exponential filters and (2) optimization routines utilizing the gradients of the model. We demonstrate the utility of 3d convolution filters with a simple direction selective filter. Also we show that it is possible to optimize the input for a certain goal, rather than the parameters, which can aid the design of experiments as well as closed-loop online stimulus generation. Yet, Convis is more than a retina simulator. For instance it can also predict the response of V1 orientation selective cells. Convis is open source under the GPL-3.0 license and available from https://github.com/jahuth/convis/ with documentation at https://jahuth.github.io/convis/.
Collapse
Affiliation(s)
- Jacob Huth
- Centre National de la Recherche Scientifique, INSERM, Sorbonne Universités, UPMC Univ Paris 06, Paris, France
| | - Timothée Masquelier
- CERCO UMR5549, Centre National de la Recherche Scientifique, University Toulouse 3, Toulouse, France
| | - Angelo Arleo
- Centre National de la Recherche Scientifique, INSERM, Sorbonne Universités, UPMC Univ Paris 06, Paris, France
| |
Collapse
|
17
|
Kheradpisheh SR, Ganjtabesh M, Thorpe SJ, Masquelier T. STDP-based spiking deep convolutional neural networks for object recognition. Neural Netw 2018; 99:56-67. [PMID: 29328958 DOI: 10.1016/j.neunet.2017.12.005] [Citation(s) in RCA: 204] [Impact Index Per Article: 29.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2017] [Revised: 11/23/2017] [Accepted: 12/08/2017] [Indexed: 11/25/2022]
|
18
|
Martínez-Cañada P, Mobarhan MH, Halnes G, Fyhn M, Morillas C, Pelayo F, Einevoll GT. Biophysical network modeling of the dLGN circuit: Effects of cortical feedback on spatial response properties of relay cells. PLoS Comput Biol 2018; 14:e1005930. [PMID: 29377888 PMCID: PMC5805346 DOI: 10.1371/journal.pcbi.1005930] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2017] [Revised: 02/08/2018] [Accepted: 12/17/2017] [Indexed: 11/19/2022] Open
Abstract
Despite half-a-century of research since the seminal work of Hubel and Wiesel, the role of the dorsal lateral geniculate nucleus (dLGN) in shaping the visual signals is not properly understood. Placed on route from retina to primary visual cortex in the early visual pathway, a striking feature of the dLGN circuit is that both the relay cells (RCs) and interneurons (INs) not only receive feedforward input from retinal ganglion cells, but also a prominent feedback from cells in layer 6 of visual cortex. This feedback has been proposed to affect synchronicity and other temporal properties of the RC firing. It has also been seen to affect spatial properties such as the center-surround antagonism of thalamic receptive fields, i.e., the suppression of the response to very large stimuli compared to smaller, more optimal stimuli. Here we explore the spatial effects of cortical feedback on the RC response by means of a a comprehensive network model with biophysically detailed, single-compartment and multicompartment neuron models of RCs, INs and a population of orientation-selective layer 6 simple cells, consisting of pyramidal cells (PY). We have considered two different arrangements of synaptic feedback from the ON and OFF zones in the visual cortex to the dLGN: phase-reversed (‘push-pull’) and phase-matched (‘push-push’), as well as different spatial extents of the corticothalamic projection pattern. Our simulation results support that a phase-reversed arrangement provides a more effective way for cortical feedback to provide the increased center-surround antagonism seen in experiments both for flashing spots and, even more prominently, for patch gratings. This implies that ON-center RCs receive direct excitation from OFF-dominated cortical cells and indirect inhibitory feedback from ON-dominated cortical cells. The increased center-surround antagonism in the model is accompanied by spatial focusing, i.e., the maximum RC response occurs for smaller stimuli when feedback is present. The functional role of the dorsal lateral geniculate nucleus (dLGN), placed on route from retina to primary visual cortex in the early visual pathway, is still poorly understood. A striking feature of the dLGN circuit is that dLGN cells not only receive feedforward input from the retina, but also a prominent feedback from cells in the visual cortex. It has been seen in experiments that cortical feedback modifies the spatial properties of dLGN cells in response to visual stimuli. In particular, it has been shown to increase the center-surround antagonism for flashing-spot and patch-grating visual stimuli, i.e., the suppression of responses to very large stimuli compared to smaller stimuli. Here we investigate the putative mechanisms behind this feature by means of a comprehensive network model of biophysically detailed neuron models for RCs and INs in the dLGN and orientation-selective cortical cells providing the feedback. Our results support that the experimentally observed feedback effects may be due to a phase-reversed (‘push-pull’) arrangement of the cortical feedback where ON-symmetry RCs receive (indirect) inhibitory feedback from ON-dominated cortical cell and excitation from OFF-dominated cortical cells, and vice versa for OFF-symmetry RCs.
Collapse
Affiliation(s)
- Pablo Martínez-Cañada
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Milad Hobbi Mobarhan
- Center for Integrative Neuroplasticity (CINPLA), University of Oslo, Oslo, Norway
- Department of Biosciences, University of Oslo, Oslo, Norway
| | - Geir Halnes
- Center for Integrative Neuroplasticity (CINPLA), University of Oslo, Oslo, Norway
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Marianne Fyhn
- Center for Integrative Neuroplasticity (CINPLA), University of Oslo, Oslo, Norway
- Department of Biosciences, University of Oslo, Oslo, Norway
| | - Christian Morillas
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Francisco Pelayo
- Department of Computer Architecture and Technology, University of Granada, Granada, Spain
- Centro de Investigación en Tecnologías de la Información y de las Comunicaciones (CITIC), University of Granada, Granada, Spain
| | - Gaute T. Einevoll
- Center for Integrative Neuroplasticity (CINPLA), University of Oslo, Oslo, Norway
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
- Department of Physics, University of Oslo, Oslo, Norway
- * E-mail:
| |
Collapse
|
19
|
|
20
|
|
21
|
Falotico E, Vannucci L, Ambrosano A, Albanese U, Ulbrich S, Vasquez Tieck JC, Hinkel G, Kaiser J, Peric I, Denninger O, Cauli N, Kirtay M, Roennau A, Klinker G, Von Arnim A, Guyot L, Peppicelli D, Martínez-Cañada P, Ros E, Maier P, Weber S, Huber M, Plecher D, Röhrbein F, Deser S, Roitberg A, van der Smagt P, Dillman R, Levi P, Laschi C, Knoll AC, Gewaltig MO. Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform. Front Neurorobot 2017; 11:2. [PMID: 28179882 PMCID: PMC5263131 DOI: 10.3389/fnbot.2017.00002] [Citation(s) in RCA: 53] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2016] [Accepted: 01/04/2017] [Indexed: 11/13/2022] Open
Abstract
Combined efforts in the fields of neuroscience, computer science, and biology allowed to design biologically realistic models of the brain based on spiking neural networks. For a proper validation of these models, an embodiment in a dynamic and rich sensory environment, where the model is exposed to a realistic sensory-motor task, is needed. Due to the complexity of these brain models that, at the current stage, cannot deal with real-time constraints, it is not possible to embed them into a real-world task. Rather, the embodiment has to be simulated as well. While adequate tools exist to simulate either complex neural networks or robots and their environments, there is so far no tool that allows to easily establish a communication between brain and body models. The Neurorobotics Platform is a new web-based environment that aims to fill this gap by offering scientists and technology developers a software infrastructure allowing them to connect brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems for in silico experimentation. In order to simplify the workflow and reduce the level of the required programming skills, the platform provides editors for the specification of experimental sequences and conditions, environments, robots, and brain-body connectors. In addition to that, a variety of existing robots and environments are provided. This work presents the architecture of the first release of the Neurorobotics Platform developed in subproject 10 "Neurorobotics" of the Human Brain Project (HBP). At the current state, the Neurorobotics Platform allows researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to simplified versions of brain models. We illustrate the capabilities of the platform with three example experiments: a Braitenberg task implemented on a mobile robot, a sensory-motor learning task based on a robotic controller, and a visual tracking embedding a retina model on the iCub humanoid robot. These use-cases allow to assess the applicability of the Neurorobotics Platform for robotic tasks as well as in neuroscientific experiments.
Collapse
Affiliation(s)
- Egidio Falotico
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | - Lorenzo Vannucci
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | | | - Ugo Albanese
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | - Stefan Ulbrich
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Juan Camilo Vasquez Tieck
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Georg Hinkel
- Department of Software Engineering (SE), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Jacques Kaiser
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Igor Peric
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Oliver Denninger
- Department of Software Engineering (SE), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Nino Cauli
- Computer and Robot Vision Laboratory, Instituto de Sistemas e Robotica, Instituto Superior Tecnico, Lisbon, Portugal
| | - Murat Kirtay
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | - Arne Roennau
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Gudrun Klinker
- Department of Informatics, Technical University of Munich, Garching, Germany
| | | | - Luc Guyot
- Blue Brain Project (BBP), École polytechnique fédérale de Lausanne (EPFL), Genève, Switzerland
| | - Daniel Peppicelli
- Blue Brain Project (BBP), École polytechnique fédérale de Lausanne (EPFL), Genève, Switzerland
| | - Pablo Martínez-Cañada
- Department of Computer Architecture and Technology, CITIC, University of Granada, Granada, Spain
| | - Eduardo Ros
- Department of Computer Architecture and Technology, CITIC, University of Granada, Granada, Spain
| | - Patrick Maier
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Sandro Weber
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Manuel Huber
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - David Plecher
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Florian Röhrbein
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Stefan Deser
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Alina Roitberg
- Department of Informatics, Technical University of Munich, Garching, Germany
| | | | - Rüdiger Dillman
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Paul Levi
- Department of Intelligent Systems and Production Engineering (ISPE – IDS/TKS), FZI Research Center for Information Technology, Karlsruhe, Germany
| | - Cecilia Laschi
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy
| | - Alois C. Knoll
- Department of Informatics, Technical University of Munich, Garching, Germany
| | - Marc-Oliver Gewaltig
- Blue Brain Project (BBP), École polytechnique fédérale de Lausanne (EPFL), Genève, Switzerland
| |
Collapse
|