1
|
Rush ER, Heckman C, Jayaram K, Humbert JS. Neural dynamics of robust legged robots. Front Robot AI 2024; 11:1324404. [PMID: 38699630 PMCID: PMC11063321 DOI: 10.3389/frobt.2024.1324404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 03/26/2024] [Indexed: 05/05/2024] Open
Abstract
Legged robot control has improved in recent years with the rise of deep reinforcement learning, however, much of the underlying neural mechanisms remain difficult to interpret. Our aim is to leverage bio-inspired methods from computational neuroscience to better understand the neural activity of robust robot locomotion controllers. Similar to past work, we observe that terrain-based curriculum learning improves agent stability. We study the biomechanical responses and neural activity within our neural network controller by simultaneously pairing physical disturbances with targeted neural ablations. We identify an agile hip reflex that enables the robot to regain its balance and recover from lateral perturbations. Model gradients are employed to quantify the relative degree that various sensory feedback channels drive this reflexive behavior. We also find recurrent dynamics are implicated in robust behavior, and utilize sampling-based ablation methods to identify these key neurons. Our framework combines model-based and sampling-based methods for drawing causal relationships between neural network activity and robust embodied robot behavior.
Collapse
Affiliation(s)
- Eugene R. Rush
- Department of Mechanical Engineering, University of Colorado Boulder, Boulder, CO, United States
| | - Christoffer Heckman
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
| | - Kaushik Jayaram
- Department of Mechanical Engineering, University of Colorado Boulder, Boulder, CO, United States
| | - J. Sean Humbert
- Department of Mechanical Engineering, University of Colorado Boulder, Boulder, CO, United States
| |
Collapse
|
2
|
Weninger L, Srivastava P, Zhou D, Kim JZ, Cornblath EJ, Bertolero MA, Habel U, Merhof D, Bassett DS. Information content of brain states is explained by structural constraints on state energetics. Phys Rev E 2022; 106:014401. [PMID: 35974521 DOI: 10.1103/physreve.106.014401] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 04/27/2022] [Indexed: 06/15/2023]
Abstract
Signal propagation along the structural connectome of the brain induces changes in the patterns of activity. These activity patterns define global brain states and contain information in accordance with their expected probability of occurrence. Being the physical substrate upon which information propagates, the structural connectome, in conjunction with the dynamics, determines the set of possible brain states and constrains the transition between accessible states. Yet, precisely how these structural constraints on state transitions relate to their information content remains unexplored. To address this gap in knowledge, we defined the information content as a function of the activation distribution, where statistically rare values of activation correspond to high information content. With this numerical definition in hand, we studied the spatiotemporal distribution of information content in functional magnetic resonance imaging (fMRI) data from the Human Connectome Project during different tasks, and report four key findings. First, information content strongly depends on cognitive context; its absolute level and spatial distribution depend on the cognitive task. Second, while information content shows similarities to other measures of brain activity, it is distinct from both Neurosynth maps and task contrast maps generated by a general linear model applied to the fMRI data. Third, the brain's structural wiring constrains the cost to control its state, where the cost to transition into high information content states is larger than that to transition into low information content states. Finally, all state transitions-especially those to high information content states-are less costly than expected from random network null models, thereby indicating the brains marked efficiency. Taken together, our findings establish an explanatory link between the information contained in a brain state and the energetic cost of attaining that state, thereby laying important groundwork for our understanding of large-scale cognitive computations.
Collapse
Affiliation(s)
- Leon Weninger
- Department of Bioengineering, School of Engineering & Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Institute of Imaging & Computer Vision, RWTH Aachen University, 52072 Aachen, Germany
| | - Pragya Srivastava
- Department of Bioengineering, School of Engineering & Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Dale Zhou
- Department of Bioengineering, School of Engineering & Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Jason Z Kim
- Department of Bioengineering, School of Engineering & Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Eli J Cornblath
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Maxwell A Bertolero
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Ute Habel
- Department of Psychiatry, Psychotherapy and Psychosomatics, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany
- Institute of Neuroscience and Medicine 10, Research Centre Jülich, 52428 Jülich, Germany
| | - Dorit Merhof
- Institute of Imaging & Computer Vision, RWTH Aachen University, 52072 Aachen, Germany
| | - Dani S Bassett
- Department of Bioengineering, School of Engineering & Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Department of Physics & Astronomy, College of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Department of Electrical & Systems Engineering, School of Engineering and Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
- Santa Fe Institute, Santa Fe, New Mexico 87501, USA
| |
Collapse
|
3
|
Chiêm B, Crevecoeur F, Delvenne JC. Structure-informed functional connectivity driven by identifiable and state-specific control regions. Netw Neurosci 2021; 5:591-613. [PMID: 34189379 PMCID: PMC8233121 DOI: 10.1162/netn_a_00192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 03/17/2021] [Indexed: 11/19/2022] Open
Abstract
Describing how the brain anatomical wiring contributes to the emergence of coordinated neural activity underlying complex behavior remains challenging. Indeed, patterns of remote coactivations that adjust with the ongoing task-demand do not systematically match direct, static anatomical links. Here, we propose that observed coactivation patterns, known as functional connectivity (FC), can be explained by a controllable linear diffusion dynamics defined on the brain architecture. Our model, termed structure-informed FC, is based on the hypothesis that different sets of brain regions controlling the information flow on the anatomical wiring produce state-specific functional patterns. We thus introduce a principled framework for the identification of potential control centers in the brain. We find that well-defined, sparse, and robust sets of control regions, partially overlapping across several tasks and resting state, produce FC patterns comparable to empirical ones. Our findings suggest that controllability is a fundamental feature allowing the brain to reach different states. Understanding how brain anatomy promotes particular patterns of coactivations among neural regions is a key challenge in neuroscience. This challenge can be addressed using network science and systems theory. Here, we propose that coactivations result from the diffusion of information through the network of anatomical links connecting brain regions, with certain regions controlling the dynamics. We translate this hypothesis into a model called structure-informed functional connectivity, and we introduce a framework for identifying control regions based on empirical data. We find that our model produces coactivation patterns comparable to empirical ones, and that distinct sets of control regions are associated with different functional states. These findings suggest that controllability is an important feature allowing the brain to reach different states.
Collapse
Affiliation(s)
- Benjamin Chiêm
- Institute of Communication Technologies, Electronics, and Applied Mathematics, Department of Mathematical Engineering, Université Catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Frédéric Crevecoeur
- Institute of Communication Technologies, Electronics, and Applied Mathematics, Department of Mathematical Engineering, Université Catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Jean-Charles Delvenne
- Institute of Communication Technologies, Electronics, and Applied Mathematics, Department of Mathematical Engineering, Université Catholique de Louvain, Louvain-la-Neuve, Belgium
| |
Collapse
|
4
|
Inhibitory neurons exhibit high controlling ability in the cortical microconnectome. PLoS Comput Biol 2021; 17:e1008846. [PMID: 33831009 PMCID: PMC8031186 DOI: 10.1371/journal.pcbi.1008846] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Accepted: 03/01/2021] [Indexed: 02/08/2023] Open
Abstract
The brain is a network system in which excitatory and inhibitory neurons keep activity balanced in the highly non-random connectivity pattern of the microconnectome. It is well known that the relative percentage of inhibitory neurons is much smaller than excitatory neurons in the cortex. So, in general, how inhibitory neurons can keep the balance with the surrounding excitatory neurons is an important question. There is much accumulated knowledge about this fundamental question. This study quantitatively evaluated the relatively higher functional contribution of inhibitory neurons in terms of not only properties of individual neurons, such as firing rate, but also in terms of topological mechanisms and controlling ability on other excitatory neurons. We combined simultaneous electrical recording (~2.5 hours) of ~1000 neurons in vitro, and quantitative evaluation of neuronal interactions including excitatory-inhibitory categorization. This study accurately defined recording brain anatomical targets, such as brain regions and cortical layers, by inter-referring MRI and immunostaining recordings. The interaction networks enabled us to quantify topological influence of individual neurons, in terms of controlling ability to other neurons. Especially, the result indicated that highly influential inhibitory neurons show higher controlling ability of other neurons than excitatory neurons, and are relatively often distributed in deeper layers of the cortex. Furthermore, the neurons having high controlling ability are more effectively limited in number than central nodes of k-cores, and these neurons also participate in more clustered motifs. In summary, this study suggested that the high controlling ability of inhibitory neurons is a key mechanism to keep balance with a large number of other excitatory neurons beyond simple higher firing rate. Application of the selection method of limited important neurons would be also applicable for the ability to effectively and selectively stimulate E/I imbalanced disease states. How small numbers of inhibitory neurons functionally keep balance with large numbers of excitatory neurons in the brain by controlling each other is a fundamental question. Especially, this study quantitatively evaluated a topological mechanism of interaction networks in terms of controlling abilities of individual cortical neurons to other neurons. Combination of simultaneous electrical recording of ~1000 neurons and a quantitative evaluation method of neuronal interactions including excitatory-inhibitory categories, enabled us to evaluate the influence of individual neurons not only about firing rate but also about their relative positions in the networks and controllable ability of other neurons. Especially, the result showed that inhibitory neurons have more controlling ability than excitatory neurons, and such neurons were more often observed in deep layers. Because the limited number of neurons in terms controlling ability were much smaller than neurons based on centrality measure and, of course, more directly selected neurons based on their ability to control other neurons, the selection method of important neurons will help not only to produce realistic computational models but also will help to stimulate brain to effectively treat imbalanced disease states.
Collapse
|
5
|
Targeted Central Nervous System Irradiation of Caenorhabditis elegans Induces a Limited Effect on Motility. BIOLOGY 2020; 9:biology9090289. [PMID: 32937967 PMCID: PMC7564402 DOI: 10.3390/biology9090289] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/26/2020] [Revised: 09/09/2020] [Accepted: 09/10/2020] [Indexed: 12/29/2022]
Abstract
To clarify the tissue responsible for a biological function, that function can be experimentally perturbed by an external stimulus, such as radiation. Radiation can be precisely and finely administered and any subsequent change in function examined. To investigate the involvement of the central nervous system (CNS) in Caenorhabditis elegans’ locomotion, we irradiated a limited 20-µm-diameter area of the CNS with a single dose and evaluated the resulting effects on motility. However, whether irradiated area (beam size)-dependent or dose-dependent effects on motility occur via targeted irradiation remain unknown. In the present study, we examined the irradiated area- and dose-dependent effects of CNS-targeted irradiation on the motility of C. elegans using a collimating microbeam system and confirmed the involvement of the CNS and body-wall muscle cells around the CNS in motility. After CNS-targeted microbeam irradiation, C. elegans’ motility was assayed. The results demonstrated a dose-dependent effect of CNS-targeted irradiation on motility reflecting direct effects on the irradiated CNS. In addition, when irradiated with 1000-Gy irradiation, irradiated area (beam size)-dependent effects were observed. This method has two technical advantages: Performing a series of on-chip imaging analyses before and after irradiation and targeted irradiation using a distinct ion-beam size.
Collapse
|