1
|
Chiappa AS, Tano P, Patel N, Ingster A, Pouget A, Mathis A. Acquiring musculoskeletal skills with curriculum-based reinforcement learning. Neuron 2024; 112:3969-3983.e5. [PMID: 39357519 DOI: 10.1016/j.neuron.2024.09.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Revised: 07/29/2024] [Accepted: 09/04/2024] [Indexed: 10/04/2024]
Abstract
Efficient musculoskeletal simulators and powerful learning algorithms provide computational tools to tackle the grand challenge of understanding biological motor control. Our winning solution for the inaugural NeurIPS MyoChallenge leverages an approach mirroring human skill learning. Using a novel curriculum learning approach, we trained a recurrent neural network to control a realistic model of the human hand with 39 muscles to rotate two Baoding balls in the palm of the hand. In agreement with data from human subjects, the policy uncovers a small number of kinematic synergies, even though it is not explicitly biased toward low-dimensional solutions. However, selectively inactivating parts of the control signal, we found that more dimensions contribute to the task performance than suggested by traditional synergy analysis. Overall, our work illustrates the emerging possibilities at the interface of musculoskeletal physics engines, reinforcement learning, and neuroscience to advance our understanding of biological motor control.
Collapse
Affiliation(s)
- Alberto Silvio Chiappa
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Pablo Tano
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Nisheet Patel
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Abigaïl Ingster
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Alexandre Pouget
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Alexander Mathis
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland.
| |
Collapse
|
2
|
Li Y, An X, Mulcahey PJ, Qian Y, Xu XH, Zhao S, Mohan H, Suryanarayana SM, Bachschmid-Romano L, Brunel N, Whishaw IQ, Huang ZJ. Cortico-thalamic communication for action coordination in a skilled motor sequence. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.25.563871. [PMID: 37961483 PMCID: PMC10634836 DOI: 10.1101/2023.10.25.563871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
The coordination of forelimb and orofacial movements to compose an ethological reach-to-consume behavior likely involves neural communication across brain regions. Leveraging wide-field imaging and photo-inhibition to survey across the cortex, we identified a cortical network and a high-order motor area (MOs-c), which coordinate action progression in a mouse reach-and-withdraw-to-drink (RWD) behavior. Electrophysiology and photo-inhibition across multiple projection neuron types within the MOs-c revealed differential contributions of pyramidal tract and corticothalamic (CTMOs) output channels to action progression and hand-mouth coordination. Notably, CTMOs display sustained firing throughout RWD sequence and selectively enhance RWD-relevant activity in postsynaptic thalamus neurons, which also contribute to action coordination. CTMOs receive converging monosynaptic inputs from forelimb and orofacial sensorimotor areas and are reciprocally connected to thalamic neurons, which project back to the cortical network. Therefore, motor cortex corticothalamic channel may selectively amplify the thalamic integration of cortical and subcortical sensorimotor streams to coordinate a skilled motor sequence.
Collapse
Affiliation(s)
- Yi Li
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
| | - Xu An
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
| | | | - Yongjun Qian
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
- Current affiliation: College of Future technology, Peking-Tsinghua Center for Life Sciences, IDG/McGovern Institute for Brain Research, Beijing Advanced Center of RNA Biology, Peking University, China
| | - X. Hermione Xu
- Department of Biomedical Engineering, Duke University, Durham, NC 27710, USA
| | - Shengli Zhao
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
| | - Hemanth Mohan
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
| | | | | | - Nicolas Brunel
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
| | - Ian Q. Whishaw
- Department of Neuroscience, Canadian Centre for Behavioural Research, University of Lethbridge, Lethbridge, AB, T1K 3M4, Canada
| | - Z. Josh Huang
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
- Department of Biomedical Engineering, Duke University, Durham, NC 27710, USA
| |
Collapse
|
3
|
Wenger M, Maimon A, Yizhar O, Snir A, Sasson Y, Amedi A. Hearing temperatures: employing machine learning for elucidating the cross-modal perception of thermal properties through audition. Front Psychol 2024; 15:1353490. [PMID: 39156805 PMCID: PMC11327021 DOI: 10.3389/fpsyg.2024.1353490] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Accepted: 06/28/2024] [Indexed: 08/20/2024] Open
Abstract
People can use their sense of hearing for discerning thermal properties, though they are for the most part unaware that they can do so. While people unequivocally claim that they cannot perceive the temperature of pouring water through the auditory properties of hearing it being poured, our research further strengthens the understanding that they can. This multimodal ability is implicitly acquired in humans, likely through perceptual learning over the lifetime of exposure to the differences in the physical attributes of pouring water. In this study, we explore people's perception of this intriguing cross modal correspondence, and investigate the psychophysical foundations of this complex ecological mapping by employing machine learning. Our results show that not only can the auditory properties of pouring water be classified by humans in practice, the physical characteristics underlying this phenomenon can also be classified by a pre-trained deep neural network.
Collapse
Affiliation(s)
- Mohr Wenger
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
- Department of Cognitive and Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Amber Maimon
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
- Computational Psychiatry and Neurotechnology Lab, Department of Brain and Cognitive Sciences, Ben Gurion University, Be’er Sheva, Israel
| | - Or Yizhar
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
- Department of Cognitive and Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
- Research Group Adaptive Memory and Decision Making, Max Planck Institute for Human Development, Berlin, Germany
| | - Adi Snir
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
| | - Yonatan Sasson
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
| | - Amir Amedi
- Baruch Ivcher Institute for Brain Cognition and Technology, Baruch Ivcher School of Psychology, Reichman University, Herzliya, Israel
| |
Collapse
|
4
|
Ye S, Filippova A, Lauer J, Schneider S, Vidal M, Qiu T, Mathis A, Mathis MW. SuperAnimal pretrained pose estimation models for behavioral analysis. Nat Commun 2024; 15:5165. [PMID: 38906853 PMCID: PMC11192880 DOI: 10.1038/s41467-024-48792-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Accepted: 04/26/2024] [Indexed: 06/23/2024] Open
Abstract
Quantification of behavior is critical in diverse applications from neuroscience, veterinary medicine to animal conservation. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present SuperAnimal, a method to develop unified foundation models that can be used on over 45 species, without additional manual labels. These models show excellent performance across six pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100× more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification and kinematic analysis. Collectively, we present a data-efficient solution for animal pose estimation.
Collapse
Affiliation(s)
- Shaokai Ye
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Anastasiia Filippova
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Jessy Lauer
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Steffen Schneider
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Maxime Vidal
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Tian Qiu
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Alexander Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Mackenzie Weygandt Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland.
| |
Collapse
|
5
|
Juárez Tello A, van der Zouwen CI, Dejas L, Duque-Yate J, Boutin J, Medina-Ortiz K, Suresh JS, Swiegers J, Sarret P, Ryczko D. Dopamine-sensitive neurons in the mesencephalic locomotor region control locomotion initiation, stop, and turns. Cell Rep 2024; 43:114187. [PMID: 38722743 PMCID: PMC11157412 DOI: 10.1016/j.celrep.2024.114187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 03/31/2024] [Accepted: 04/17/2024] [Indexed: 06/01/2024] Open
Abstract
The locomotor role of dopaminergic neurons is traditionally attributed to their ascending projections to the basal ganglia, which project to the mesencephalic locomotor region (MLR). In addition, descending dopaminergic projections to the MLR are present from basal vertebrates to mammals. However, the neurons targeted in the MLR and their behavioral role are unknown in mammals. Here, we identify genetically defined MLR cells that express D1 or D2 receptors and control different motor behaviors in mice. In the cuneiform nucleus, D1-expressing neurons promote locomotion, while D2-expressing neurons stop locomotion. In the pedunculopontine nucleus, D1-expressing neurons promote locomotion, while D2-expressing neurons evoke ipsilateral turns. Using RNAscope, we show that MLR dopamine-sensitive neurons comprise a combination of glutamatergic, GABAergic, and cholinergic neurons, suggesting that different neurotransmitter-based cell types work together to control distinct behavioral modules. Altogether, our study uncovers behaviorally relevant cell types in the mammalian MLR based on the expression of dopaminergic receptors.
Collapse
Affiliation(s)
- Andrea Juárez Tello
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Cornelis Immanuel van der Zouwen
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Léonie Dejas
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Juan Duque-Yate
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Joël Boutin
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Katherine Medina-Ortiz
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Jacinthlyn Sylvia Suresh
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Jordan Swiegers
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Philippe Sarret
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada; Centre de recherche du Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, QC, Canada; Neurosciences Sherbrooke, Institut de Pharmacologie de Sherbrooke, Sherbrooke, QC, Canada
| | - Dimitri Ryczko
- Département de Pharmacologie-Physiologie, Faculté de médecine et des sciences de la santé, Université de Sherbrooke, Sherbrooke, QC, Canada; Centre de recherche du Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, QC, Canada; Neurosciences Sherbrooke, Institut de Pharmacologie de Sherbrooke, Sherbrooke, QC, Canada.
| |
Collapse
|
6
|
Marin Vargas A, Bisi A, Chiappa AS, Versteeg C, Miller LE, Mathis A. Task-driven neural network models predict neural dynamics of proprioception. Cell 2024; 187:1745-1761.e19. [PMID: 38518772 DOI: 10.1016/j.cell.2024.02.036] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 12/06/2023] [Accepted: 02/27/2024] [Indexed: 03/24/2024]
Abstract
Proprioception tells the brain the state of the body based on distributed sensory neurons. Yet, the principles that govern proprioceptive processing are poorly understood. Here, we employ a task-driven modeling approach to investigate the neural code of proprioceptive neurons in cuneate nucleus (CN) and somatosensory cortex area 2 (S1). We simulated muscle spindle signals through musculoskeletal modeling and generated a large-scale movement repertoire to train neural networks based on 16 hypotheses, each representing different computational goals. We found that the emerging, task-optimized internal representations generalize from synthetic data to predict neural dynamics in CN and S1 of primates. Computational tasks that aim to predict the limb position and velocity were the best at predicting the neural activity in both areas. Since task optimization develops representations that better predict neural activity during active than passive movements, we postulate that neural activity in the CN and S1 is top-down modulated during goal-directed movements.
Collapse
Affiliation(s)
- Alessandro Marin Vargas
- Brain Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Axel Bisi
- Brain Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Alberto S Chiappa
- Brain Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Chris Versteeg
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA; Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA; Department of Biomedical Engineering, McCormick School of Engineering, Northwestern University, Evanston, IL 60208, USA; Shirley Ryan AbilityLab, Chicago, IL 60611, USA
| | - Lee E Miller
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA; Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA; Department of Biomedical Engineering, McCormick School of Engineering, Northwestern University, Evanston, IL 60208, USA; Shirley Ryan AbilityLab, Chicago, IL 60611, USA
| | - Alexander Mathis
- Brain Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland.
| |
Collapse
|
7
|
Hasnain MA, Birnbaum JE, Nunez JLU, Hartman EK, Chandrasekaran C, Economo MN. Separating cognitive and motor processes in the behaving mouse. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.23.554474. [PMID: 37662199 PMCID: PMC10473744 DOI: 10.1101/2023.08.23.554474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
The cognitive processes supporting complex animal behavior are closely associated with ubiquitous movements responsible for our posture, facial expressions, ability to actively sample our sensory environments, and other critical processes. These movements are strongly related to neural activity across much of the brain and are often highly correlated with ongoing cognitive processes, making it challenging to dissociate the neural dynamics that support cognitive processes from those supporting related movements. In such cases, a critical issue is whether cognitive processes are separable from related movements, or if they are driven by common neural mechanisms. Here, we demonstrate how the separability of cognitive and motor processes can be assessed, and, when separable, how the neural dynamics associated with each component can be isolated. We establish a novel two-context behavioral task in mice that involves multiple cognitive processes and show that commonly observed dynamics taken to support cognitive processes are strongly contaminated by movements. When cognitive and motor components are isolated using a novel approach for subspace decomposition, we find that they exhibit distinct dynamical trajectories. Further, properly accounting for movement revealed that largely separate populations of cells encode cognitive and motor variables, in contrast to the 'mixed selectivity' often reported. Accurately isolating the dynamics associated with particular cognitive and motor processes will be essential for developing conceptual and computational models of neural circuit function and evaluating the function of the cell types of which neural circuits are composed.
Collapse
Affiliation(s)
- Munib A. Hasnain
- Department of Biomedical Engineering, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
| | - Jaclyn E. Birnbaum
- Graduate Program for Neuroscience, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
| | | | - Emma K. Hartman
- Department of Biomedical Engineering, Boston University, Boston, MA
| | - Chandramouli Chandrasekaran
- Department of Psychological and Brain Sciences, Boston University, Boston, MA
- Department of Neurobiology & Anatomy, Boston University, Boston, MA
- Center for Systems Neuroscience, Boston University, Boston, MA
| | - Michael N. Economo
- Department of Biomedical Engineering, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
- Center for Systems Neuroscience, Boston University, Boston, MA
| |
Collapse
|
8
|
Hascher S, Shuster A, Mukamel R, Ossmy O. The power of multivariate approach in identifying EEG correlates of interlimb coupling. Front Hum Neurosci 2023; 17:1256497. [PMID: 37900731 PMCID: PMC10603300 DOI: 10.3389/fnhum.2023.1256497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 09/07/2023] [Indexed: 10/31/2023] Open
Abstract
Interlimb coupling refers to the interaction between movements of one limb and movements of other limbs. Understanding mechanisms underlying this effect is important to real life because it reflects the level of interdependence between the limbs that plays a role in daily activities including tool use, cooking, or playing musical instruments. Interlimb coupling involves multiple brain regions working together, including coordination of neural activity in sensory and motor regions across the two hemispheres. Traditional neuroscience research took a univariate approach to identify neural features that correspond to behavioural coupling measures. Yet, this approach reduces the complexity of the neural activity during interlimb tasks to one value. In this brief research report, we argue that identifying neural correlates of interlimb coupling would benefit from a multivariate approach in which full patterns from multiple sources are used to predict behavioural coupling. We demonstrate the feasibility of this approach in an exploratory EEG study where participants (n = 10) completed 240 trials of a well-established drawing paradigm that involves interlimb coupling. Using artificial neural network (ANN), we show that multivariate representation of the EEG signal significantly captures the interlimb coupling during bimanual drawing whereas univariate analyses failed to identify such correlates. Our findings demonstrate that analysing distributed patterns of multiple EEG channels is more sensitive than single-value techniques in uncovering subtle differences between multiple neural signals. Using such techniques can improve identification of neural correlates of complex motor behaviours.
Collapse
Affiliation(s)
- Sophie Hascher
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| | - Anastasia Shuster
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| | - Roy Mukamel
- Sagol School of Neuroscience and School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Ori Ossmy
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| |
Collapse
|
9
|
Butler DJ, Keim AP, Ray S, Azim E. Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models. Nat Commun 2023; 14:5866. [PMID: 37752123 PMCID: PMC10522643 DOI: 10.1038/s41467-023-41565-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 09/08/2023] [Indexed: 09/28/2023] Open
Abstract
Deep learning-based markerless tracking has revolutionized studies of animal behavior. Yet the generalizability of trained models tends to be limited, as new training data typically needs to be generated manually for each setup or visual environment. With each model trained from scratch, researchers track distinct landmarks and analyze the resulting kinematic data in idiosyncratic ways. Moreover, due to inherent limitations in manual annotation, only a sparse set of landmarks are typically labeled. To address these issues, we developed an approach, which we term GlowTrack, for generating orders of magnitude more training data, enabling models that generalize across experimental contexts. We describe: a) a high-throughput approach for producing hidden labels using fluorescent markers; b) a multi-camera, multi-light setup for simulating diverse visual conditions; and c) a technique for labeling many landmarks in parallel, enabling dense tracking. These advances lay a foundation for standardized behavioral pipelines and more complete scrutiny of movement.
Collapse
Affiliation(s)
- Daniel J Butler
- Molecular Neurobiology Laboratory, Salk Institute for Biological Studies, 10010 N. Torrey Pines Road, La Jolla, CA, 92037, USA
| | - Alexander P Keim
- Molecular Neurobiology Laboratory, Salk Institute for Biological Studies, 10010 N. Torrey Pines Road, La Jolla, CA, 92037, USA
| | - Shantanu Ray
- Molecular Neurobiology Laboratory, Salk Institute for Biological Studies, 10010 N. Torrey Pines Road, La Jolla, CA, 92037, USA
| | - Eiman Azim
- Molecular Neurobiology Laboratory, Salk Institute for Biological Studies, 10010 N. Torrey Pines Road, La Jolla, CA, 92037, USA.
| |
Collapse
|
10
|
Mangin EN, Chen J, Lin J, Li N. Behavioral measurements of motor readiness in mice. Curr Biol 2023; 33:3610-3624.e4. [PMID: 37582373 PMCID: PMC10529875 DOI: 10.1016/j.cub.2023.07.029] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2023] [Revised: 05/09/2023] [Accepted: 07/18/2023] [Indexed: 08/17/2023]
Abstract
Motor planning facilitates rapid and precise execution of volitional movements. Although motor planning has been classically studied in humans and monkeys, the mouse has become an increasingly popular model system to study neural mechanisms of motor planning. It remains yet untested whether mice and primates share common behavioral features of motor planning. We combined videography and a delayed response task paradigm in an autonomous behavioral system to measure motor planning in non-body-restrained mice. Motor planning resulted in both reaction time (RT) savings and increased movement accuracy, replicating classic effects in primates. We found that motor planning was reflected in task-relevant body features. Both the specific actions prepared and the degree of motor readiness could be read out online during motor planning. The online readout further revealed behavioral evidence of simultaneous preparation for multiple actions under uncertain conditions. These results validate the mouse as a model to study motor planning, demonstrate body feature movements as a powerful real-time readout of motor readiness, and offer behavioral evidence that motor planning can be a parallel process that permits rapid selection of multiple prepared actions.
Collapse
Affiliation(s)
- Elise N Mangin
- Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA
| | - Jian Chen
- Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA
| | - Jing Lin
- Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA
| | - Nuo Li
- Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA.
| |
Collapse
|
11
|
Llobera J, Charbonnier C. Physics-based character animation and human motor control. Phys Life Rev 2023; 46:190-219. [PMID: 37480729 DOI: 10.1016/j.plrev.2023.06.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 06/25/2023] [Indexed: 07/24/2023]
Abstract
Motor neuroscience and physics-based character animation (PBCA) approach human and humanoid control from different perspectives. The primary goal of PBCA is to control the movement of a ragdoll (humanoid or animal) applying forces and torques within a physical simulation. The primary goal of motor neuroscience is to understand the contribution of different parts of the nervous system to generate coordinated movements. We review the functional principles and the functional anatomy of human motor control and the main strategies used in PBCA. We then explore common research points by discussing the functional anatomy and ongoing debates in motor neuroscience from the perspective of PBCA. We also suggest there are several benefits to be found in studying sensorimotor integration and human-character coordination through closer collaboration between these two fields.
Collapse
Affiliation(s)
- Joan Llobera
- Artanim Foundation, 40, chemin du Grand-Puits, 1217 Meyrin - Geneva, Switzerland.
| | - Caecilia Charbonnier
- Artanim Foundation, 40, chemin du Grand-Puits, 1217 Meyrin - Geneva, Switzerland
| |
Collapse
|
12
|
Yozevitch R, Dahan A, Seada T, Appel D, Gvirts H. Classifying interpersonal synchronization states using a data-driven approach: implications for social interaction understanding. Sci Rep 2023; 13:11150. [PMID: 37429957 PMCID: PMC10333224 DOI: 10.1038/s41598-023-37316-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2022] [Accepted: 06/20/2023] [Indexed: 07/12/2023] Open
Abstract
This study presents a data-driven approach to identifying interpersonal motor synchrony states by analyzing hand movements captured from a 3D depth camera. Utilizing a single frame from the experiment, an XGBoost machine learning model was employed to differentiate between spontaneous and intentional synchrony modes with nearly [Formula: see text] accuracy. Our findings demonstrate a consistent pattern across subjects, revealing that movement velocity tends to be slower in synchrony modes. These insights support the notion that the relationship between velocity and synchrony is influenced by the cognitive load required for the task, with slower movements leading to higher synchrony in tasks demanding higher cognitive load. This work not only contributes to the limited literature on algorithms for identifying interpersonal synchrony but also has potential implications for developing new metrics to assess real-time human social interactions, understanding social interaction, and diagnosing and developing treatment strategies for social deficits associated with conditions such as Autism Spectrum Disorder.
Collapse
Affiliation(s)
- Roi Yozevitch
- Department of Computer Science, Ariel University, Ariel, 40700, Israel.
| | - Anat Dahan
- Department of Software Engineering, Braude College of Engineering, Karmiel, 216100, Israel
| | - Talia Seada
- Department of Computer Science, Ariel University, Ariel, 40700, Israel
| | - Daniel Appel
- Department of Computer Science, Ariel University, Ariel, 40700, Israel
| | - Hila Gvirts
- Department of Behavioral Sciences, Ariel University, Ariel, 40700, Israel
| |
Collapse
|
13
|
Wang ZA, Chen S, Liu Y, Liu D, Svoboda K, Li N, Druckmann S. Not everything, not everywhere, not all at once: a study of brain-wide encoding of movement. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.06.08.544257. [PMID: 37333216 PMCID: PMC10274914 DOI: 10.1101/2023.06.08.544257] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/20/2023]
Abstract
Activity related to movement is found throughout sensory and motor regions of the brain. However, it remains unclear how movement-related activity is distributed across the brain and whether systematic differences exist between brain areas. Here, we analyzed movement related activity in brain-wide recordings containing more than 50,000 neurons in mice performing a decision-making task. Using multiple techniques, from markers to deep neural networks, we find that movement-related signals were pervasive across the brain, but systematically differed across areas. Movement-related activity was stronger in areas closer to the motor or sensory periphery. Delineating activity in terms of sensory- and motor-related components revealed finer scale structures of their encodings within brain areas. We further identified activity modulation that correlates with decision-making and uninstructed movement. Our work charts out a largescale map of movement encoding and provides a roadmap for dissecting different forms of movement and decision-making related encoding across multi-regional neural circuits.
Collapse
|
14
|
Sandbrink KJ, Mamidanna P, Michaelis C, Bethge M, Mathis MW, Mathis A. Contrasting action and posture coding with hierarchical deep neural network models of proprioception. eLife 2023; 12:e81499. [PMID: 37254843 PMCID: PMC10361732 DOI: 10.7554/elife.81499] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 05/16/2023] [Indexed: 06/01/2023] Open
Abstract
Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body's state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one's posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks' units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.
Collapse
Affiliation(s)
- Kai J Sandbrink
- The Rowland Institute at Harvard, Harvard UniversityCambridgeUnited States
| | - Pranav Mamidanna
- Tübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical PhysicsTübingenGermany
| | - Claudio Michaelis
- Tübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical PhysicsTübingenGermany
| | - Matthias Bethge
- Tübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical PhysicsTübingenGermany
| | - Mackenzie Weygandt Mathis
- The Rowland Institute at Harvard, Harvard UniversityCambridgeUnited States
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de LausanneGenèveSwitzerland
| | - Alexander Mathis
- The Rowland Institute at Harvard, Harvard UniversityCambridgeUnited States
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de LausanneGenèveSwitzerland
| |
Collapse
|
15
|
Couzin ID, Heins C. Emerging technologies for behavioral research in changing environments. Trends Ecol Evol 2023; 38:346-354. [PMID: 36509561 DOI: 10.1016/j.tree.2022.11.008] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 11/14/2022] [Accepted: 11/21/2022] [Indexed: 12/13/2022]
Abstract
The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.
Collapse
Affiliation(s)
- Iain D Couzin
- Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour & Department of Biology, University of Konstanz, Germany.
| | - Conor Heins
- Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour & Department of Biology, University of Konstanz, Germany
| |
Collapse
|
16
|
Zhao Y, Zhang M, Wu H, He X, Todoh M. Neuromechanics-Based Neural Feedback Controller for Planar Arm Reaching Movements. Bioengineering (Basel) 2023; 10:bioengineering10040436. [PMID: 37106623 PMCID: PMC10136284 DOI: 10.3390/bioengineering10040436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 03/22/2023] [Accepted: 03/24/2023] [Indexed: 04/29/2023] Open
Abstract
Based on the principles of neuromechanics, human arm movements result from the dynamic interaction between the nervous, muscular, and skeletal systems. To develop an effective neural feedback controller for neuro-rehabilitation training, it is important to consider both the effects of muscles and skeletons. In this study, we designed a neuromechanics-based neural feedback controller for arm reaching movements. To achieve this, we first constructed a musculoskeletal arm model based on the actual biomechanical structure of the human arm. Subsequently, a hybrid neural feedback controller was developed that mimics the multifunctional areas of the human arm. The performance of this controller was then validated through numerical simulation experiments. The simulation results demonstrated a bell-shaped movement trajectory, consistent with the natural motion of human arm movements. Furthermore, the experiment testing the tracking ability of the controller revealed real-time errors within one millimeter, with the tensile force generated by the controller's muscles being stable and maintained at a low value, thereby avoiding the issue of muscle strain that can occur due to excessive excitation during the neurorehabilitation process.
Collapse
Affiliation(s)
- Yongkun Zhao
- Division of Human Mechanical Systems and Design, Graduate School of Engineering, Hokkaido University, Sapporo 060-8628, Japan
- Division of Bioengineering, Graduate School of Engineering Science, Osaka University, Osaka 560-8531, Japan
| | - Mingquan Zhang
- State Key Laboratory of Bioelectronics, Jiangsu Provincial Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Haijun Wu
- Division of Mechanical and Aerospace Engineering, Faculty of Engineering, Hokkaido University, Sapporo 060-8628, Japan
| | - Xiangkun He
- Department of Bioengineering, Faculty of Engineering, Imperial College London, London SW7 2AZ, UK
| | - Masahiro Todoh
- Division of Mechanical and Aerospace Engineering, Faculty of Engineering, Hokkaido University, Sapporo 060-8628, Japan
| |
Collapse
|
17
|
Frey M, Mathis MW, Mathis A. NeuroAI: If grid cells are the answer, is path integration the question? Curr Biol 2023; 33:R190-R192. [PMID: 36917942 DOI: 10.1016/j.cub.2023.01.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2023]
Abstract
Spatially modulated neurons known as grid cells are thought to play an important role in spatial cognition. A new study has found that units with grid-cell-like properties can emerge within artificial neural networks trained to path integrate, and developed a unifying theory explaining the formation of these cells which shows what circuit constraints are necessary and how learned systems carry out path integration.
Collapse
Affiliation(s)
- Markus Frey
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland.
| | - Mackenzie W Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland.
| | - Alexander Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland.
| |
Collapse
|
18
|
Kim J, Kim DG, Jung W, Suh GSB. Evaluation of mouse behavioral responses to nutritive versus nonnutritive sugar using a deep learning-based 3D real-time pose estimation system. J Neurogenet 2023:1-6. [PMID: 36790034 DOI: 10.1080/01677063.2023.2174982] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/16/2023]
Abstract
Animals are able to detect the nutritional content of sugar independently of taste. When given a choice between nutritive sugar and nonnutritive sugar, animals develop a preference for nutritive sugar over nonnutritive sugar during a period of food deprivation (Buchanan et al., 2022; Dus et al., 2011; 2015; Tan et al., 2020; Tellez et al., 2016). To quantify behavioral features during an episode of licking nutritive versus nonnutritive sugar, we implemented a multi-vision, deep learning-based 3D pose estimation system, termed the AI Vision Analysis for Three-dimensional Action in Real-Time (AVATAR)(Kim et al., 2022). Using this method, we found that mice exhibit significantly different approach behavioral responses toward nutritive sugar versus nonnutritive sugar even before licking a sugar solution. Notably, the behavioral sequences during the approach toward nutritive versus nonnutritive sugar became significantly different over time. These results suggest that the nutritional value of sugar not only promotes its consumption but also elicits distinct repertoires of feeding behavior in deprived mice.
Collapse
Affiliation(s)
- Jineun Kim
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Dae-Gun Kim
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Wongyo Jung
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Greg S B Suh
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| |
Collapse
|
19
|
Mangin EN, Chen J, Lin J, Li N. Behavioral measurements of motor readiness in mice. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.03.527054. [PMID: 36778494 PMCID: PMC9915731 DOI: 10.1101/2023.02.03.527054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Motor planning facilitates rapid and precise execution of volitional movements. Although motor planning has been classically studied in humans and monkeys, the mouse has become an increasingly popular model system to study neural mechanisms of motor planning. It remains yet untested whether mice and primates share common behavioral features of motor planning. We combined videography and a delayed response task paradigm in an autonomous behavioral system to measure motor planning in non-body- restrained mice. Motor planning resulted in both reaction time savings and increased movement accuracy, replicating classic effects in primates. We found that motor planning was reflected in task-relevant body features. Both the specific actions prepared and the degree of motor readiness could be read out online during motor planning. The online readout further revealed behavioral evidence of simultaneous preparation for multiple actions under uncertain conditions. These results validate the mouse as a model to study motor planning, demonstrate body feature movements as a powerful real-time readout of motor readiness, and offer behavioral evidence that motor planning can be a parallel process that permits rapid selection of multiple prepared actions.
Collapse
Affiliation(s)
| | - Jian Chen
- Department of Neuroscience, Baylor College of Medicine
| | - Jing Lin
- Department of Neuroscience, Baylor College of Medicine
| | - Nuo Li
- Department of Neuroscience, Baylor College of Medicine
| |
Collapse
|
20
|
Momennejad I. A rubric for human-like agents and NeuroAI. Philos Trans R Soc Lond B Biol Sci 2023; 378:20210446. [PMID: 36511409 PMCID: PMC9745874 DOI: 10.1098/rstb.2021.0446] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Accepted: 10/27/2022] [Indexed: 12/15/2022] Open
Abstract
Researchers across cognitive, neuro- and computer sciences increasingly reference 'human-like' artificial intelligence and 'neuroAI'. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here, a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility or benchmark/engineering/computer science goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests-leading to both known and yet-unknown advances that may span decades to come. This article is part of a discussion meeting issue 'New approaches to 3D vision'.
Collapse
Affiliation(s)
- Ida Momennejad
- Microsoft Research NYC, Reinforcement Learning Station, 300 Lafayette, New York, NY 10012, USA
| |
Collapse
|
21
|
Korivand S, Jalili N, Gong J. Experiment protocols for brain-body imaging of locomotion: A systematic review. Front Neurosci 2023; 17:1051500. [PMID: 36937690 PMCID: PMC10014824 DOI: 10.3389/fnins.2023.1051500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Accepted: 02/06/2023] [Indexed: 03/05/2023] Open
Abstract
Introduction Human locomotion is affected by several factors, such as growth and aging, health conditions, and physical activity levels for maintaining overall health and well-being. Notably, impaired locomotion is a prevalent cause of disability, significantly impacting the quality of life of individuals. The uniqueness and high prevalence of human locomotion have led to a surge of research to develop experimental protocols for studying the brain substrates, muscle responses, and motion signatures associated with locomotion. However, from a technical perspective, reproducing locomotion experiments has been challenging due to the lack of standardized protocols and benchmarking tools, which impairs the evaluation of research quality and the validation of previous findings. Methods This paper addresses the challenges by conducting a systematic review of existing neuroimaging studies on human locomotion, focusing on the settings of experimental protocols, such as locomotion intensity, duration, distance, adopted brain imaging technologies, and corresponding brain activation patterns. Also, this study provides practical recommendations for future experiment protocols. Results The findings indicate that EEG is the preferred neuroimaging sensor for detecting brain activity patterns, compared to fMRI, fNIRS, and PET. Walking is the most studied human locomotion task, likely due to its fundamental nature and status as a reference task. In contrast, running has received little attention in research. Additionally, cycling on an ergometer at a speed of 60 rpm using fNIRS has provided some research basis. Dual-task walking tasks are typically used to observe changes in cognitive function. Moreover, research on locomotion has primarily focused on healthy individuals, as this is the scenario most closely resembling free-living activity in real-world environments. Discussion Finally, the paper outlines the standards and recommendations for setting up future experiment protocols based on the review findings. It discusses the impact of neurological and musculoskeletal factors, as well as the cognitive and locomotive demands, on the experiment design. It also considers the limitations imposed by the sensing techniques used, including the acceptable level of motion artifacts in brain-body imaging experiments and the effects of spatial and temporal resolutions on brain sensor performance. Additionally, various experiment protocol constraints that need to be addressed and analyzed are explained.
Collapse
Affiliation(s)
- Soroush Korivand
- Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL, United States
- Department of Computer Science, The University of Alabama, Tuscaloosa, AL, United States
| | - Nader Jalili
- Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL, United States
| | - Jiaqi Gong
- Department of Computer Science, The University of Alabama, Tuscaloosa, AL, United States
- *Correspondence: Jiaqi Gong
| |
Collapse
|
22
|
Kuo JY, Denman AJ, Beacher NJ, Glanzberg JT, Zhang Y, Li Y, Lin DT. Using deep learning to study emotional behavior in rodent models. Front Behav Neurosci 2022; 16:1044492. [PMID: 36483523 PMCID: PMC9722968 DOI: 10.3389/fnbeh.2022.1044492] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 11/02/2022] [Indexed: 11/25/2023] Open
Abstract
Quantifying emotional aspects of animal behavior (e.g., anxiety, social interactions, reward, and stress responses) is a major focus of neuroscience research. Because manual scoring of emotion-related behaviors is time-consuming and subjective, classical methods rely on easily quantified measures such as lever pressing or time spent in different zones of an apparatus (e.g., open vs. closed arms of an elevated plus maze). Recent advancements have made it easier to extract pose information from videos, and multiple approaches for extracting nuanced information about behavioral states from pose estimation data have been proposed. These include supervised, unsupervised, and self-supervised approaches, employing a variety of different model types. Representations of behavioral states derived from these methods can be correlated with recordings of neural activity to increase the scope of connections that can be drawn between the brain and behavior. In this mini review, we will discuss how deep learning techniques can be used in behavioral experiments and how different model architectures and training paradigms influence the type of representation that can be obtained.
Collapse
Affiliation(s)
- Jessica Y. Kuo
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| | - Alexander J. Denman
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| | - Nicholas J. Beacher
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| | - Joseph T. Glanzberg
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| | - Yan Zhang
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| | - Yun Li
- Department of Zoology and Physiology, University of Wyoming, Laramie, WY, United States
| | - Da-Ting Lin
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, United States
| |
Collapse
|
23
|
Stoumpou V, Vargas CDM, Schade PF, Boyd JL, Giannakopoulos T, Jarvis ED. Analysis of Mouse Vocal Communication (AMVOC): a deep, unsupervised method for rapid detection, analysis and classification of ultrasonic vocalisations. BIOACOUSTICS 2022. [DOI: 10.1080/09524622.2022.2099973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Affiliation(s)
- Vasiliki Stoumpou
- School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | - César D. M. Vargas
- Laboratory of Neurogenetics of Language, The Rockefeller University, New York, NY, USA
| | - Peter F. Schade
- Laboratory of Neurogenetics of Language, The Rockefeller University, New York, NY, USA
- Laboratory of Neural Systems, The Rockefeller University, New York, NY, USA
| | - J. Lomax Boyd
- Berman Institute of Bioethics, Johns Hopkins University, Baltimore, MD, USA
| | - Theodoros Giannakopoulos
- Computational Intelligence Lab, Institute of Informatics and Telecommunications, National Center of Scientific Research 'Demokritos', Athens, Greece
| | - Erich D. Jarvis
- Laboratory of Neurogenetics of Language, The Rockefeller University, New York, NY, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| |
Collapse
|
24
|
Avitan L, Stringer C. Not so spontaneous: Multi-dimensional representations of behaviors and context in sensory areas. Neuron 2022; 110:3064-3075. [PMID: 35863344 DOI: 10.1016/j.neuron.2022.06.019] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 06/20/2022] [Accepted: 06/22/2022] [Indexed: 11/27/2022]
Abstract
Sensory areas are spontaneously active in the absence of sensory stimuli. This spontaneous activity has long been studied; however, its functional role remains largely unknown. Recent advances in technology, allowing large-scale neural recordings in the awake and behaving animal, have transformed our understanding of spontaneous activity. Studies using these recordings have discovered high-dimensional spontaneous activity patterns, correlation between spontaneous activity and behavior, and dissimilarity between spontaneous and sensory-driven activity patterns. These findings are supported by evidence from developing animals, where a transition toward these characteristics is observed as the circuit matures, as well as by evidence from mature animals across species. These newly revealed characteristics call for the formulation of a new role for spontaneous activity in neural sensory computation.
Collapse
Affiliation(s)
- Lilach Avitan
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem 9190401, Israel.
| | | |
Collapse
|
25
|
Multi-Output Sequential Deep Learning Model for Athlete Force Prediction on a Treadmill Using 3D Markers. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12115424] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Reliable and innovative methods for estimating forces are critical aspects of biomechanical sports research. Using them, athletes can improve their performance and technique and reduce the possibility of fractures and other injuries. For this purpose, throughout this project, we proceeded to research the use of video in biomechanics. To refine this method, we propose an RNN trained on a biomechanical dataset of regular runners that measures both kinematics and kinetics. The model will allow analyzing, extracting, and drawing conclusions about continuous variable predictions through the body. It marks different anatomical and reflective points (96 in total, 32 per dimension) that will allow the prediction of forces (N) in three dimensions (Fx, Fy, Fz), measured on a treadmill with a force plate at different velocities (2.5 m/s, 3.5 m/s, 4.5 m/s). In order to obtain the best model, a grid search of different parameters that combined various types of layers (Simple, GRU, LSTM), loss functions (MAE, MSE, MSLE), and sampling techniques (down-sampling, up-sampling) helped obtain the best performing model (LSTM, MSE, down-sampling) achieved an average coefficient of determination of 0.68, although when excluding Fz it reached 0.92.
Collapse
|
26
|
de Bivort B, Buchanan S, Skutt-Kakaria K, Gajda E, Ayroles J, O’Leary C, Reimers P, Akhund-Zade J, Senft R, Maloney R, Ho S, Werkhoven Z, Smith MAY. Precise Quantification of Behavioral Individuality From 80 Million Decisions Across 183,000 Flies. Front Behav Neurosci 2022; 16:836626. [PMID: 35692381 PMCID: PMC9178272 DOI: 10.3389/fnbeh.2022.836626] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 03/22/2022] [Indexed: 01/18/2023] Open
Abstract
Individual animals behave differently from each other. This variability is a component of personality and arises even when genetics and environment are held constant. Discovering the biological mechanisms underlying behavioral variability depends on efficiently measuring individual behavioral bias, a requirement that is facilitated by automated, high-throughput experiments. We compiled a large data set of individual locomotor behavior measures, acquired from over 183,000 fruit flies walking in Y-shaped mazes. With this data set we first conducted a "computational ethology natural history" study to quantify the distribution of individual behavioral biases with unprecedented precision and examine correlations between behavioral measures with high power. We discovered a slight, but highly significant, left-bias in spontaneous locomotor decision-making. We then used the data to evaluate standing hypotheses about biological mechanisms affecting behavioral variability, specifically: the neuromodulator serotonin and its precursor transporter, heterogametic sex, and temperature. We found a variety of significant effects associated with each of these mechanisms that were behavior-dependent. This indicates that the relationship between biological mechanisms and behavioral variability may be highly context dependent. Going forward, automation of behavioral experiments will likely be essential in teasing out the complex causality of individuality.
Collapse
|
27
|
Parallel locomotor control strategies in mice and flies. Curr Opin Neurobiol 2022; 73:102516. [DOI: 10.1016/j.conb.2022.01.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Revised: 12/23/2021] [Accepted: 01/06/2022] [Indexed: 12/26/2022]
|
28
|
Connecting the dots in ethology: applying network theory to understand neural and animal collectives. Curr Opin Neurobiol 2022; 73:102532. [DOI: 10.1016/j.conb.2022.102532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2021] [Revised: 02/04/2022] [Accepted: 03/02/2022] [Indexed: 11/24/2022]
|
29
|
Kiang L, Woodington B, Carnicer-Lombarte A, Malliaras G, Barone DG. Spinal cord bioelectronic interfaces: opportunities in neural recording and clinical challenges. J Neural Eng 2022; 19. [PMID: 35320780 DOI: 10.1088/1741-2552/ac605f] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 03/23/2022] [Indexed: 11/11/2022]
Abstract
Bioelectronic stimulation of the spinal cord has demonstrated significant progress in restoration of motor function in spinal cord injury (SCI). The proximal, uninjured spinal cord presents a viable target for the recording and generation of control signals to drive targeted stimulation. Signals have been directly recorded from the spinal cord in behaving animals and correlated with limb kinematics. Advances in flexible materials, electrode impedance and signal analysis will allow SCR to be used in next-generation neuroprosthetics. In this review, we summarize the technological advances enabling progress in SCR and describe systematically the clinical challenges facing spinal cord bioelectronic interfaces and potential solutions, from device manufacture, surgical implantation to chronic effects of foreign body reaction and stress-strain mismatches between electrodes and neural tissue. Finally, we establish our vision of bi-directional closed-loop spinal cord bioelectronic bypass interfaces that enable the communication of disrupted sensory signals and restoration of motor function in SCI.
Collapse
Affiliation(s)
- Lei Kiang
- Orthopaedic Surgery, Singapore General Hospital, Outram Road, Singapore, Singapore, 169608, SINGAPORE
| | - Ben Woodington
- Department of Engineering, University of Cambridge, Electrical Engineering Division, 9 JJ Thomson Ave, Cambridge, Cambridge, CB2 1TN, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Alejandro Carnicer-Lombarte
- Clinical Neurosciences, University of Cambridge, Bioelectronics Laboratory, Cambridge, CB2 0PY, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - George Malliaras
- University of Cambridge, University of Cambridge, Cambridge, CB2 1TN, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Damiano G Barone
- Department of Engineering, University of Cambridge, Electrical Engineering Division, 9 JJ Thomson Ave, Cambridge, Cambridge, Cambridgeshire, CB2 1TN, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
30
|
Vandevoorde K, Vollenkemper L, Schwan C, Kohlhase M, Schenck W. Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks. SENSORS (BASEL, SWITZERLAND) 2022; 22:2481. [PMID: 35408094 PMCID: PMC9002555 DOI: 10.3390/s22072481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Revised: 03/18/2022] [Accepted: 03/20/2022] [Indexed: 11/03/2022]
Abstract
Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.
Collapse
Affiliation(s)
- Koenraad Vandevoorde
- Center for Applied Data Science (CfADS), Faculty of Engineering and Mathematics, Bielefeld University of Applied Sciences, 33619 Bielefeld, Germany; (L.V.); (C.S.); (M.K.)
| | | | | | | | - Wolfram Schenck
- Center for Applied Data Science (CfADS), Faculty of Engineering and Mathematics, Bielefeld University of Applied Sciences, 33619 Bielefeld, Germany; (L.V.); (C.S.); (M.K.)
| |
Collapse
|
31
|
Optogenetic stimulation of glutamatergic neurons in the cuneiform nucleus controls locomotion in a mouse model of Parkinson's disease. Proc Natl Acad Sci U S A 2021; 118:2110934118. [PMID: 34670837 DOI: 10.1073/pnas.2110934118] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/07/2021] [Indexed: 01/22/2023] Open
Abstract
In Parkinson's disease (PD), the loss of midbrain dopaminergic cells results in severe locomotor deficits, such as gait freezing and akinesia. Growing evidence indicates that these deficits can be attributed to the decreased activity in the mesencephalic locomotor region (MLR), a brainstem region controlling locomotion. Clinicians are exploring the deep brain stimulation of the MLR as a treatment option to improve locomotor function. The results are variable, from modest to promising. However, within the MLR, clinicians have targeted the pedunculopontine nucleus exclusively, while leaving the cuneiform nucleus unexplored. To our knowledge, the effects of cuneiform nucleus stimulation have never been determined in parkinsonian conditions in any animal model. Here, we addressed this issue in a mouse model of PD, based on the bilateral striatal injection of 6-hydroxydopamine, which damaged the nigrostriatal pathway and decreased locomotor activity. We show that selective optogenetic stimulation of glutamatergic neurons in the cuneiform nucleus in mice expressing channelrhodopsin in a Cre-dependent manner in Vglut2-positive neurons (Vglut2-ChR2-EYFP mice) increased the number of locomotor initiations, increased the time spent in locomotion, and controlled locomotor speed. Using deep learning-based movement analysis, we found that the limb kinematics of optogenetic-evoked locomotion in pathological conditions were largely similar to those recorded in intact animals. Our work identifies the glutamatergic neurons of the cuneiform nucleus as a potentially clinically relevant target to improve locomotor activity in parkinsonian conditions. Our study should open avenues to develop the targeted stimulation of these neurons using deep brain stimulation, pharmacotherapy, or optogenetics.
Collapse
|
32
|
Mocellin P, Mikulovic S. The Role of the Medial Septum-Associated Networks in Controlling Locomotion and Motivation to Move. Front Neural Circuits 2021; 15:699798. [PMID: 34366795 PMCID: PMC8340000 DOI: 10.3389/fncir.2021.699798] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2021] [Accepted: 06/28/2021] [Indexed: 12/29/2022] Open
Abstract
The Medial Septum and diagonal Band of Broca (MSDB) was initially studied for its role in locomotion. However, the last several decades were focussed on its intriguing function in theta rhythm generation. Early studies relied on electrical stimulation, lesions and pharmacological manipulation, and reported an inconclusive picture regarding the role of the MSDB circuits. Recent studies using more specific methodologies have started to elucidate the differential role of the MSDB's specific cell populations in controlling both theta rhythm and behaviour. In particular, a novel theory is emerging showing that different MSDB's cell populations project to different brain regions and control distinct aspects of behaviour. While the majority of these behaviours involve movement, increasing evidence suggests that MSDB-related networks govern the motivational aspect of actions, rather than locomotion per se. Here, we review the literature that links MSDB, theta activity, and locomotion and propose open questions, future directions, and methods that could be employed to elucidate the diverse roles of the MSDB-associated networks.
Collapse
Affiliation(s)
- Petra Mocellin
- Department of Cellular Neuroscience, Leibniz Institute for Neurobiology, Magdeburg, Germany
- International Max Planck Research School for Brain and Behavior, Bonn, Germany
| | - Sanja Mikulovic
- Research Group Cognition and Emotion, Leibniz Institute for Neurobiology, Magdeburg, Germany
| |
Collapse
|