1
|
Fontana BD, Canzian J, Rosemberg DB. Swimming into the future: Machine learning in zebrafish behavioral research. Prog Neuropsychopharmacol Biol Psychiatry 2025; 139:111398. [PMID: 40368230 DOI: 10.1016/j.pnpbp.2025.111398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/17/2024] [Revised: 05/05/2025] [Accepted: 05/10/2025] [Indexed: 05/16/2025]
Abstract
The zebrafish (Danio rerio) has emerged as a powerful organism in behavioral neuroscience, offering invaluable insights into the neural circuits and molecular pathways underlying complex behaviors. Although the knowledge of zebrafish behavioral repertoire is expanding rapidly, fundamental questions regarding complex behaviors remain poorly explored. Recent advances in machine learning offer potential for enhancing zebrafish behavioral analysis, enabling more precise, scalable, and unbiased assessments when compared to the traditional method. Thus, machine learning automates tracking and pattern recognition, uncovering new behavioral phenotypes and streamlining analysis typically manually assessed. Here, we highlight the potential use of machine learning tools in zebrafish-based models uncovering nuanced behavioral phenotypes to accelerate discoveries in translational neurobehavioral research, addressing the challenges and ethical considerations in the field. We emphasize that associating machine learning with zebrafish behavioral research, significant advances to elucidate neural and molecular mechanisms driving complex behaviors are expected. Collectively, the progressive refinement of these methods by enabling more detailed and efficient analysis will not only enhance the utility of zebrafish in translational neuroscience, but also contribute to develop more effective models of human disorders and in the search of potential neuroprotective strategies.
Collapse
Affiliation(s)
- Barbara D Fontana
- Laboratory of Experimental Neuropsychobiology, Department of Biochemistry and Molecular Biology, Federal University of Santa Maria, Santa Maria, RS, Brazil; Graduate Program in Biological Sciences: Toxicological Biochemistry, Federal University of Santa Maria, Santa Maria, RS, Brazil.
| | - Julia Canzian
- Laboratory of Experimental Neuropsychobiology, Department of Biochemistry and Molecular Biology, Federal University of Santa Maria, Santa Maria, RS, Brazil; Graduate Program in Biological Sciences: Toxicological Biochemistry, Federal University of Santa Maria, Santa Maria, RS, Brazil
| | - Denis B Rosemberg
- Laboratory of Experimental Neuropsychobiology, Department of Biochemistry and Molecular Biology, Federal University of Santa Maria, Santa Maria, RS, Brazil; Graduate Program in Biological Sciences: Toxicological Biochemistry, Federal University of Santa Maria, Santa Maria, RS, Brazil; The International Zebrafish Neuroscience Research Consortium (ZNRC), Slidell, LA, United States.
| |
Collapse
|
2
|
Liu J, Liu T, Hu Z, Wu F, Guo W, Wu H, Wang Z, Men Y, Yin S, Garber PA, Dunn D, Chapman CA, He G, Guo F, Pan R, Zhang T, Zhao Y, Xu P, Li B, Guo S. An Automated AI Framework for Quantitative Measurement of Mammalian Behavior. Integr Zool 2025. [PMID: 40230073 DOI: 10.1111/1749-4877.12985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/16/2025]
Abstract
Despite the large amount of video data captured during ethological studies of wild mammals, there is no widely accepted method available to automatically and quantitatively measure and analyze animal behavior. We developed a framework using facial recognition and deep learning to automatically track, measure, and quantify the behavior of single or multiple individuals from 10 distinct mammalian taxa, including three species of primates, three species of bovids, three species of carnivores, and one species of equid. We used spatiotemporal information based on animal skeleton models to recognize a set of distinct behaviors such as walking, feeding, grooming, and resting, and achieved an accuracy ranging from 0.82 to 0.96. Accuracies of validation videos ranged from 0.80 to 0.99. Our study offers an innovative analytical platform for the rapid and accurate evaluation of animal behavior in both captive and field settings.
Collapse
Affiliation(s)
- Jia Liu
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Tao Liu
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Zhengfeng Hu
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Fan Wu
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Wenjie Guo
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Haojie Wu
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Zhan Wang
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Yiyi Men
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Shuang Yin
- School of Information Sciences and Technology, Northwest University, Xi'an, China
| | - Paul A Garber
- Department of Anthropology and Program in Ecology, Evolution, and Conservation Biology, University of Illinois, Urbana, Illinois, USA
- International Centre of Biodiversity and Primate Conservation, Dali University, Dali, China
| | - Derek Dunn
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Colin A Chapman
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
- Biology Department, Vancouver Island University, Nanaimo, British Columbia, Canada
| | - Gang He
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
| | - Felix Guo
- Rangitoto College, Auckland, New Zealand
| | - Ruliang Pan
- PRL School of Anatomy, Physiology and Human Biology, University of Western Australia (M309), Crawley, Western Australia, Australia
| | - Tongzuo Zhang
- Northwest Institute of Plateau Biology, Chinese Academy of Sciences, Xining, China
| | - Yang Zhao
- Xi'an Qinling Wildlife Park, Xi'an, 710100, China
| | - Pengfei Xu
- School of Information Sciences and Technology, Northwest University, Xi'an, China
- Shaanxi International Joint Research Centre for the Battery-free Internet of Things, Xi'an, China
- Institute of Internet of Things, Northwest University, Xi'an, 710127, China
| | - Baoguo Li
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
- Shaanxi Institute of Zoology, Xi'an, China
- College of Life Science, Yanan University, Yanan, China
| | - Songtao Guo
- Shaanxi Key Laboratory for Animal Conservation, School of Life Sciences, Northwest University, Xi'an, China
- Northwest Institute of Plateau Biology, Chinese Academy of Sciences, Xining, China
| |
Collapse
|
3
|
Vogg R, Lüddecke T, Henrich J, Dey S, Nuske M, Hassler V, Murphy D, Fischer J, Ostner J, Schülke O, Kappeler PM, Fichtel C, Gail A, Treue S, Scherberger H, Wörgötter F, Ecker AS. Computer vision for primate behavior analysis in the wild. Nat Methods 2025:10.1038/s41592-025-02653-y. [PMID: 40211003 DOI: 10.1038/s41592-025-02653-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Accepted: 02/28/2025] [Indexed: 04/12/2025]
Abstract
Advances in computer vision and increasingly widespread video-based behavioral monitoring are currently transforming how we study animal behavior. However, there is still a gap between the prospects and practical application, especially in videos from the wild. In this Perspective, we aim to present the capabilities of current methods for behavioral analysis, while at the same time highlighting unsolved computer vision problems that are relevant to the study of animal behavior. We survey state-of-the-art methods for computer vision problems relevant to the video-based study of individualized animal behavior, including object detection, multi-animal tracking, individual identification and (inter)action understanding. We then review methods for effort-efficient learning, one of the challenges from a practical perspective. In our outlook on the emerging field of computer vision for animal behavior, we argue that the field should develop approaches to unify detection, tracking, identification and (inter)action understanding in a single, video-based framework.
Collapse
Affiliation(s)
- Richard Vogg
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
| | - Timo Lüddecke
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
| | - Jonathan Henrich
- Chairs of Statistics and Econometrics and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
| | - Sharmita Dey
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
| | - Matthias Nuske
- Department for Computational Neuroscience, Third Physics Institute, University of Göttingen, Göttingen, Germany
| | - Valentin Hassler
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
| | - Derek Murphy
- Cognitive Ethology Laboratory, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Department for Primate Cognition, Johann-Friedrich-Blumenbach Institute, University of Göttingen, Göttingen, Germany
| | - Julia Fischer
- Cognitive Ethology Laboratory, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Department for Primate Cognition, Johann-Friedrich-Blumenbach Institute, University of Göttingen, Göttingen, Germany
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany
| | - Julia Ostner
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Behavioral Ecology Department, University of Göttingen, Göttingen, Germany
- Social Evolution in Primates Group, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
| | - Oliver Schülke
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Behavioral Ecology Department, University of Göttingen, Göttingen, Germany
- Social Evolution in Primates Group, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
| | - Peter M Kappeler
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Behavioral Ecology & Sociobiology Unit, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Department of Sociobiology/Anthropology, University of Göttingen, Göttingen, Germany
| | - Claudia Fichtel
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Behavioral Ecology & Sociobiology Unit, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
| | - Alexander Gail
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany
- Sensorimotor Group, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Sensorimotor Neuroscience and Neuroprosthetics, Georg-Elias-Müller Institute, University of Göttingen, Göttingen, Germany
| | - Stefan Treue
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany
- Cognitive Neuroscience Laboratory, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Biological Psychology & Cognitive Neuroscience, Georg-Elias-Müller-Institute of Psychology, University of Göttingen, Göttingen, Germany
| | - Hansjörg Scherberger
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany
- Primate Neurobiology, Johann-Friedrich-Blumenbach-Institute for Zoology & Anthropology, University of Göttingen, Göttingen, Germany
- Neurobiology Laboratory, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
| | - Florentin Wörgötter
- Department for Computational Neuroscience, Third Physics Institute, University of Göttingen, Göttingen, Germany
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany
| | - Alexander S Ecker
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany.
- Leibniz ScienceCampus, German Primate Center, Leibniz Institute for Primate Research, Göttingen, Germany.
- Bernstein Center for Computational Neuroscience, University of Göttingen, Göttingen, Germany.
- Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany.
| |
Collapse
|
4
|
Moat GJ, Gaudet-Trafit M, Paul J, Bacardit J, Ben Hamed S, Poirier C. The MacqD deep-learning-based model for automatic detection of socially housed laboratory macaques. Sci Rep 2025; 15:11883. [PMID: 40195447 PMCID: PMC11977019 DOI: 10.1038/s41598-025-95180-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2024] [Accepted: 03/19/2025] [Indexed: 04/09/2025] Open
Abstract
Despite advancements in video-based behaviour analysis and detection models for various species, existing methods are suboptimal to detect macaques in complex laboratory environments. To address this gap, we present MacqD, a modified Mask R-CNN model incorporating a SWIN transformer backbone for enhanced attention-based feature extraction. MacqD robustly detects macaques in their home-cage under challenging scenarios, including occlusions, glass reflections, and overexposure to light. To evaluate MacqD and compare its performance against pre-existing macaque detection models, we collected and analysed video frames from 20 caged rhesus macaques at Newcastle University, UK. Our results demonstrate MacqD's superiority, achieving a median F1-score of 99% for frames with a single macaque in the focal cage (surpassing the next-best model by 21%) and 90% for frames with two macaques. Generalisation tests on frames from a different set of macaques from the same animal facility yielded median F1-scores of 95% for frames with a single macaque (surpassing the next-best model by 15%) and 81% for frames with two macaques (surpassing the alternative approach by 39% ). Finally, MacqD was applied to videos of paired macaques from another facility and resulted in F1-score of 90%, reflecting its strong generalisation capacity. This study highlights MacqD's effectiveness in accurately detecting macaques across diverse settings.
Collapse
Affiliation(s)
| | - Maxime Gaudet-Trafit
- Institut des Sciences Cognitives Marc Jeannerod, UMR5229, CNRS-Université Claude Bernard Lyon I, Bron, France
| | - Julian Paul
- Institut des Sciences Cognitives Marc Jeannerod, UMR5229, CNRS-Université Claude Bernard Lyon I, Bron, France
| | - Jaume Bacardit
- School of Computing, Newcastle University, Newcastle upon Tyne, UK.
| | - Suliann Ben Hamed
- Institut des Sciences Cognitives Marc Jeannerod, UMR5229, CNRS-Université Claude Bernard Lyon I, Bron, France
| | - Colline Poirier
- Biosciences Institute Centre for Behaviour and Evolution, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, UK.
| |
Collapse
|
5
|
Bucci MP, Dewberry LS, Staiger EA, Allen K, Brooks SA. AI-assisted digital video analysis reveals changes in gait among three-day event horses during competition. J Equine Vet Sci 2025; 146:105344. [PMID: 39778726 DOI: 10.1016/j.jevs.2025.105344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 12/30/2024] [Accepted: 12/31/2024] [Indexed: 01/11/2025]
Abstract
The value and welfare of performance horses is closely tied to locomotor behaviors, but we lack objective and quantitative measures for these characteristics, and qualitative approaches for assessing gait do not provide measures suitable for large-scale biomechanical research studies. Digital video analysis utilizing artificial intelligence-based strategies holds promise to meet the need for an economical, accurate, repeatable and objective technique for field quantification of equine locomotion. Here we describe pilot work using a consumer-level digital video camera to capture high-resolution and high-speed videos of horses moving at the trot during mandatory inspections for international-level eventing competitions. We assessed 194 horses from five different competition venues, recorded at pre-competition (first) and post-cross-country (second) inspections as a model of gait change following exertion. We labeled twenty-six keypoints on each frame with DeepLabCut and processed the resulting tracking data using MatLab to derive quantitative gait parameters. Once trained, the DeepLabCut model labeled the 388 videos in just minutes, a task that would have otherwise taken months of human effort to complete. A Generalized Linear Mixed Model (GLMM) examining seven gait parameters identified significant changes in duty factor, speed, and forelimb swing range following the completion of the cross-country phase (P ≤ 0.05). Despite some limitations, video analysis through artificial intelligence proved capable of quantifying several gait parameters quickly, efficiently, and without the need for specialized equipment, making this tool a promising option for future biomechanical research in the athletic horse.
Collapse
Affiliation(s)
- Madelyn P Bucci
- University of Florida Department of Animal Sciences, 2250 Shealy Dr., Gainesville, FL, 32611 United States.
| | - L Savannah Dewberry
- University of Florida Department of Biomedical Engineering, 1275 Center Dr., Gainesville, FL, 32610 United States.
| | - Elizabeth A Staiger
- Texas A&M University - Kingsville Department of Animal Science and Veterinary Technology, 1150 W. Engineering Ave., Kleberg Hall, Kingsville, TX, 78363 United States.
| | - Kyle Allen
- University of Florida Department of Biomedical Engineering, 1275 Center Dr., Gainesville, FL, 32610 United States; University of Florida Department of Orthopedics and Sports Medicine, 3450 Hull Rd., Gainesville, FL 32607, United States.
| | - Samantha A Brooks
- University of Florida Department of Animal Sciences, 2250 Shealy Dr., Gainesville, FL, 32611 United States; UF Genetics Institute, 2033 Mowry Rd., Gainesville, FL, 32611 United States.
| |
Collapse
|
6
|
Tozzi F, Zhang Y, Narayanan R, Roqueiro D, O'Connor E. Forestwalk: A Machine Learning Workflow Brings New Insights Into Posture and Balance in Rodent Beam Walking. Eur J Neurosci 2025; 61:e70033. [PMID: 40070112 PMCID: PMC11897687 DOI: 10.1111/ejn.70033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2024] [Revised: 02/11/2025] [Accepted: 02/14/2025] [Indexed: 03/15/2025]
Abstract
The beam walk is widely used to study coordination and balance in rodents. While the task has ethological validity, the main endpoints of "foot slip counts" and "time to cross" are prone to human-rater variability and offer limited sensitivity and specificity. We asked if machine learning-based methods could reveal previously hidden, but biologically relevant, insights from the task. Marker-less pose estimation, using DeepLabCut, was deployed to label 13 anatomical key points on mice traversing the beam. Next, we automated classical endpoint detection, including foot slips, with high recall (> 90%) and precision (> 80%). Using data derived from key point tracking, a total of 395 features were engineered and a random forest classifier deployed that, together with skeletal visualizations, could test for group differences and identify determinant features. This workflow, named Forestwalk, uncovered pharmacological treatment effects in C57BL/6J mice, revealed phenotypes in transgenic mice used to study Angelman syndrome and SLC6A1-related neurodevelopmental disorder, and will facilitate a deeper understanding of how the brain controls balance in health and disease.
Collapse
Affiliation(s)
- Francesca Tozzi
- Neuroscience and Rare Diseases Discovery and Translational Area, Roche Pharma Research and Early Development, Roche Innovation Center BaselF. Hoffmann‐La Roche LtdBaselSwitzerland
| | - Yan‐Ping Zhang
- Data and Analytics, Roche Pharma Research and Early Development, Roche Innovation Center BaselF. Hoffmann‐La Roche LtdBaselSwitzerland
| | - Ramanathan Narayanan
- Neuroscience and Rare Diseases Discovery and Translational Area, Roche Pharma Research and Early Development, Roche Innovation Center BaselF. Hoffmann‐La Roche LtdBaselSwitzerland
| | - Damian Roqueiro
- Neuroscience and Rare Diseases Discovery and Translational Area, Roche Pharma Research and Early Development, Roche Innovation Center BaselF. Hoffmann‐La Roche LtdBaselSwitzerland
| | - Eoin C. O'Connor
- Neuroscience and Rare Diseases Discovery and Translational Area, Roche Pharma Research and Early Development, Roche Innovation Center BaselF. Hoffmann‐La Roche LtdBaselSwitzerland
| |
Collapse
|
7
|
Rizzo A, Garçon-Poca MZ, Essmann A, Souza AJ, Michaelides M, Ciruela F, Bonaventura J. The dopaminergic effects of esketamine are mediated by a dual mechanism involving glutamate and opioid receptors. Mol Psychiatry 2025:10.1038/s41380-025-02931-3. [PMID: 39972056 DOI: 10.1038/s41380-025-02931-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2024] [Revised: 01/13/2025] [Accepted: 02/11/2025] [Indexed: 02/21/2025]
Abstract
Esketamine represents a new class of drugs for treating mood disorders. Unlike traditional monoaminergic-based therapies, esketamine primarily targets N-methyl-D-aspartate receptors (NMDAR). However, esketamine is a complex drug with low affinity for NMDAR and can also bind to other targets, such as opioid receptors. Its precise mechanism of action for its antidepressant properties remains debated, as does its potential for misuse. A key component at the intersection of mood and reward processing is the dopaminergic system. In this study, we evaluated the effects of esketamine in locomotion, anxiety tests and operant responding and we used in vivo fiber photometry to explore the neurochemical effects of esketamine in the nucleus accumbens of mice. Our findings demonstrated multifaceted effects of esketamine on neurotransmitter dynamics. In freely behaving mice, esketamine increased locomotion and increased extracellular dopamine tone -by impairing dopamine clearance rather than promoting dopamine release- while decreasing glutamatergic activity. However, it decreased dopamine spontaneous release event frequency and impaired reward-evoked dopamine release, leading to a reduction in operant responding rates. These dopaminergic effects were partially, and conditionally, blocked by the opioid antagonist naloxone and required glutamatergic input. In summary, our study reveals a complex interaction between neurotransmitter systems, suggesting that the neurochemical effects of esketamine are both circuit- and state-dependent.
Collapse
Affiliation(s)
- Arianna Rizzo
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain
| | - Maria Zelai Garçon-Poca
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain
| | - Amelie Essmann
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain
| | - Adriana Jesus Souza
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain
| | - Michael Michaelides
- Biobehavioral Imaging and Molecular Neuropsychopharmacology Section, National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, USA
| | - Francisco Ciruela
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain
| | - Jordi Bonaventura
- Departament de Patologia i Terapèutica Experimental, Institut de Neurociències, Universitat de Barcelona, L'Hospitalet de Llobregat, Barcelona, Spain.
- Neuropharmacology and Pain Group, Neuroscience Program, Institut d'Investigació Biomèdica de Bellvitge, L'Hospitalet de Llobregat, Barcelona, Spain.
| |
Collapse
|
8
|
Kumaran SK, Solberg LE, Izquierdo-Gomez D, Cañon-Jones HA, Mage I, Noble C. Applying deep learning and the ecological home range concept to document the spatial distribution of Atlantic salmon parr (Salmo salar L.) in experimental tanks. Sci Rep 2025; 15:5976. [PMID: 39966514 PMCID: PMC11836443 DOI: 10.1038/s41598-025-90118-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2024] [Accepted: 02/11/2025] [Indexed: 02/20/2025] Open
Abstract
Measuring and monitoring fish welfare in aquaculture research relies on the use of outcome- (biotic) and input-based (e.g., abiotic) welfare indicators (WIs). Incorporating behavioural auditing into this toolbox can sometimes be challenging because sourcing quantitative data is often labour intensive and it can be a time-consuming process. Digitalization of this process via the use of computer vision and artificial intelligence can help automate and streamline the procedure, help gather continuous quantitative data and help process optimisation and assist in decision-making. The tool introduced in this study (1) adapts the DeepLabCut framework, based on computer vision and machine learning, to obtain pose estimation of Atlantic salmon parr under replicated experimental conditions, (2) quantifies the spatial distribution of the fish through a toolbox of metrics inspired by the ecological concepts home range and core area, and (3) applies it to inspect behavioural variability in and around feeding. This proof of concept study demonstrates the potential of our methodology for automating the analysis of fish behaviour in relation to home range and core area, including fish detection, spatial distribution and the variations within and between tanks. The impact of feeding on these patterns is also briefly outlined, using 5 days of experimental data as a demonstrative case study. This approach can provide stakeholders with valuable information on how the fish use their rearing environment in small-scale experimental settings and can be used for the further development of technologies for measuring and monitoring the behaviour of fish in research settings in future studies.
Collapse
|
9
|
Gedela NSS, Radawiec RD, Salim S, Richie J, Chestek C, Draelos A, Pelled G. In vivo electrophysiology recordings and computational modeling can predict octopus arm movement. Bioelectron Med 2025; 11:4. [PMID: 39948616 PMCID: PMC11827351 DOI: 10.1186/s42234-025-00166-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2024] [Accepted: 01/27/2025] [Indexed: 02/16/2025] Open
Abstract
The octopus has many features that make it advantageous for revealing principles of motor circuits and control and predicting behavior. Here, an array of carbon electrodes providing single-unit electrophysiology recordings were implanted into the octopus anterior nerve cord. The number of spikes and arm movements in response to stimulation at different locations along the arm were recorded. We observed that the number of spikes occurring within the first 100 ms after stimulation were predictive of the resultant movement response. Machine learning models showed that temporal electrophysiological features could be used to predict whether an arm movement occurred with 88.64% confidence, and if it was a lateral arm movement or a grasping motion with 75.45% confidence. Both supervised and unsupervised methods were applied to gain streaming measurements of octopus arm movements and how their motor circuitry produces rich movement types in real time. For kinematic analysis, deep learning models and unsupervised dimensionality reduction identified a consistent set of features that could be used to distinguish different types of arm movements. The neural circuits and the computational models identified here generated predictions for how to evoke a particular, complex movement in an orchestrated sequence for an individual motor circuit. This study demonstrates how real-time motor behaviors can be predicted and distinguished, contributing to the development of brain-machine interfaces. The ability to accurately model and predict complex movement patterns has broad implications for advancing technologies in robotics, neuroprosthetics, and artificial intelligence, paving the way for more sophisticated and adaptable systems.
Collapse
Affiliation(s)
| | - Ryan D Radawiec
- Department of Mechanical Engineering, Michigan State University, East Lansing, MI, USA
| | - Sachin Salim
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Julianna Richie
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Cynthia Chestek
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Anne Draelos
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
- Department of Computational Medicine & Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
| | - Galit Pelled
- Department of Mechanical Engineering, Michigan State University, East Lansing, MI, USA.
- Department of Radiology, Michigan State University, East Lansing, MI, USA.
| |
Collapse
|
10
|
Srivastava V, Muralidharan A, Swaminathan A, Poulose A. Anxiety in aquatics: Leveraging machine learning models to predict adult zebrafish behavior. Neuroscience 2025; 565:577-587. [PMID: 39675692 DOI: 10.1016/j.neuroscience.2024.12.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2024] [Revised: 12/04/2024] [Accepted: 12/06/2024] [Indexed: 12/17/2024]
Abstract
Accurate analysis of anxiety behaviors in animal models is pivotal for advancing neuroscience research and drug discovery. This study compares the potential of DeepLabCut, ZebraLab, and machine learning models to analyze anxiety-related behaviors in adult zebrafish. Using a dataset comprising video recordings of unstressed and pre-stressed zebrafish, we extracted features such as total inactivity duration/immobility, time spent at the bottom, time spent at the top and turn angles (large and small). We observed that the data obtained using DeepLabCut and ZebraLab were highly correlated. Using this data, we annotated behaviors as anxious and not anxious and trained several machine learning models, including Logistic Regression, Decision Tree, K-Nearest Neighbours (KNN), Random Forests, Naive Bayes Classifiers, and Support Vector Machines (SVMs). The effectiveness of these machine learning models was validated and tested on independent datasets. We found that some machine learning models, such as Decision Tree and Random Forests, performed excellently to differentiate between anxious and non-anxious behavior, even in the control group, where the differences between subjects were more subtle. Our findings show that upcoming technologies, such as machine learning models, are able to effectively and accurately analyze anxiety behaviors in zebrafish and provide a cost-effective method to analyze animal behavior.
Collapse
Affiliation(s)
- Vartika Srivastava
- School of Biology, Indian Institute of Science Education and Research Thiruvananthapuram (IISER TVM), Vithura, Thiruvananthapuram 695551, Kerala, India.
| | - Anagha Muralidharan
- School of Biology, Indian Institute of Science Education and Research Thiruvananthapuram (IISER TVM), Vithura, Thiruvananthapuram 695551, Kerala, India.
| | - Amrutha Swaminathan
- School of Biology, Indian Institute of Science Education and Research Thiruvananthapuram (IISER TVM), Vithura, Thiruvananthapuram 695551, Kerala, India.
| | - Alwin Poulose
- School of Data Science, Indian Institute of Science Education and Research Thiruvananthapuram (IISER TVM), Vithura, Thiruvananthapuram 695551, Kerala, India.
| |
Collapse
|
11
|
Michelot B, Corneyllie A, Thevenet M, Duffner S, Perrin F. A modular machine learning tool for holistic and fine-grained behavioral analysis. Behav Res Methods 2024; 57:24. [PMID: 39702505 DOI: 10.3758/s13428-024-02511-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/06/2024] [Indexed: 12/21/2024]
Abstract
Artificial intelligence techniques offer promising avenues for exploring human body features from videos, yet no freely accessible tool has reliably provided holistic and fine-grained behavioral analyses to date. To address this, we developed a machine learning tool based on a two-level approach: a first lower-level processing using computer vision for extracting fine-grained and comprehensive behavioral features such as skeleton or facial points, gaze, and action units; a second level of machine learning classification coupled with explainability providing modularity, to determine which behavioral features are triggered by specific environments. To validate our tool, we filmed 16 participants across six conditions, varying according to the presence of a person ("Pers"), a sound ("Snd"), or silence ("Rest"), and according to emotional levels using self-referential ("Self") and control ("Ctrl") stimuli. We demonstrated the effectiveness of our approach by extracting and correcting behavior from videos using two computer vision software (OpenPose and OpenFace) and by training two algorithms (XGBoost and long short-term memory [LSTM]) to differentiate between experimental conditions. High classification rates were achieved for "Pers" conditions versus "Snd" or "Rest" (AUC = 0.8-0.9), with explainability revealing actions units and gaze as key features. Additionally, moderate classification rates were attained for "Snd" versus "Rest" (AUC = 0.7), attributed to action units, limbs and head points, as well as for "Self" versus "Ctrl" (AUC = 0.7-0.8), due to facial points. These findings were consistent with a more conventional hypothesis-driven approach. Overall, our study suggests that our tool is well suited for holistic and fine-grained behavioral analysis and offers modularity for extension into more complex naturalistic environments.
Collapse
Affiliation(s)
- Bruno Michelot
- CAP Team, Centre de Recherche en Neurosciences de Lyon - INSERM U1028 - CNRS UMR 5292 - UCBL - UJM, 95 Boulevard Pinel, 69675, Bron, France.
| | - Alexandra Corneyllie
- CAP Team, Centre de Recherche en Neurosciences de Lyon - INSERM U1028 - CNRS UMR 5292 - UCBL - UJM, 95 Boulevard Pinel, 69675, Bron, France
| | - Marc Thevenet
- CAP Team, Centre de Recherche en Neurosciences de Lyon - INSERM U1028 - CNRS UMR 5292 - UCBL - UJM, 95 Boulevard Pinel, 69675, Bron, France
| | - Stefan Duffner
- IMAGINE Team, Laboratoire d'InfoRmatique en Image et Systèmes d'information - UMR 5205 CNRS - INSA Lyon, Université Claude Bernard Lyon 1 - Université Lumière Lyon 2 - École Centrale de Lyon, Lyon, France
| | - Fabien Perrin
- CAP Team, Centre de Recherche en Neurosciences de Lyon - INSERM U1028 - CNRS UMR 5292 - UCBL - UJM, 95 Boulevard Pinel, 69675, Bron, France
| |
Collapse
|
12
|
Grammer J, Valles R, Bowles A, Zelikowsky M. SAUSI: an integrative assay for measuring social aversion and motivation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.13.594023. [PMID: 38798428 PMCID: PMC11118329 DOI: 10.1101/2024.05.13.594023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Social aversion is a key feature of numerous mental health disorders such as Social Anxiety and Autism Spectrum Disorders. Nevertheless, the biobehavioral mechanisms underlying social aversion remain poorly understood. Progress in understanding the etiology of social aversion has been hindered by the lack of comprehensive tools to assess social aversion in model systems. Here, we created a new behavioral task - Selective Access to Unrestricted Social Interaction (SAUSI), which integrates elements of social motivation, hesitancy, decision-making, and free interaction to enable the wholistic assessment of social aversion in mice. Using this novel assay, we found that social isolation-induced social aversion in mice is largely driven by increases in social fear and social motivation. Deep learning analyses revealed a unique behavioral footprint underlying the socially aversive state produced by isolation, demonstrating the compatibility of modern computational approaches with SAUSI. Social aversion was further assessed using traditional assays - including the 3-chamber sociability assay and the resident intruder assay - which were sufficient to reveal fragments of a social aversion phenotype, including changes to either social motivation or social interaction, but which failed to provide a wholistic assessment of social aversion. Critically, these assays were not sufficient to reveal key components of social aversion, including social freezing and social hesitancy behaviors. Lastly, we demonstrated that SAUSI is generalizable, as it can be used to assess social aversion induced by non-social stressors, such as foot shock. Our findings debut a novel task for the behavioral toolbox - one which overcomes limitations of previous assays, allowing for both social choice as well as free interaction, and offers a new approach for assessing social aversion in rodents.
Collapse
Affiliation(s)
- Jordan Grammer
- Department of Neurobiology, University of Utah, United States
| | - Rene Valles
- Department of Neurobiology, University of Utah, United States
| | - Alexis Bowles
- Department of Neurobiology, University of Utah, United States
| | | |
Collapse
|
13
|
Lancaster TJ, Leatherbury KN, Shilova K, Streelman JT, McGrath PT. SARTAB, a scalable system for automated real-time behavior detection based on animal tracking and Region Of Interest analysis: validation on fish courtship behavior. Front Behav Neurosci 2024; 18:1509369. [PMID: 39703614 PMCID: PMC11655190 DOI: 10.3389/fnbeh.2024.1509369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2024] [Accepted: 11/13/2024] [Indexed: 12/21/2024] Open
Abstract
Methods from Machine Learning (ML) and Computer Vision (CV) have proven powerful tools for quickly and accurately analyzing behavioral recordings. The computational complexity of these techniques, however, often precludes applications that require real-time analysis: for example, experiments where a stimulus must be applied in response to a particular behavior or samples must be collected soon after the behavior occurs. Here, we describe SARTAB (Scalable Automated Real-Time Analysis of Behavior), a system that achieves automated real-time behavior detection by continuously monitoring animal positions relative to behaviorally relevant Regions Of Interest (ROIs). We then show how we used this system to detect infrequent courtship behaviors in Pseudotropheus demasoni (a species of Lake Malawi African cichlid fish) to collect neural tissue samples from actively behaving individuals for multiomic profiling at single nucleus resolution. Within this experimental context, we achieve high ROI and animal detection accuracies (mAP@[.5 : .95] of 0.969 and 0.718, respectively) and 100% classification accuracy on a set of 32 manually selected behavioral clips. SARTAB is unique in that all analysis runs on low-cost, edge-deployed hardware, making it a highly scalable and energy-efficient solution for real-time experimental feedback. Although our solution was developed specifically to study cichlid courtship behavior, the intrinsic flexibility of neural network analysis ensures that our approach can be adapted to novel species, behaviors, and environments.
Collapse
Affiliation(s)
- Tucker J. Lancaster
- McGrath Lab, Georgia Institute of Technology, School of Biological Sciences, Atlanta, GA, United States
| | - Kathryn N. Leatherbury
- Streelman Lab, Georgia Institute of Technology, School of Biological Sciences, Atlanta, GA, United States
| | - Kseniia Shilova
- McGrath Lab, Georgia Institute of Technology, School of Biological Sciences, Atlanta, GA, United States
| | - Jeffrey T. Streelman
- Streelman Lab, Georgia Institute of Technology, School of Biological Sciences, Atlanta, GA, United States
| | - Patrick T. McGrath
- McGrath Lab, Georgia Institute of Technology, School of Biological Sciences, Atlanta, GA, United States
| |
Collapse
|
14
|
Chiappa AS, Tano P, Patel N, Ingster A, Pouget A, Mathis A. Acquiring musculoskeletal skills with curriculum-based reinforcement learning. Neuron 2024; 112:3969-3983.e5. [PMID: 39357519 DOI: 10.1016/j.neuron.2024.09.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Revised: 07/29/2024] [Accepted: 09/04/2024] [Indexed: 10/04/2024]
Abstract
Efficient musculoskeletal simulators and powerful learning algorithms provide computational tools to tackle the grand challenge of understanding biological motor control. Our winning solution for the inaugural NeurIPS MyoChallenge leverages an approach mirroring human skill learning. Using a novel curriculum learning approach, we trained a recurrent neural network to control a realistic model of the human hand with 39 muscles to rotate two Baoding balls in the palm of the hand. In agreement with data from human subjects, the policy uncovers a small number of kinematic synergies, even though it is not explicitly biased toward low-dimensional solutions. However, selectively inactivating parts of the control signal, we found that more dimensions contribute to the task performance than suggested by traditional synergy analysis. Overall, our work illustrates the emerging possibilities at the interface of musculoskeletal physics engines, reinforcement learning, and neuroscience to advance our understanding of biological motor control.
Collapse
Affiliation(s)
- Alberto Silvio Chiappa
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Pablo Tano
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Nisheet Patel
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Abigaïl Ingster
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Alexandre Pouget
- Department of Fundamental Neuroscience, University of Geneva, 1205 Geneva, Switzerland
| | - Alexander Mathis
- Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland; Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland.
| |
Collapse
|
15
|
Martins DM, Manda JM, Goard MJ, Parker PRL. Building egocentric models of local space from retinal input. Curr Biol 2024; 34:R1185-R1202. [PMID: 39626632 PMCID: PMC11620475 DOI: 10.1016/j.cub.2024.10.057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/08/2024]
Abstract
Determining the location of objects relative to ourselves is essential for interacting with the world. Neural activity in the retina is used to form a vision-independent model of the local spatial environment relative to the body. For example, when an animal navigates through a forest, it rapidly shifts its gaze to identify the position of important objects, such as a tree obstructing its path. This seemingly trivial behavior belies a sophisticated neural computation. Visual information entering the brain in a retinocentric reference frame must be transformed into an egocentric reference frame to guide motor planning and action. This, in turn, allows the animal to extract the location of the tree and plan a path around it. In this review, we explore the anatomical, physiological, and computational implementation of retinocentric-to-egocentric reference frame transformations - a research area undergoing rapid progress stimulated by an ever-expanding molecular, physiological, and computational toolbox for probing neural circuits. We begin by summarizing evidence for retinocentric and egocentric reference frames in the brains of diverse organisms, from insects to primates. Next, we cover how distance estimation contributes to creating a three-dimensional representation of local space. We then review proposed implementations of reference frame transformations across different biological and artificial neural networks. Finally, we discuss how an internal egocentric model of the environment is maintained independently of the sensory inputs from which it is derived. By comparing findings across a variety of nervous systems and behaviors, we aim to inspire new avenues for investigating the neural basis of reference frame transformation, a canonical computation critical for modeling the external environment and guiding goal-directed behavior.
Collapse
Affiliation(s)
- Dylan M Martins
- Graduate Program in Dynamical Neuroscience, University of California, Santa Barbara, Santa Barbara, CA 93106, USA
| | - Joy M Manda
- Behavioral and Systems Neuroscience, Department of Psychology, Rutgers University, New Brunswick, NJ 08854, USA
| | - Michael J Goard
- Department of Psychological and Brain Sciences and Department of Molecular, Cellular, and Developmental Biology, University of California, Santa Barbara, Santa Barbara, CA 93106, USA.
| | - Philip R L Parker
- Behavioral and Systems Neuroscience, Department of Psychology, Rutgers University, New Brunswick, NJ 08854, USA.
| |
Collapse
|
16
|
Krishnan S, Dong C, Ratigan H, Morales-Rodriguez D, Cherian C, Sheffield M. A contextual fear conditioning paradigm in head-fixed mice exploring virtual reality. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.11.26.625482. [PMID: 39651122 PMCID: PMC11623582 DOI: 10.1101/2024.11.26.625482] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2024]
Abstract
Contextual fear conditioning is a classical laboratory task that tests associative memory formation and recall. Techniques such as multi-photon microscopy and holographic stimulation offer tremendous opportunities to understand the neural underpinnings of these memories. However, these techniques generally require animals to be head-fixed. There are few paradigms that test contextual fear conditioning in head-fixed mice, and none where the behavioral outcome following fear conditioning is freezing, the most common measure of fear in freely moving animals. To address this gap, we developed a contextual fear conditioning paradigm in head-fixed mice using virtual reality (VR) environments. We designed an apparatus to deliver tail shocks (unconditioned stimulus, US) while mice navigated a VR environment (conditioned stimulus, CS). The acquisition of contextual fear was tested when the mice were reintroduced to the shock-paired VR environment the following day. We tested three different variations of this paradigm and, in all of them, observed an increased conditioned fear response characterized by increased freezing behavior. This was especially prominent during the first trial in the shock-paired VR environment, compared to a neutral environment where the mice received no shocks. Our results demonstrate that head-fixed mice can be fear conditioned in VR, discriminate between a feared and neutral VR context, and display freezing as a conditioned response, similar to freely behaving animals. Furthermore, using a two-photon microscope, we imaged from large populations of hippocampal CA1 neurons before, during, and following contextual fear conditioning. Our findings reconfirmed those from the literature on freely moving animals, showing that CA1 place cells undergo remapping and show narrower place fields following fear conditioning. Our approach offers new opportunities to study the neural mechanisms underlying the formation, recall, and extinction of contextual fear memories. As the head-fixed preparation is compatible with multi-photon microscopy and holographic stimulation, it enables long-term tracking and manipulation of cells throughout distinct memory stages and provides subcellular resolution for investigating axonal, dendritic, and synaptic dynamics in real-time.
Collapse
|
17
|
Hille M, Kühn S, Kempermann G, Bonhoeffer T, Lindenberger U. From animal models to human individuality: Integrative approaches to the study of brain plasticity. Neuron 2024; 112:3522-3541. [PMID: 39461332 DOI: 10.1016/j.neuron.2024.10.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 06/02/2024] [Accepted: 10/04/2024] [Indexed: 10/29/2024]
Abstract
Plasticity allows organisms to form lasting adaptive changes in neural structures in response to interactions with the environment. It serves both species-general functions and individualized skill acquisition. To better understand human plasticity, we need to strengthen the dialogue between human research and animal models. Therefore, we propose to (1) enhance the interpretability of macroscopic methods used in human research by complementing molecular and fine-structural measures used in animals with such macroscopic methods, preferably applied to the same animals, to create macroscopic metrics common to both examined species; (2) launch dedicated cross-species research programs, using either well-controlled experimental paradigms, such as motor skill acquisition, or more naturalistic environments, where individuals of either species are observed in their habitats; and (3) develop conceptual and computational models linking molecular and fine-structural events to phenomena accessible by macroscopic methods. In concert, these three component strategies can foster new insights into the nature of plastic change.
Collapse
Affiliation(s)
- Maike Hille
- Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany; Center for Environmental Neuroscience, Max Planck Institute for Human Development, Berlin, Germany.
| | - Simone Kühn
- Center for Environmental Neuroscience, Max Planck Institute for Human Development, Berlin, Germany; Clinic and Policlinic for Psychiatry and Psychotherapy, University Clinic Hamburg-Eppendorf, Hamburg, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany, and London, UK
| | - Gerd Kempermann
- German Center for Neurodegenerative Diseases (DZNE), Dresden, Germany; CRTD - Center for Regenerative Therapies Dresden, TU Dresden, Dresden, Germany
| | - Tobias Bonhoeffer
- Synapses-Circuits-Plasticity, Max Planck Institute for Biological Intelligence, Martinsried, Germany
| | - Ulman Lindenberger
- Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany, and London, UK.
| |
Collapse
|
18
|
Somasundaram S, D F, Genasan K, Kamarul T, Raghavendran HRB. Implications of Biomaterials and Adipose-Derived Stem Cells in the Management of Calvarial Bone Defects. REGENERATIVE ENGINEERING AND TRANSLATIONAL MEDICINE 2024. [DOI: 10.1007/s40883-024-00358-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 08/25/2024] [Accepted: 09/13/2024] [Indexed: 01/03/2025]
|
19
|
Lin S, Gillis WF, Weinreb C, Zeine A, Jones SC, Robinson EM, Markowitz J, Datta SR. Characterizing the structure of mouse behavior using Motion Sequencing. Nat Protoc 2024; 19:3242-3291. [PMID: 38926589 PMCID: PMC11552546 DOI: 10.1038/s41596-024-01015-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 04/12/2024] [Indexed: 06/28/2024]
Abstract
Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.
Collapse
Affiliation(s)
- Sherry Lin
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Caleb Weinreb
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Ayman Zeine
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Samuel C Jones
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Emma M Robinson
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Jeffrey Markowitz
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | | |
Collapse
|
20
|
Eberle O, Büttner J, el-Hajj H, Montavon G, Müller KR, Valleriani M. Historical insights at scale: A corpus-wide machine learning analysis of early modern astronomic tables. SCIENCE ADVANCES 2024; 10:eadj1719. [PMID: 39441928 PMCID: PMC11498222 DOI: 10.1126/sciadv.adj1719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 09/19/2024] [Indexed: 10/25/2024]
Abstract
Understanding the evolution and dissemination of human knowledge over time faces challenges due to the abundance of historical materials and limited specialist resources. However, the digitization of historical archives presents an opportunity for AI-supported analysis. This study advances historical analysis by using an atomization-recomposition method that relies on unsupervised machine learning and explainable AI techniques. Focusing on the "Sacrobosco Collection," consisting of 359 early modern printed editions of astronomy textbooks from European universities (1472-1650), totaling 76,000 pages, our analysis uncovers temporal and geographic patterns in knowledge transformation. We highlight the relevant role of astronomy textbooks in shaping a unified mathematical culture, driven by competition among educational institutions and market dynamics. This approach deepens our understanding by grounding insights in historical context, integrating with traditional methodologies. Case studies illustrate how communities embraced scientific advancements, reshaping astronomic and geographical views and exploring scientific roots amidst a changing world.
Collapse
Affiliation(s)
- Oliver Eberle
- Machine Learning Group, Technische Universität Berlin, Marchstr. 23, 10587 Berlin, Germany
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
| | - Jochen Büttner
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Max Planck Institute of Geoanthropology, Kahlaische Str. 10, 07745 Jena, Germany
| | - Hassan el-Hajj
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Max Planck Institute for the History of Science,Boltzmannstr. 22, 14195 Berlin, Germany
| | - Grégoire Montavon
- Machine Learning Group, Technische Universität Berlin, Marchstr. 23, 10587 Berlin, Germany
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Department of Mathematics and Computer Science, Freie Universität Berlin, Arnimallee 14, 14195 Berlin, Germany
| | - Klaus-Robert Müller
- Machine Learning Group, Technische Universität Berlin, Marchstr. 23, 10587 Berlin, Germany
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Department of Artificial Intelligence, Korea University, Seoul 136-713, South Korea
- Max Planck Institute for Informatics, Stuhlsatzenhausweg 4, 66123 Saarbrücken, Germany
| | - Matteo Valleriani
- BIFOLD–Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Max Planck Institute for the History of Science,Boltzmannstr. 22, 14195 Berlin, Germany
- Institute of History and Philosophy of Science, Technology, and Literature, Faculty I–Humanities and Educational Sciences, Technische Universität Berlin, Straße des 17. Juni 135, 10623 Berlin, Germany
- The Cohn Institute for the History and Philosophy of Science and Ideas, Faculty of Humanities, Tel Aviv University, P.O. Box 39040, Ramat Aviv, Tel Aviv 6139001, Israel
| |
Collapse
|
21
|
Leopold DA. The big mixup: Neural representation during natural modes of primate visual behavior. Curr Opin Neurobiol 2024; 88:102913. [PMID: 39214044 PMCID: PMC11392606 DOI: 10.1016/j.conb.2024.102913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2024] [Revised: 08/06/2024] [Accepted: 08/07/2024] [Indexed: 09/04/2024]
Abstract
The primate brain has evolved specialized visual capacities to navigate complex physical and social environments. Researchers studying cortical circuits underlying these capacities have traditionally favored the use of simplified tasks and brief stimulus presentations in order to isolate cognitive variables with tight experimental control. As a result, operational theories about visual brain function have come to emphasize feature detection, hierarchical stimulus encoding, top-down task modulation, and functional segregation in distinct cortical areas. Recently, however, experimental paradigms combining natural behavior with electrophysiological recordings have begun to offer a distinctly different portrait of how the brain takes in and analyzes its visual surroundings. The present article reviews recent work in this area, highlighting some of the more surprising findings in domains of social vision and spatial navigation along with shifts in thinking that have begun to emanate from this approach.
Collapse
Affiliation(s)
- David A Leopold
- Section on Cognitive Neurophysiology and Imaging, Systems Neurodevelopment Laboratory, National Institute of Mental Health, Bethesda, MD 20892, USA; National Institute of Neurological Disorders and Stroke, Bethesda, MD 20892, USA.
| |
Collapse
|
22
|
Gedela NSS, Salim S, Radawiec RD, Richie J, Chestek C, Draelos A, Pelled G. Single unit electrophysiology recordings and computational modeling can predict octopus arm movement. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.09.13.612676. [PMID: 39345497 PMCID: PMC11430158 DOI: 10.1101/2024.09.13.612676] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 10/01/2024]
Abstract
The octopus simplified nervous system holds the potential to reveal principles of motor circuits and improve brain-machine interface devices through computational modeling with machine learning and statistical analysis. Here, an array of carbon electrodes providing single-unit electrophysiology recordings were implanted into the octopus anterior nerve cord. The number of spikes and arm movements in response to stimulation at different locations along the arm were recorded. We observed that the number of spikes occurring within the first 100ms after stimulation were predictive of the resultant movement response. Computational models showed that temporal electrophysiological features could be used to predict whether an arm movement occurred with 88.64% confidence, and if it was a lateral arm movement or a grasping motion with 75.45% confidence. Both supervised and unsupervised methods were applied to gain streaming measurements of octopus arm movements and how their motor circuitry produces rich movement types in real time. Deep learning models and unsupervised dimension reduction identified a consistent set of features that could be used to distinguish different types of arm movements. These models generated predictions for how to evoke a particular, complex movement in an orchestrated sequence for an individual motor circuit.
Collapse
Affiliation(s)
- Nitish Satya Sai Gedela
- Department of Mechanical Engineering, Michigan State University, East Lansing, MI, United States
| | - Sachin Salim
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Ryan D Radawiec
- Department of Mechanical Engineering, Michigan State University, East Lansing, MI, United States
| | - Julianna Richie
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Cynthia Chestek
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Anne Draelos
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
- Department of Computational Medicine & Bioinformatics, University of Michigan, Ann Arbor, MI, United States
| | - Galit Pelled
- Department of Mechanical Engineering, Michigan State University, East Lansing, MI, United States
- Department of Radiology, Michigan State University, East Lansing, MI, United States
| |
Collapse
|
23
|
Mimura K, Matsumoto J, Mochihashi D, Nakamura T, Nishijo H, Higuchi M, Hirabayashi T, Minamimoto T. Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs. Commun Biol 2024; 7:1080. [PMID: 39227400 PMCID: PMC11371840 DOI: 10.1038/s42003-024-06786-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Accepted: 08/27/2024] [Indexed: 09/05/2024] Open
Abstract
Nonhuman primates (NHPs) exhibit complex and diverse behavior that typifies advanced cognitive function and social communication, but quantitative and systematical measure of this natural nonverbal processing has been a technical challenge. Specifically, a method is required to automatically segment time series of behavior into elemental motion motifs, much like finding meaningful words in character strings. Here, we propose a solution called SyntacticMotionParser (SMP), a general-purpose unsupervised behavior parsing algorithm using a nonparametric Bayesian model. Using three-dimensional posture-tracking data from NHPs, SMP automatically outputs an optimized sequence of latent motion motifs classified into the most likely number of states. When applied to behavioral datasets from common marmosets and rhesus monkeys, SMP outperformed conventional posture-clustering models and detected a set of behavioral ethograms from publicly available data. SMP also quantified and visualized the behavioral effects of chemogenetic neural manipulations. SMP thus has the potential to dramatically improve our understanding of natural NHP behavior in a variety of contexts.
Collapse
Affiliation(s)
- Koki Mimura
- Advanced Neuroimaging Center, National Institutes for Quantum Science and Technology, Chiba, 263-8555, Japan.
- Research Center for Medical and Health Data Science, The Institute of Statistical Mathematics, Tokyo, 190-0014, Japan.
| | - Jumpei Matsumoto
- Department of System Emotional Science, Faculty of Medicine, University of Toyama, Toyama, 930-8555, Japan
- Research Center for Idling Brain Science, University of Toyama, Toyama, 930-8555, Japan
| | - Daichi Mochihashi
- Department of Statistical Inference and Mathematics, The Institute of Statistical Mathematics, Tokyo, 190-9562, Japan
| | - Tomoaki Nakamura
- Department of Mechanical Engineering and Intelligent Systems, The University of Electro-Communications, Tokyo, 182-8585, Japan
| | - Hisao Nishijo
- Department of System Emotional Science, Faculty of Medicine, University of Toyama, Toyama, 930-8555, Japan
- Research Center for Idling Brain Science, University of Toyama, Toyama, 930-8555, Japan
| | - Makoto Higuchi
- Advanced Neuroimaging Center, National Institutes for Quantum Science and Technology, Chiba, 263-8555, Japan
| | - Toshiyuki Hirabayashi
- Advanced Neuroimaging Center, National Institutes for Quantum Science and Technology, Chiba, 263-8555, Japan
| | - Takafumi Minamimoto
- Advanced Neuroimaging Center, National Institutes for Quantum Science and Technology, Chiba, 263-8555, Japan.
| |
Collapse
|
24
|
Schilling A, Gerum R, Boehm C, Rasheed J, Metzner C, Maier A, Reindl C, Hamer H, Krauss P. Deep learning based decoding of single local field potential events. Neuroimage 2024; 297:120696. [PMID: 38909761 DOI: 10.1016/j.neuroimage.2024.120696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 06/12/2024] [Accepted: 06/18/2024] [Indexed: 06/25/2024] Open
Abstract
How is information processed in the cerebral cortex? In most cases, recorded brain activity is averaged over many (stimulus) repetitions, which erases the fine-structure of the neural signal. However, the brain is obviously a single-trial processor. Thus, we here demonstrate that an unsupervised machine learning approach can be used to extract meaningful information from electro-physiological recordings on a single-trial basis. We use an auto-encoder network to reduce the dimensions of single local field potential (LFP) events to create interpretable clusters of different neural activity patterns. Strikingly, certain LFP shapes correspond to latency differences in different recording channels. Hence, LFP shapes can be used to determine the direction of information flux in the cerebral cortex. Furthermore, after clustering, we decoded the cluster centroids to reverse-engineer the underlying prototypical LFP event shapes. To evaluate our approach, we applied it to both extra-cellular neural recordings in rodents, and intra-cranial EEG recordings in humans. Finally, we find that single channel LFP event shapes during spontaneous activity sample from the realm of possible stimulus evoked event shapes. A finding which so far has only been demonstrated for multi-channel population coding.
Collapse
Affiliation(s)
- Achim Schilling
- Neuroscience Lab, University Hospital Erlangen, Germany; Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany
| | - Richard Gerum
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany; Department of Physics and Center for Vision Research, York University, Toronto, Canada
| | - Claudia Boehm
- Neuroscience Lab, University Hospital Erlangen, Germany; Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany
| | - Jwan Rasheed
- Neuroscience Lab, University Hospital Erlangen, Germany; Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany
| | - Claus Metzner
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany; Pattern Recognition Lab, University Erlangen-Nürnberg, Germany
| | - Andreas Maier
- Pattern Recognition Lab, University Erlangen-Nürnberg, Germany
| | - Caroline Reindl
- Epilepsy Center, Department of Neurology, University Hospital Erlangen, Germany
| | - Hajo Hamer
- Epilepsy Center, Department of Neurology, University Hospital Erlangen, Germany
| | - Patrick Krauss
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Germany; Pattern Recognition Lab, University Erlangen-Nürnberg, Germany.
| |
Collapse
|
25
|
Costa AC, Ahamed T, Jordan D, Stephens GJ. A Markovian dynamics for Caenorhabditis elegans behavior across scales. Proc Natl Acad Sci U S A 2024; 121:e2318805121. [PMID: 39083417 PMCID: PMC11317559 DOI: 10.1073/pnas.2318805121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 07/01/2024] [Indexed: 08/02/2024] Open
Abstract
How do we capture the breadth of behavior in animal movement, from rapid body twitches to aging? Using high-resolution videos of the nematode worm Caenorhabditis elegans, we show that a single dynamics connects posture-scale fluctuations with trajectory diffusion and longer-lived behavioral states. We take short posture sequences as an instantaneous behavioral measure, fixing the sequence length for maximal prediction. Within the space of posture sequences, we construct a fine-scale, maximum entropy partition so that transitions among microstates define a high-fidelity Markov model, which we also use as a means of principled coarse-graining. We translate these dynamics into movement using resistive force theory, capturing the statistical properties of foraging trajectories. Predictive across scales, we leverage the longest-lived eigenvectors of the inferred Markov chain to perform a top-down subdivision of the worm's foraging behavior, revealing both "runs-and-pirouettes" as well as previously uncharacterized finer-scale behaviors. We use our model to investigate the relevance of these fine-scale behaviors for foraging success, recovering a trade-off between local and global search strategies.
Collapse
Affiliation(s)
- Antonio C. Costa
- Department of Physics and Astronomy, Vrije Universiteit Amsterdam, Amsterdam1081HV, The Netherlands
| | | | - David Jordan
- Department of Biochemistry, University of Cambridge, CambridgeCB2 1GA, United Kingdom
| | - Greg J. Stephens
- Department of Physics and Astronomy, Vrije Universiteit Amsterdam, Amsterdam1081HV, The Netherlands
- Biological Physics Theory Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa904-0495, Japan
| |
Collapse
|
26
|
Borzelli D, De Marchis C, Quercia A, De Pasquale P, Casile A, Quartarone A, Calabrò RS, d’Avella A. Muscle Synergy Analysis as a Tool for Assessing the Effectiveness of Gait Rehabilitation Therapies: A Methodological Review and Perspective. Bioengineering (Basel) 2024; 11:793. [PMID: 39199751 PMCID: PMC11351442 DOI: 10.3390/bioengineering11080793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2024] [Revised: 07/19/2024] [Accepted: 07/29/2024] [Indexed: 09/01/2024] Open
Abstract
According to the modular hypothesis for the control of movement, muscles are recruited in synergies, which capture muscle coordination in space, time, or both. In the last two decades, muscle synergy analysis has become a well-established framework in the motor control field and for the characterization of motor impairments in neurological patients. Altered modular control during a locomotion task has been often proposed as a potential quantitative metric for characterizing pathological conditions. Therefore, the purpose of this systematic review is to analyze the recent literature that used a muscle synergy analysis of neurological patients' locomotion as an indicator of motor rehabilitation therapy effectiveness, encompassing the key methodological elements to date. Searches for the relevant literature were made in Web of Science, PubMed, and Scopus. Most of the 15 full-text articles which were retrieved and included in this review identified an effect of the rehabilitation intervention on muscle synergies. However, the used experimental and methodological approaches varied across studies. Despite the scarcity of studies that investigated the effect of rehabilitation on muscle synergies, this review supports the utility of muscle synergies as a marker of the effectiveness of rehabilitative therapy and highlights the challenges and open issues that future works need to address to introduce the muscle synergies in the clinical practice and decisional process.
Collapse
Affiliation(s)
- Daniele Borzelli
- Department of Biomedical and Dental Sciences and Morphofunctional Imaging, University of Messina, 98125 Messina, Italy; (A.Q.); (A.C.)
- Laboratory of Neuromotor Physiology, IRCCS Fondazione Santa Lucia, 00179 Rome, Italy;
| | | | - Angelica Quercia
- Department of Biomedical and Dental Sciences and Morphofunctional Imaging, University of Messina, 98125 Messina, Italy; (A.Q.); (A.C.)
| | - Paolo De Pasquale
- IRCCS Centro Neurolesi “Bonino Pulejo”, 98124 Messina, Italy; (P.D.P.); (A.Q.); (R.S.C.)
| | - Antonino Casile
- Department of Biomedical and Dental Sciences and Morphofunctional Imaging, University of Messina, 98125 Messina, Italy; (A.Q.); (A.C.)
| | - Angelo Quartarone
- IRCCS Centro Neurolesi “Bonino Pulejo”, 98124 Messina, Italy; (P.D.P.); (A.Q.); (R.S.C.)
| | | | - Andrea d’Avella
- Laboratory of Neuromotor Physiology, IRCCS Fondazione Santa Lucia, 00179 Rome, Italy;
- Department of Biology, University of Rome Tor Vergata, 00133 Rome, Italy
| |
Collapse
|
27
|
Laurent F, Blanc A, May L, Gándara L, Cocanougher BT, Jones BMW, Hague P, Barré C, Vestergaard CL, Crocker J, Zlatic M, Jovanic T, Masson JB. LarvaTagger: manual and automatic tagging of Drosophila larval behaviour. Bioinformatics 2024; 40:btae441. [PMID: 38970365 PMCID: PMC11262801 DOI: 10.1093/bioinformatics/btae441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Revised: 06/03/2024] [Accepted: 07/03/2024] [Indexed: 07/08/2024] Open
Abstract
MOTIVATION As more behavioural assays are carried out in large-scale experiments on Drosophila larvae, the definitions of the archetypal actions of a larva are regularly refined. In addition, video recording and tracking technologies constantly evolve. Consequently, automatic tagging tools for Drosophila larval behaviour must be retrained to learn new representations from new data. However, existing tools cannot transfer knowledge from large amounts of previously accumulated data. We introduce LarvaTagger, a piece of software that combines a pre-trained deep neural network, providing a continuous latent representation of larva actions for stereotypical behaviour identification, with a graphical user interface to manually tag the behaviour and train new automatic taggers with the updated ground truth. RESULTS We reproduced results from an automatic tagger with high accuracy, and we demonstrated that pre-training on large databases accelerates the training of a new tagger, achieving similar prediction accuracy using less data. AVAILABILITY AND IMPLEMENTATION All the code is free and open source. Docker images are also available. See gitlab.pasteur.fr/nyx/LarvaTagger.jl.
Collapse
Affiliation(s)
- François Laurent
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- Épiméthée, INRIA, 75015 Paris, France
- Institut Pasteur, Université Paris Cité, Bioinformatics and Biostatistics Hub, F-75015 Paris, France
| | - Alexandre Blanc
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- Épiméthée, INRIA, 75015 Paris, France
| | - Lilly May
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- TUM School of Computation, Information and Technology, 80333 Munich, Germany
| | - Lautaro Gándara
- European Molecular Biology Laboratory, Developmental Biology, 69117 Heidelberg, Germany
| | - Benjamin T Cocanougher
- Department of Zoology, University of Cambridge, Cambridge CB2 3EJ, United Kingdom
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA 20147, United States
- MRC Laboratory of Molecular Biology, Cambridge CB2 0QH, United Kingdom
| | - Benjamin M W Jones
- Department of Zoology, University of Cambridge, Cambridge CB2 3EJ, United Kingdom
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA 20147, United States
- MRC Laboratory of Molecular Biology, Cambridge CB2 0QH, United Kingdom
| | - Peter Hague
- Department of Zoology, University of Cambridge, Cambridge CB2 3EJ, United Kingdom
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA 20147, United States
- MRC Laboratory of Molecular Biology, Cambridge CB2 0QH, United Kingdom
| | - Chloé Barré
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- Épiméthée, INRIA, 75015 Paris, France
| | - Christian L Vestergaard
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- Épiméthée, INRIA, 75015 Paris, France
| | - Justin Crocker
- European Molecular Biology Laboratory, Developmental Biology, 69117 Heidelberg, Germany
| | - Marta Zlatic
- Department of Zoology, University of Cambridge, Cambridge CB2 3EJ, United Kingdom
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA 20147, United States
- MRC Laboratory of Molecular Biology, Cambridge CB2 0QH, United Kingdom
| | - Tihana Jovanic
- Institut des Neurosciences Paris-Saclay, Université Paris-Saclay, Centre National de la Recherche Scientifique, UMR 9197, 91400 Saclay, France
| | - Jean-Baptiste Masson
- Institut Pasteur, Université Paris Cité, CNRS UMR 3571, Decision and Bayesian Computation, 75015 Paris, France
- Épiméthée, INRIA, 75015 Paris, France
| |
Collapse
|
28
|
Ye S, Filippova A, Lauer J, Schneider S, Vidal M, Qiu T, Mathis A, Mathis MW. SuperAnimal pretrained pose estimation models for behavioral analysis. Nat Commun 2024; 15:5165. [PMID: 38906853 PMCID: PMC11192880 DOI: 10.1038/s41467-024-48792-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Accepted: 04/26/2024] [Indexed: 06/23/2024] Open
Abstract
Quantification of behavior is critical in diverse applications from neuroscience, veterinary medicine to animal conservation. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present SuperAnimal, a method to develop unified foundation models that can be used on over 45 species, without additional manual labels. These models show excellent performance across six pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100× more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification and kinematic analysis. Collectively, we present a data-efficient solution for animal pose estimation.
Collapse
Affiliation(s)
- Shaokai Ye
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Anastasiia Filippova
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Jessy Lauer
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Steffen Schneider
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Maxime Vidal
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Tian Qiu
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Alexander Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland
| | - Mackenzie Weygandt Mathis
- École Polytechnique Fédérale de Lausanne (EPFL), Brain Mind Institute & Neuro-X Institute, Geneva, Switzerland.
| |
Collapse
|
29
|
Ouyang W, Kilner KJ, Xavier RMP, Liu Y, Lu Y, Feller SM, Pitts KM, Wu M, Ausra J, Jones I, Wu Y, Luan H, Trueb J, Higbee-Dempsey EM, Stepien I, Ghoreishi-Haack N, Haney CR, Li H, Kozorovitskiy Y, Heshmati M, Banks AR, Golden SA, Good CH, Rogers JA. An implantable device for wireless monitoring of diverse physio-behavioral characteristics in freely behaving small animals and interacting groups. Neuron 2024; 112:1764-1777.e5. [PMID: 38537641 PMCID: PMC11256974 DOI: 10.1016/j.neuron.2024.02.020] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 02/08/2024] [Accepted: 02/28/2024] [Indexed: 06/09/2024]
Abstract
Comprehensive, continuous quantitative monitoring of intricately orchestrated physiological processes and behavioral states in living organisms can yield essential data for elucidating the function of neural circuits under healthy and diseased conditions, for defining the effects of potential drugs and treatments, and for tracking disease progression and recovery. Here, we report a wireless, battery-free implantable device and a set of associated algorithms that enable continuous, multiparametric physio-behavioral monitoring in freely behaving small animals and interacting groups. Through advanced analytics approaches applied to mechano-acoustic signals of diverse body processes, the device yields heart rate, respiratory rate, physical activity, temperature, and behavioral states. Demonstrations in pharmacological, locomotor, and acute and social stress tests and in optogenetic studies offer unique insights into the coordination of physio-behavioral characteristics associated with healthy and perturbed states. This technology has broad utility in neuroscience, physiology, behavior, and other areas that rely on studies of freely moving, small animal models.
Collapse
Affiliation(s)
- Wei Ouyang
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA; Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, USA
| | - Keith J Kilner
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA; NeuroLux Inc., Northfield, IL 60093, USA
| | | | - Yiming Liu
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | - Yinsheng Lu
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | | | - Kayla M Pitts
- Department of Biological Structure, University of Washington, Seattle, WA 98195, USA
| | - Mingzheng Wu
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | | | - Ian Jones
- NeuroLux Inc., Northfield, IL 60093, USA
| | - Yunyun Wu
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | - Haiwen Luan
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | - Jacob Trueb
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA
| | | | - Iwona Stepien
- Developmental Therapeutics Core, Northwestern University, Evanston, IL 60208, USA
| | | | - Chad R Haney
- Center for Advanced Molecular Imaging, Northwestern University, Evanston, IL 60208, USA
| | - Hao Li
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Evanston, IL 60208, USA; Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Evanston, IL 60208, USA
| | - Yevgenia Kozorovitskiy
- Department of Neurobiology, Northwestern University, Evanston, IL 60208, USA; Chemistry of Life Processes Institute, Northwestern University, Evanston, IL 60208, USA
| | - Mitra Heshmati
- Department of Biological Structure, University of Washington, Seattle, WA 98195, USA; Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), University of Washington, Seattle, WA 98195, USA; Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA 98195, USA
| | - Anthony R Banks
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA; NeuroLux Inc., Northfield, IL 60093, USA
| | - Sam A Golden
- Department of Biological Structure, University of Washington, Seattle, WA 98195, USA; Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), University of Washington, Seattle, WA 98195, USA.
| | - Cameron H Good
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA; NeuroLux Inc., Northfield, IL 60093, USA.
| | - John A Rogers
- Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA; Department of Materials Science and Engineering, Northwestern University, Evanston, IL 60208, USA; Department of Mechanical Engineering, Northwestern University, Evanston, IL 60208, USA; Department of Biomedical Engineering, Northwestern University, Evanston, IL 60208, USA; Department of Chemistry, Northwestern University, Evanston, IL 60208, USA; Department of Neurological Surgery, Feinberg School of Medicine, Northwestern University, Evanston, IL 60208, USA.
| |
Collapse
|
30
|
Brown RE. Measuring the replicability of our own research. J Neurosci Methods 2024; 406:110111. [PMID: 38521128 DOI: 10.1016/j.jneumeth.2024.110111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Revised: 03/08/2024] [Accepted: 03/18/2024] [Indexed: 03/25/2024]
Abstract
In the study of transgenic mouse models of neurodevelopmental and neurodegenerative disorders, we use batteries of tests to measure deficits in behaviour and from the results of these tests, we make inferences about the mental states of the mice that we interpret as deficits in "learning", "memory", "anxiety", "depression", etc. This paper discusses the problems of determining whether a particular transgenic mouse is a valid mouse model of disease X, the problem of background strains, and the question of whether our behavioural tests are measuring what we say they are. The problem of the reliability of results is then discussed: are they replicable between labs and can we replicate our results in our own lab? This involves the study of intra- and inter- experimenter reliability. The variables that influence replicability and the importance of conducting a complete behavioural phenotype: sensory, motor, cognitive and social emotional behaviour are discussed. Then the thorny question of failure to replicate is examined: Is it a curse or a blessing? Finally, the role of failure in research and what it tells us about our research paradigms is examined.
Collapse
Affiliation(s)
- Richard E Brown
- Department of Psychology and Neuroscience, Dalhousie University, Halifax, NS B3H 4R2, Canada.
| |
Collapse
|
31
|
Li H, Deng Z, Yu X, Lin J, Xie Y, Liao W, Ma Y, Zheng Q. Combining dual-view fusion pose estimation and multi-type motion feature extraction to assess arthritis pain in mice. Biomed Signal Process Control 2024; 92:106080. [DOI: 10.1016/j.bspc.2024.106080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/07/2024]
|
32
|
Monosov IE, Zimmermann J, Frank MJ, Mathis MW, Baker JT. Ethological computational psychiatry: Challenges and opportunities. Curr Opin Neurobiol 2024; 86:102881. [PMID: 38696972 PMCID: PMC11162904 DOI: 10.1016/j.conb.2024.102881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 04/02/2024] [Accepted: 04/03/2024] [Indexed: 05/04/2024]
Abstract
Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.
Collapse
Affiliation(s)
- Ilya E. Monosov
- Departments of Neuroscience, Biomedical Engineering, Electrical Engineering, and Neurosurgery, Washington University School of Medicine, St. Louis, MO, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Michael J. Frank
- Carney Center for Computational Brain Science, Brown University, Providence, RI, USA
| | | | | |
Collapse
|
33
|
Broomer MC, Beacher NJ, Wang MW, Lin DT. Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning. ADDICTION NEUROSCIENCE 2024; 11:100154. [PMID: 38680653 PMCID: PMC11044849 DOI: 10.1016/j.addicn.2024.100154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/01/2024]
Abstract
In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.
Collapse
Affiliation(s)
- Matthew C. Broomer
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Nicholas J. Beacher
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Michael W. Wang
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Da-Ting Lin
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
- The Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| |
Collapse
|
34
|
Sabnis G, Hession L, Mahoney JM, Mobley A, Santos M, Kumar V. Visual detection of seizures in mice using supervised machine learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.29.596520. [PMID: 38868170 PMCID: PMC11167691 DOI: 10.1101/2024.05.29.596520] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/14/2024]
Abstract
Seizures are caused by abnormally synchronous brain activity that can result in changes in muscle tone, such as twitching, stiffness, limpness, or rhythmic jerking. These behavioral manifestations are clear on visual inspection and the most widely used seizure scoring systems in preclinical models, such as the Racine scale in rodents, use these behavioral patterns in semiquantitative seizure intensity scores. However, visual inspection is time-consuming, low-throughput, and partially subjective, and there is a need for rigorously quantitative approaches that are scalable. In this study, we used supervised machine learning approaches to develop automated classifiers to predict seizure severity directly from noninvasive video data. Using the PTZ-induced seizure model in mice, we trained video-only classifiers to predict ictal events, combined these events to predict an univariate seizure intensity for a recording session, as well as time-varying seizure intensity scores. Our results show, for the first time, that seizure events and overall intensity can be rigorously quantified directly from overhead video of mice in a standard open field using supervised approaches. These results enable high-throughput, noninvasive, and standardized seizure scoring for downstream applications such as neurogenetics and therapeutic discovery.
Collapse
Affiliation(s)
| | | | | | | | | | - Vivek Kumar
- The Jackson Laboratory, Bar Harbor, ME USA
- School of Graduate Biomedical Sciences, Tufts University School of Medicine, Boston, MA USA
- Graduate School of Biomedical Sciences and Engineering, University of Maine, Orono, ME USA
| |
Collapse
|
35
|
Lv S, Wang J, Chen X, Liao X. STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation. iScience 2024; 27:109772. [PMID: 38711440 PMCID: PMC11070338 DOI: 10.1016/j.isci.2024.109772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Revised: 03/15/2024] [Accepted: 04/15/2024] [Indexed: 05/08/2024] Open
Abstract
Animal behavior analysis plays a crucial role in contemporary neuroscience research. However, the performance of the frame-by-frame approach may degrade in scenarios with occlusions or motion blur. In this study, we propose a spatiotemporal network model based on YOLOv8 to enhance the accuracy of key-point detection in mouse behavioral experimental videos. This model integrates a time-domain tracking strategy comprising two components: the first part utilizes key-point detection results from the previous frame to detect potential target locations in the subsequent frame; the second part employs Kalman filtering to analyze key-point changes prior to detection, allowing for the estimation of missing key-points. In the comparison of pose estimation results between our approach, YOLOv8, DeepLabCut and SLEAP on videos of three mouse behavioral experiments, our approach demonstrated significantly superior performance. This suggests that our method offers a new and effective means of accurately tracking and estimating pose in mice through spatiotemporal processing.
Collapse
Affiliation(s)
- Songyan Lv
- Guangxi Key Laboratory of Special Biomedicine & Advanced Institute for Brain and Intelligence, School of Medicine, Guangxi University, Nanning 530004, China
| | - Jincheng Wang
- Guangxi Key Laboratory of Special Biomedicine & Advanced Institute for Brain and Intelligence, School of Medicine, Guangxi University, Nanning 530004, China
| | - Xiaowei Chen
- Guangxi Key Laboratory of Special Biomedicine & Advanced Institute for Brain and Intelligence, School of Medicine, Guangxi University, Nanning 530004, China
| | - Xiang Liao
- Center for Neurointelligence, School of Medicine, Chongqing University, Chongqing 400030, China
| |
Collapse
|
36
|
Luo W, Zhang G, Shao Q, Zhao Y, Wang D, Zhang X, Liu K, Li X, Liu J, Wang P, Li L, Wang G, Wang F, Yu Z. An efficient visual servo tracker for herd monitoring by UAV. Sci Rep 2024; 14:10463. [PMID: 38714785 PMCID: PMC11582714 DOI: 10.1038/s41598-024-60445-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Accepted: 04/23/2024] [Indexed: 05/10/2024] Open
Abstract
It is a challenging and meaningful task to carry out UAV-based livestock monitoring in high-altitude (more than 4500 m on average) and cold regions (annual average - 4 °C) on the Qinghai Tibet Plateau. The purpose of artificial intelligence (AI) is to execute automated tasks and to solve practical problems in actual applications by combining the software technology with the hardware carrier to create integrated advanced devices. Only in this way, the maximum value of AI could be realized. In this paper, a real-time tracking system with dynamic target tracking ability is proposed. It is developed based on the tracking-by-detection architecture using YOLOv7 and Deep SORT algorithms for target detection and tracking, respectively. In response to the problems encountered in the tracking process of complex and dense scenes, our work (1) Uses optical flow to compensate the Kalman filter, to solve the problem of mismatch between the target bounding box predicted by the Kalman filter (KF) and the input when the target detection in the current frame is complex, thereby improving the prediction accuracy; (2) Using a low confidence trajectory filtering method to reduce false positive trajectories generated by Deep SORT, thereby mitigating the impact of unreliable detection on target tracking. (3) A visual servo controller has been designed for the Unmanned Aerial Vehicle (UAV) to reduce the impact of rapid movement on tracking and ensure that the target is always within the field of view of the UAV camera, thereby achieving automatic tracking tasks. Finally, the system was tested using Tibetan yaks on the Qinghai Tibet Plateau as tracking targets, and the results showed that the system has real-time multi tracking ability and ideal visual servo effect in complex and dense scenes.
Collapse
Affiliation(s)
- Wei Luo
- North China Institute of Aerospace Engineering, Langfang, 065000, China
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, 100101, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang, 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang, 065000, China
| | - Guoqing Zhang
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Quanqin Shao
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, 100101, China
- University of Chinese Academy of Sciences, Beijing, 101407, China
| | - Yongxiang Zhao
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Dongliang Wang
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, 100101, China.
| | - Xiongyi Zhang
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, 100101, China
| | - Ke Liu
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Xiaoliang Li
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Jiandong Liu
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Penggang Wang
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Lin Li
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Guanwu Wang
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Fulong Wang
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| | - Zhongde Yu
- North China Institute of Aerospace Engineering, Langfang, 065000, China
| |
Collapse
|
37
|
Loukola OJ, Antinoja A, Mäkelä K, Arppi J, Peng F, Solvi C. Evidence for socially influenced and potentially actively coordinated cooperation by bumblebees. Proc Biol Sci 2024; 291:20240055. [PMID: 38689557 PMCID: PMC11061644 DOI: 10.1098/rspb.2024.0055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 02/26/2024] [Accepted: 03/25/2024] [Indexed: 05/02/2024] Open
Abstract
Cooperation is common in animals, yet the specific mechanisms driving collaborative behaviour in different species remain unclear. We investigated the proximate mechanisms underlying the cooperative behaviour of bumblebees in two different tasks, where bees had to simultaneously push a block in an arena or a door at the end of a tunnel for access to reward. In both tasks, when their partner's entry into the arena/tunnel was delayed, bees took longer to first push the block/door compared with control bees that learned to push alone. In the tunnel task, just before gaining access to reward, bees were more likely to face towards their partner than expected by chance or compared with controls. These results show that bumblebees' cooperative behaviour is not simply a by-product of individual efforts but is socially influenced. We discuss how bees' turning behaviours, e.g. turning around before first reaching the door when their partner was delayed and turning back towards the door in response to seeing their partner heading towards the door, suggest the potential for active coordination. However, because these behaviours could also be interpreted as combined responses to social and secondary reinforcement cues, future studies are needed to help clarify whether bumblebees truly use active coordination.
Collapse
Affiliation(s)
- Olli J. Loukola
- Ecology and Genetics Research Unit, University of Oulu, Oulu, 90014, Finland
| | - Anna Antinoja
- Ecology and Genetics Research Unit, University of Oulu, Oulu, 90014, Finland
- Biology Centre of the Czech Academy of Sciences, Institute of Entomology, Faculty of Science, University of South Bohemia, Branisovska 31, 37005, Czech Republic
| | - Kaarle Mäkelä
- Ecology and Genetics Research Unit, University of Oulu, Oulu, 90014, Finland
| | - Janette Arppi
- Ecology and Genetics Research Unit, University of Oulu, Oulu, 90014, Finland
| | - Fei Peng
- Department of Psychology, School of Public Health, Southern Medical University, Guangzhou, 510515, People's Republic of China
- Guangdong-Hong Kong-Macao Greater Bay Area Center for Brain Science and Brain-Inspired Intelligence, Southern Medical University, Guangzhou, 510515, People's Republic of China
| | - Cwyn Solvi
- Guangdong-Hong Kong-Macao Greater Bay Area Center for Brain Science and Brain-Inspired Intelligence, Southern Medical University, Guangzhou, 510515, People's Republic of China
| |
Collapse
|
38
|
Biderman D, Whiteway MR, Hurwitz C, Greenspan N, Lee RS, Vishnubhotla A, Warren R, Pedraja F, Noone D, Schartner M, Huntenburg JM, Khanal A, Meijer GT, Noel JP, Pan-Vazquez A, Socha KZ, Urai AE, Cunningham JP, Sawtell NB, Paninski L. Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.04.28.538703. [PMID: 37162966 PMCID: PMC10168383 DOI: 10.1101/2023.04.28.538703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce "Lightning Pose," an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry, and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post-hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We release a cloud application that allows users to label data, train networks, and predict new videos directly from the browser.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | | | - Anup Khanal
- University of California Los Angeles, Los Angeles, USA
| | | | | | | | | | | | | | | | | |
Collapse
|
39
|
Tillmann JF, Hsu AI, Schwarz MK, Yttri EA. A-SOiD, an active-learning platform for expert-guided, data-efficient discovery of behavior. Nat Methods 2024; 21:703-711. [PMID: 38383746 DOI: 10.1038/s41592-024-02200-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 01/29/2024] [Indexed: 02/23/2024]
Abstract
To identify and extract naturalistic behavior, two methods have become popular: supervised and unsupervised. Each approach carries its own strengths and weaknesses (for example, user bias, training cost, complexity and action discovery), which the user must consider in their decision. Here, an active-learning platform, A-SOiD, blends these strengths, and in doing so, overcomes several of their inherent drawbacks. A-SOiD iteratively learns user-defined groups with a fraction of the usual training data, while attaining expansive classification through directed unsupervised classification. In socially interacting mice, A-SOiD outperformed standard methods despite requiring 85% less training data. Additionally, it isolated ethologically distinct mouse interactions via unsupervised classification. We observed similar performance and efficiency using nonhuman primate and human three-dimensional pose data. In both cases, the transparency in A-SOiD's cluster definitions revealed the defining features of the supervised classification through a game-theoretic approach. To facilitate use, A-SOiD comes as an intuitive, open-source interface for efficient segmentation of user-defined behaviors and discovered sub-actions.
Collapse
Affiliation(s)
- Jens F Tillmann
- Institute of Experimental Epileptology and Cognition Research, University of Bonn, Bonn, Germany
| | - Alexander I Hsu
- Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Martin K Schwarz
- Institute of Experimental Epileptology and Cognition Research, University of Bonn, Bonn, Germany.
| | - Eric A Yttri
- Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, PA, USA.
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
| |
Collapse
|
40
|
Gore SV, Del Rosario Hernández T, Creton R. Behavioral effects of visual stimuli in adult zebrafish using a novel eight-tank imaging system. Front Behav Neurosci 2024; 18:1320126. [PMID: 38529416 PMCID: PMC10962262 DOI: 10.3389/fnbeh.2024.1320126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 02/12/2024] [Indexed: 03/27/2024] Open
Abstract
Introduction Animals respond to various environmental cues. Animal behavior is complex, and behavior analysis can greatly help to understand brain function. Most of the available behavioral imaging setups are expensive, provide limited options for customization, and allow for behavioral imaging of one animal at a time. Methods The current study takes advantage of adult zebrafish as a model organism to study behavior in a novel behavioral setup allowing one to concurrently image 8 adult zebrafish. Results Our results indicate that adult zebrafish show a unique behavioral profile in response to visual stimuli such as moving lines. In the presence of moving lines, the fish spent more time exploring the tank and spent more time toward the edges of the tanks. In addition, the fish moved and oriented themselves against the direction of the moving lines, indicating a negative optomotor response (OMR). With repeated exposure to moving lines, we observed a reduced optomotor response in adult zebrafish. Discussion Our behavioral setup is relatively inexpensive, provides flexibility in the presentation of various animated visual stimuli, and offers improved throughput for analyzing behavior in adult zebrafish. This behavioral setup shows promising potential to quantify various behavioral measures and opens new avenues to understand complex behaviors.
Collapse
Affiliation(s)
- Sayali V. Gore
- Department of Molecular Biology, Cell Biology, and Biochemistry, Brown University, Providence, RI, United States
| | | | | |
Collapse
|
41
|
Wheatley T, Thornton MA, Stolk A, Chang LJ. The Emerging Science of Interacting Minds. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024; 19:355-373. [PMID: 38096443 PMCID: PMC10932833 DOI: 10.1177/17456916231200177] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2024]
Abstract
For over a century, psychology has focused on uncovering mental processes of a single individual. However, humans rarely navigate the world in isolation. The most important determinants of successful development, mental health, and our individual traits and preferences arise from interacting with other individuals. Social interaction underpins who we are, how we think, and how we behave. Here we discuss the key methodological challenges that have limited progress in establishing a robust science of how minds interact and the new tools that are beginning to overcome these challenges. A deep understanding of the human mind requires studying the context within which it originates and exists: social interaction.
Collapse
Affiliation(s)
- Thalia Wheatley
- Consortium for Interacting Minds, Psychological and Brain Sciences, Dartmouth, Hanover, NH USA
- Santa Fe Institute
| | - Mark A. Thornton
- Consortium for Interacting Minds, Psychological and Brain Sciences, Dartmouth, Hanover, NH USA
| | - Arjen Stolk
- Consortium for Interacting Minds, Psychological and Brain Sciences, Dartmouth, Hanover, NH USA
| | - Luke J. Chang
- Consortium for Interacting Minds, Psychological and Brain Sciences, Dartmouth, Hanover, NH USA
| |
Collapse
|
42
|
Salem G, Cope N, Garmendia M, Pu A, Somenhalli A, Krynitsky J, Cubert N, Jones T, Dold G, Fletcher A, Kravitz A, Pohida T, Dennis J. MouseVUER: video based open-source system for laboratory mouse home-cage monitoring. Sci Rep 2024; 14:2662. [PMID: 38302573 PMCID: PMC10834510 DOI: 10.1038/s41598-024-52788-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 01/23/2024] [Indexed: 02/03/2024] Open
Abstract
Video monitoring of mice in the home-cage reveals behavior profiles without the disruptions caused by specialized test setups and makes it possible to quantify changes in behavior patterns continually over long time frames. Several commercial home-cage monitoring systems are available with varying costs and capabilities; however there are currently no open-source systems for home-cage monitoring. We present an open-source system for top-down video monitoring of research mice in a slightly modified home-cage. The system is designed for integration with Allentown NexGen ventilated racks and allows unobstructed view of up to three mice, but can also be operated outside the rack. The system has an easy to duplicate and assemble home-cage design along with a video acquisition solution. The system utilizes a depth video camera, and we demonstrate the robustness of depth video for home-cage mice monitoring. For researchers without access to Allentown NexGen ventilated racks, we provide designs and assembly instructions for a standalone non-ventilated rack solution that holds three systems for more compact and efficient housing. We make all the design files, along with detailed assembly and installation instructions, available on the project webpage ( https://github.com/NIH-CIT-OIR-SPIS/MouseVUER ).
Collapse
Affiliation(s)
- Ghadi Salem
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA.
| | - Niall Cope
- Oak Ridge Institute for Science and Education (ORISE), US Department of Energy, Oak Ridge, TN, USA
| | - Marcial Garmendia
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - Alex Pu
- Oak Ridge Institute for Science and Education (ORISE), US Department of Energy, Oak Ridge, TN, USA
| | - Abhishek Somenhalli
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - Jonathan Krynitsky
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - Noah Cubert
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - Thomas Jones
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - George Dold
- Section On Instrumentation, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, USA
| | - Anthony Fletcher
- Scientific Information Office, National Heart Lung and Blood Institute, National Institutes of Health, Bethesda, MD, USA
| | - Alexxai Kravitz
- Dept of Psychiatry, Washington University in St Louis, St Louis, MO, USA
| | - Thomas Pohida
- Instrumentation Development and Engineering Application Solutions, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD, USA
| | - John Dennis
- Division of Veterinary Services, Center for Biologics Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| |
Collapse
|
43
|
Beaulieu M. Capturing wild animal welfare: a physiological perspective. Biol Rev Camb Philos Soc 2024; 99:1-22. [PMID: 37635128 DOI: 10.1111/brv.13009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 08/07/2023] [Accepted: 08/07/2023] [Indexed: 08/29/2023]
Abstract
Affective states, such as emotions, are presumably widespread across the animal kingdom because of the adaptive advantages they are supposed to confer. However, the study of the affective states of animals has thus far been largely restricted to enhancing the welfare of animals managed by humans in non-natural contexts. Given the diversity of wild animals and the variable conditions they can experience, extending studies on animal affective states to the natural conditions that most animals experience will allow us to broaden and deepen our general understanding of animal welfare. Yet, this same diversity makes examining animal welfare in the wild highly challenging. There is therefore a need for unifying theoretical frameworks and methodological approaches that can guide researchers keen to engage in this promising research area. The aim of this article is to help advance this important research area by highlighting the central relationship between physiology and animal welfare and rectify its apparent oversight, as revealed by the current scientific literature on wild animals. Moreover, this article emphasises the advantages of including physiological markers to assess animal welfare in the wild (e.g. objectivity, comparability, condition range, temporality), as well as their concomitant limitations (e.g. only access to peripheral physiological markers with complex relationships with affective states). Best-practice recommendations (e.g. replication and multifactorial approaches) are also provided to allow physiological markers to be used most effectively and appropriately when assessing the welfare of animals in their natural habitat. This review seeks to provide the foundation for a new and distinct research area with a vast theoretical and applied potential: wild animal welfare physiology.
Collapse
Affiliation(s)
- Michaël Beaulieu
- Wild Animal Initiative, 5123 W 98th St, 1204, Minneapolis, MN, 55437, USA
| |
Collapse
|
44
|
Bialek W, Shaevitz JW. Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior. PHYSICAL REVIEW LETTERS 2024; 132:048401. [PMID: 38335334 DOI: 10.1103/physrevlett.132.048401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Accepted: 11/27/2023] [Indexed: 02/12/2024]
Abstract
The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.
Collapse
Affiliation(s)
- William Bialek
- Joseph Henry Laboratories of Physics and Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, New Jersey 08544, USA
- Center for Studies in Physics and Biology, Rockefeller University, New York, New York 10065, USA
| | - Joshua W Shaevitz
- Joseph Henry Laboratories of Physics and Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, New Jersey 08544, USA
| |
Collapse
|
45
|
Bidgood R, Zubelzu M, Ruiz-Ortega JA, Morera-Herreras T. Automated procedure to detect subtle motor alterations in the balance beam test in a mouse model of early Parkinson's disease. Sci Rep 2024; 14:862. [PMID: 38195974 PMCID: PMC10776624 DOI: 10.1038/s41598-024-51225-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 01/02/2024] [Indexed: 01/11/2024] Open
Abstract
Parkinson's disease (PD) is the most common motor neurodegenerative disorder, characterised by aggregated α-synuclein (α-syn) constituting Lewy bodies. We aimed to investigate temporal changes in motor impairments in a PD mouse model induced by overexpression of α-syn with the conventional manual analysis of the balance beam test and a novel approach using machine learning algorithms to automate behavioural analysis. We combined automated animal tracking using markerless pose estimation in DeepLabCut, with automated behavioural classification in Simple Behavior Analysis. Our automated procedure was able to detect subtle motor deficits in mouse performances in the balance beam test that the manual analysis approach could not assess. The automated model revealed time-course significant differences for the "walking" behaviour in the mean interval between each behavioural bout, the median event bout duration and the classifier probability of occurrence in male PD mice, even though no statistically significant loss of tyrosine hydroxylase in the nigrostriatal system was found in either sex. These findings are valuable for early detection of motor impairment in early PD animal models. We provide a user-friendly, step-by-step guide for automated assessment of mouse performances in the balance beam test, which aims to be replicable without any significant computational and programming knowledge.
Collapse
Affiliation(s)
- Raphaëlle Bidgood
- Department of Pharmacology, Faculty of Medicine and Nursing, University of the Basque Country (UPV/EHU), Barrio Sarriena S/N, 48940, Leioa, Biscay, Spain
| | - Maider Zubelzu
- Department of Pharmacology, Faculty of Medicine and Nursing, University of the Basque Country (UPV/EHU), Barrio Sarriena S/N, 48940, Leioa, Biscay, Spain
- Autonomic and Movement Disorders Unit, Neurodegenerative Diseases, Biobizkaia, Barakaldo, Biscay, Spain
| | - Jose Angel Ruiz-Ortega
- Department of Pharmacology, Faculty of Medicine and Nursing, University of the Basque Country (UPV/EHU), Barrio Sarriena S/N, 48940, Leioa, Biscay, Spain
- Autonomic and Movement Disorders Unit, Neurodegenerative Diseases, Biobizkaia, Barakaldo, Biscay, Spain
| | - Teresa Morera-Herreras
- Department of Pharmacology, Faculty of Medicine and Nursing, University of the Basque Country (UPV/EHU), Barrio Sarriena S/N, 48940, Leioa, Biscay, Spain.
- Autonomic and Movement Disorders Unit, Neurodegenerative Diseases, Biobizkaia, Barakaldo, Biscay, Spain.
| |
Collapse
|
46
|
Manduca G, Zeni V, Moccia S, Milano BA, Canale A, Benelli G, Stefanini C, Romano D. Learning algorithms estimate pose and detect motor anomalies in flies exposed to minimal doses of a toxicant. iScience 2023; 26:108349. [PMID: 38058310 PMCID: PMC10696104 DOI: 10.1016/j.isci.2023.108349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 10/04/2023] [Accepted: 10/24/2023] [Indexed: 12/08/2023] Open
Abstract
Pesticide exposure, even at low doses, can have detrimental effects on ecosystems. This study aimed at validating the use of machine learning for recognizing motor anomalies, produced by minimal insecticide exposure on a model insect species. The Mediterranean fruit fly, Ceratitis capitata (Diptera: Tephritidae), was exposed to food contaminated with low concentrations of Carlina acaulis essential oil (EO). A deep learning approach enabled fly pose estimation on video recordings in a custom-built arena. Five machine learning algorithms were trained on handcrafted features, extracted from the predicted pose, to distinguish treated individuals. Random Forest and K-Nearest Neighbor algorithms best performed, with an area under the receiver operating characteristic (ROC) curve of 0.75 and 0.73, respectively. Both algorithms achieved an accuracy of 0.71. Results show the machine learning potential for detecting sublethal effects arising from insecticide exposure on fly motor behavior, which could also affect other organisms and environmental health.
Collapse
Affiliation(s)
- Gianluca Manduca
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, Pisa, Italy
- Department of Excellence in Robotics and AI, Sant'Anna School of Advanced Studies, Piazza Martiri della Libertà 33, 56127, Pisa, Italy
| | - Valeria Zeni
- Department of Agriculture, Food and Environment, University of Pisa, Via del Borghetto 80, 56124, Pisa, Italy
| | - Sara Moccia
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, Pisa, Italy
- Department of Excellence in Robotics and AI, Sant'Anna School of Advanced Studies, Piazza Martiri della Libertà 33, 56127, Pisa, Italy
| | - Beatrice A. Milano
- Institute of Life Sciences, Sant'Anna School of Advanced Studies, Piazza Martiri della Libertà 33, 56127, Pisa, Italy
- Faculty of Medicine and Surgery, University of Pisa, Via Roma 55/Building 57, 56126, Pisa, Italy
| | - Angelo Canale
- Department of Agriculture, Food and Environment, University of Pisa, Via del Borghetto 80, 56124, Pisa, Italy
| | - Giovanni Benelli
- Department of Agriculture, Food and Environment, University of Pisa, Via del Borghetto 80, 56124, Pisa, Italy
| | - Cesare Stefanini
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, Pisa, Italy
- Department of Excellence in Robotics and AI, Sant'Anna School of Advanced Studies, Piazza Martiri della Libertà 33, 56127, Pisa, Italy
| | - Donato Romano
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, Pisa, Italy
- Department of Excellence in Robotics and AI, Sant'Anna School of Advanced Studies, Piazza Martiri della Libertà 33, 56127, Pisa, Italy
| |
Collapse
|
47
|
Desai N, Bala P, Richardson R, Raper J, Zimmermann J, Hayden B. OpenApePose, a database of annotated ape photographs for pose estimation. eLife 2023; 12:RP86873. [PMID: 38078902 PMCID: PMC10712952 DOI: 10.7554/elife.86873] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2023] Open
Abstract
Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present OpenApePose, a new public dataset of 71,868 photographs, annotated with 16 body landmarks of six ape species in naturalistic contexts. We show that a standard deep net (HRNet-W48) trained on ape photos can reliably track out-of-sample ape photos better than networks trained on monkeys (specifically, the OpenMonkeyPose dataset) and on humans (COCO) can. This trained network can track apes almost as well as the other networks can track their respective taxa, and models trained without one of the six ape species can track the held-out species better than the monkey and human models can. Ultimately, the results of our analyses highlight the importance of large, specialized databases for animal tracking systems and confirm the utility of our new ape database.
Collapse
Affiliation(s)
- Nisarg Desai
- Department of Neuroscience and Center for Magnetic Resonance Research, University of MinnesotaMinneapolisUnited States
| | - Praneet Bala
- Department of Computer Science, University of MinnesotaMinneapolisUnited States
| | - Rebecca Richardson
- Emory National Primate Research Center, Emory UniversityAtlantaUnited States
| | - Jessica Raper
- Emory National Primate Research Center, Emory UniversityAtlantaUnited States
| | - Jan Zimmermann
- Department of Neuroscience and Center for Magnetic Resonance Research, University of MinnesotaMinneapolisUnited States
| | - Benjamin Hayden
- Department of Neuroscience and Center for Magnetic Resonance Research, University of MinnesotaMinneapolisUnited States
| |
Collapse
|
48
|
Fernandez ME, Martinez-Romero J, Aon MA, Bernier M, Price NL, de Cabo R. How is Big Data reshaping preclinical aging research? Lab Anim (NY) 2023; 52:289-314. [PMID: 38017182 DOI: 10.1038/s41684-023-01286-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 10/10/2023] [Indexed: 11/30/2023]
Abstract
The exponential scientific and technological progress during the past 30 years has favored the comprehensive characterization of aging processes with their multivariate nature, leading to the advent of Big Data in preclinical aging research. Spanning from molecular omics to organism-level deep phenotyping, Big Data demands large computational resources for storage and analysis, as well as new analytical tools and conceptual frameworks to gain novel insights leading to discovery. Systems biology has emerged as a paradigm that utilizes Big Data to gain insightful information enabling a better understanding of living organisms, visualized as multilayered networks of interacting molecules, cells, tissues and organs at different spatiotemporal scales. In this framework, where aging, health and disease represent emergent states from an evolving dynamic complex system, context given by, for example, strain, sex and feeding times, becomes paramount for defining the biological trajectory of an organism. Using bioinformatics and artificial intelligence, the systems biology approach is leading to remarkable advances in our understanding of the underlying mechanism of aging biology and assisting in creative experimental study designs in animal models. Future in-depth knowledge acquisition will depend on the ability to fully integrate information from different spatiotemporal scales in organisms, which will probably require the adoption of theories and methods from the field of complex systems. Here we review state-of-the-art approaches in preclinical research, with a focus on rodent models, that are leading to conceptual and/or technical advances in leveraging Big Data to understand basic aging biology and its full translational potential.
Collapse
Affiliation(s)
- Maria Emilia Fernandez
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Jorge Martinez-Romero
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
- Laboratory of Epidemiology and Population Science, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Miguel A Aon
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
- Laboratory of Cardiovascular Science, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Michel Bernier
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Nathan L Price
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Rafael de Cabo
- Experimental Gerontology Section, Translational Gerontology Branch, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA.
| |
Collapse
|
49
|
Sałat K, Zaręba P, Awtoniuk M, Sałat R. Naturally Inspired Molecules for Neuropathic Pain Inhibition-Effect of Mirogabalin and Cebranopadol on Mechanical and Thermal Nociceptive Threshold in Mice. Molecules 2023; 28:7862. [PMID: 38067591 PMCID: PMC10708129 DOI: 10.3390/molecules28237862] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2023] [Revised: 11/26/2023] [Accepted: 11/29/2023] [Indexed: 12/18/2023] Open
Abstract
BACKGROUND Neuropathic pain is drug-resistant to available analgesics and therefore novel treatment options for this debilitating clinical condition are urgently needed. Recently, two drug candidates, namely mirogabalin and cebranopadol have become a subject of interest because of their potential utility as analgesics for chronic pain treatment. However, they have not been investigated thoroughly in some types of neuropathic pain, both in humans and experimental animals. METHODS This study used the von Frey test, the hot plate test and the two-plate thermal place preference test supported by image analysis and machine learning to assess the effect of intraperitoneal mirogabalin and subcutaneous cebranopadol on mechanical and thermal nociceptive threshold in mouse models of neuropathic pain induced by streptozotocin, paclitaxel and oxaliplatin. RESULTS Mirogabalin and cebranopadol effectively attenuated tactile allodynia in models of neuropathic pain induced by streptozotocin and paclitaxel. Cebranopadol was more effective than mirogabalin in this respect. Both drugs also elevated the heat nociceptive threshold in mice. In the oxaliplatin model, cebranopadol and mirogabalin reduced cold-exacerbated pain. CONCLUSIONS Since mirogabalin and cebranopadol are effective in animal models of neuropathic pain, they seem to be promising novel therapies for various types of neuropathic pain in patients, in particular those who are resistant to available analgesics.
Collapse
Affiliation(s)
- Kinga Sałat
- Department of Pharmacodynamics, Faculty of Pharmacy, Jagiellonian University, 9 Medyczna St., 30-688 Krakow, Poland
| | - Paula Zaręba
- Chair of Pharmaceutical Chemistry, Faculty of Pharmacy, Jagiellonian University, 9 Medyczna St., 30-688 Krakow, Poland;
| | - Michał Awtoniuk
- Institute of Mechanical Engineering, Warsaw University of Life Sciences, 166 Nowoursynowska St., 02-787 Warsaw, Poland;
| | - Robert Sałat
- Faculty of Electrical and Computer Engineering, Cracow University of Technology, 24 Warszawska St., 31-155 Krakow, Poland;
| |
Collapse
|
50
|
Sadedin S, Duéñez-Guzmán EA, Leibo JZ. Emotions and courtship help bonded pairs cooperate, but emotional agents are vulnerable to deceit. Proc Natl Acad Sci U S A 2023; 120:e2308911120. [PMID: 37948585 PMCID: PMC10655579 DOI: 10.1073/pnas.2308911120] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 09/21/2023] [Indexed: 11/12/2023] Open
Abstract
Coordinated pair bonds are common in birds and also occur in many other taxa. How do animals solve the social dilemmas they face in coordinating with a partner? We developed an evolutionary model to explore this question, based on observations that a) neuroendocrine feedback provides emotional bookkeeping which is thought to play a key role in vertebrate social bonds and b) these bonds are developed and maintained via courtship interactions that include low-stakes social dilemmas. Using agent-based simulation, we found that emotional bookkeeping and courtship sustained cooperation in the iterated prisoner's dilemma in noisy environments, especially when combined. However, when deceitful defection was possible at low cost, courtship often increased cooperation, whereas emotional bookkeeping decreased it.
Collapse
Affiliation(s)
- Suzanne Sadedin
- Independent Researcher, Abbots LangleyWD5 0QS, United Kingdom
| | | | | |
Collapse
|