1
|
Pellegrino A, Stein H, Cayco-Gajic NA. Dimensionality reduction beyond neural subspaces with slice tensor component analysis. Nat Neurosci 2024; 27:1199-1210. [PMID: 38710876 DOI: 10.1038/s41593-024-01626-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 03/20/2024] [Indexed: 05/08/2024]
Abstract
Recent work has argued that large-scale neural recordings are often well described by patterns of coactivation across neurons. Yet the view that neural variability is constrained to a fixed, low-dimensional subspace may overlook higher-dimensional structure, including stereotyped neural sequences or slowly evolving latent spaces. Here we argue that task-relevant variability in neural data can also cofluctuate over trials or time, defining distinct 'covariability classes' that may co-occur within the same dataset. To demix these covariability classes, we develop sliceTCA (slice tensor component analysis), a new unsupervised dimensionality reduction method for neural data tensors. In three example datasets, including motor cortical activity during a classic reaching task in primates and recent multiregion recordings in mice, we show that sliceTCA can capture more task-relevant structure in neural data using fewer components than traditional methods. Overall, our theoretical framework extends the classic view of low-dimensional population activity by incorporating additional classes of latent variables capturing higher-dimensional structure.
Collapse
Affiliation(s)
- Arthur Pellegrino
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Département D'Etudes Cognitives, Ecole Normale Supérieure, PSL University, Paris, France.
- Institute for Adaptive and Neural Computation, School of Informatics, University of Edinburgh, Edinburgh, UK.
| | - Heike Stein
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Département D'Etudes Cognitives, Ecole Normale Supérieure, PSL University, Paris, France
| | - N Alex Cayco-Gajic
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Département D'Etudes Cognitives, Ecole Normale Supérieure, PSL University, Paris, France.
| |
Collapse
|
2
|
Sihn D, Chae S, Kim SP. A method to find temporal structure of neuronal coactivity patterns with across-trial correlations. J Neurosci Methods 2024; 408:110172. [PMID: 38782124 DOI: 10.1016/j.jneumeth.2024.110172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 05/08/2024] [Accepted: 05/17/2024] [Indexed: 05/25/2024]
Abstract
BACKGROUND The across-trial correlation of neurons' coactivity patterns emerges to be important for information coding, but methods for finding their temporal structures remain largely unexplored. NEW METHOD In the present study, we propose a method to find time clusters in which coactivity patterns of neurons are correlated across trials. We transform the multidimensional neural activity at each timing into a coactivity pattern of binary states, and predict the coactivity patterns at different timings. We devise a method suitable for these coactivity pattern predictions, call general event prediction. Cross-temporal prediction accuracy is then used to estimate across-trial correlations between coactivity patterns at two timings. We extract time clusters from the cross-temporal prediction accuracy by a modified k-means algorithm. RESULTS The feasibility of the proposed method is verified through simulations based on ground truth. We apply the proposed method to a calcium imaging dataset recorded from the motor cortex of mice, and demonstrate time clusters of motor cortical coactivity patterns during a motor task. COMPARISON WITH EXISTING METHODS While the existing cosine similarity method, which does not account for across-trial correlation, shows temporal structures only for contralateral neural responses, the proposed method reveals those for both contralateral and ipsilateral neural responses, demonstrating the effect of across-trial correlations. CONCLUSIONS This study introduces a novel method for measuring the temporal structure of neuronal ensemble activity.
Collapse
Affiliation(s)
- Duho Sihn
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan 44919, the Republic of Korea
| | - Soyoung Chae
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan 44919, the Republic of Korea
| | - Sung-Phil Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan 44919, the Republic of Korea.
| |
Collapse
|
3
|
Wei 魏赣超 G, Tajik Mansouri زینب تاجیک منصوری Z, Wang 王晓婧 X, Stevenson IH. Calibrating Bayesian Decoders of Neural Spiking Activity. J Neurosci 2024; 44:e2158232024. [PMID: 38538143 PMCID: PMC11063820 DOI: 10.1523/jneurosci.2158-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 01/29/2024] [Accepted: 03/11/2024] [Indexed: 05/03/2024] Open
Abstract
Accurately decoding external variables from observations of neural activity is a major challenge in systems neuroscience. Bayesian decoders, which provide probabilistic estimates, are some of the most widely used. Here we show how, in many common settings, the probabilistic predictions made by traditional Bayesian decoders are overconfident. That is, the estimates for the decoded stimulus or movement variables are more certain than they should be. We then show how Bayesian decoding with latent variables, taking account of low-dimensional shared variability in the observations, can improve calibration, although additional correction for overconfidence is still needed. Using data from males, we examine (1) decoding the direction of grating stimuli from spike recordings in the primary visual cortex in monkeys, (2) decoding movement direction from recordings in the primary motor cortex in monkeys, (3) decoding natural images from multiregion recordings in mice, and (4) decoding position from hippocampal recordings in rats. For each setting, we characterize the overconfidence, and we describe a possible method to correct miscalibration post hoc. Properly calibrated Bayesian decoders may alter theoretical results on probabilistic population coding and lead to brain-machine interfaces that more accurately reflect confidence levels when identifying external variables.
Collapse
Affiliation(s)
- Ganchao Wei 魏赣超
- Department of Statistical Science, Duke University, Durham, North Carolina 27708
| | | | | | - Ian H Stevenson
- Departments of Biomedical Engineering, University of Connecticut, Storrs, Connecticut 06269
- Psychological Sciences, University of Connecticut, Storrs, Connecticut 06269
- Connecticut Institute for Brain and Cognitive Science, University of Connecticut, Storrs, Connecticut 06269
| |
Collapse
|
4
|
Lundqvist M, Miller EK, Nordmark J, Liljefors J, Herman P. Beta: bursts of cognition. Trends Cogn Sci 2024:S1364-6613(24)00077-9. [PMID: 38658218 DOI: 10.1016/j.tics.2024.03.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 03/11/2024] [Accepted: 03/20/2024] [Indexed: 04/26/2024]
Abstract
Beta oscillations are linked to the control of goal-directed processing of sensory information and the timing of motor output. Recent evidence demonstrates they are not sustained but organized into intermittent high-power bursts mediating timely functional inhibition. This implies there is a considerable moment-to-moment variation in the neural dynamics supporting cognition. Beta bursts thus offer new opportunities for studying how sensory inputs are selectively processed, reshaped by inhibitory cognitive operations and ultimately result in motor actions. Recent method advances reveal diversity in beta bursts that provide deeper insights into their function and the underlying neural circuit activity motifs. We propose that brain-wide, spatiotemporal patterns of beta bursting reflect various cognitive operations and that their dynamics reveal nonlinear aspects of cortical processing.
Collapse
Affiliation(s)
- Mikael Lundqvist
- Division of Psychology, Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden; The Picower Institute for Learning & Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Earl K Miller
- The Picower Institute for Learning & Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Jonatan Nordmark
- Division of Psychology, Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden
| | - Johan Liljefors
- Division of Psychology, Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden
| | - Pawel Herman
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden; Digital Futures, KTH Royal Institute of Technology, Stockholm, Sweden
| |
Collapse
|
5
|
Orellana V. D, Donoghue JP, Vargas-Irwin CE. Low frequency independent components: Internal neuromarkers linking cortical LFPs to behavior. iScience 2024; 27:108310. [PMID: 38303697 PMCID: PMC10831875 DOI: 10.1016/j.isci.2023.108310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 12/08/2022] [Accepted: 10/10/2023] [Indexed: 02/03/2024] Open
Abstract
Local field potentials (LFPs) in the primate motor cortex have been shown to reflect information related to volitional movements. However, LFPs are composite signals that receive contributions from multiple neural sources, producing a complex mix of component signals. Using a blind source separation approach, we examined the components of neural activity recorded using multielectrode arrays in motor areas of macaque monkeys during a grasping and lifting task. We found a set of independent components in the low-frequency LFP with high temporal and spatial consistency associated with each task stage. We observed that ICs often arise from electrodes distributed across multiple cortical areas and provide complementary information to external behavioral markers, specifically in task stage detection and trial alignment. Taken together, our results show that it is possible to separate useful independent components of the LFP associated with specific task-related events, potentially representing internal markers of transition between cortical network states.
Collapse
Affiliation(s)
- Diego Orellana V.
- Engineering Faculty, Pontificia Universidad Javeriana, Bogotá 110231, Colombia
- Faculty of Energy, Universidad Nacional de Loja, Loja 110101, Ecuador
| | - John P. Donoghue
- Department of Neuroscience, Brown University, Providence, RI 02912, USA
- Robert J and Nancy D Carney Institute for Brain Science, Providence, RI 02912, USA
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, Department of Veterans Affairs Medical Center, Providence, RI 02908, USA
| | - Carlos E. Vargas-Irwin
- Department of Neuroscience, Brown University, Providence, RI 02912, USA
- Robert J and Nancy D Carney Institute for Brain Science, Providence, RI 02912, USA
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, Department of Veterans Affairs Medical Center, Providence, RI 02908, USA
| |
Collapse
|
6
|
Pan H, Zhang Y, Li L, Qin X. A design and implementation of multi-character classification scheme based on motor imagery EEG signals. Neuroscience 2024; 538:22-29. [PMID: 38072171 DOI: 10.1016/j.neuroscience.2023.12.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 09/18/2023] [Accepted: 12/02/2023] [Indexed: 12/19/2023]
Abstract
In the field of brain-to-text communication, it is difficult to finish highly dexterous behaviors of writing multi-character by motor-imagery-based brain-computer interface (MI-BCI), setting a barrier to restore communication in people who have lost the ability to move and speak. In this paper, we design and implement a multi-character classification scheme based on 29 characters of motor imagery (MI) electroencephalogram (EEG) signals, which contains 26 English letters and 3 punctuation marks. Firstly, we design a novel experimental paradigm to increase the variety of BCI inputs by asking subjects to imagine the movement of writing 29 characters instead of gross motor skills such as reaching or grasping. Secondly, because of the high dimension of EEG signals, we adopt power spectral density (PSD), principal components analysis (PCA), kernel principal components analysis (KPCA) respectively to decompose EEG signals and extract feature, and then test the results with pearson product-moment correlation coefficient (PCCs). Thirdly, we respectively employ k-nearest neighbor (kNN), support vector machine (SVM), extreme learning machine (ELM) and light gradient boosting machine (LightGBM) to classify 29 characters and compare the results. We have implemented a complete scheme, including paradigm design, signal acquisition, feature extraction and classification, which can effectively classify 29 characters. The experimental results show that the KPCA has the best feature extraction effect and the kNN has the highest classification accuracy, with the final classification accuracy reaching 96.2%, which is better than other studies.
Collapse
Affiliation(s)
- Hongguang Pan
- College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China; Xi'an Key Laboratory of Electrical Equipment Condition Monitoring and Power Supply Security, Xi'an 710054, China.
| | - Yibo Zhang
- College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China; Xi'an Key Laboratory of Electrical Equipment Condition Monitoring and Power Supply Security, Xi'an 710054, China
| | - Li Li
- College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China; Xi'an Key Laboratory of Electrical Equipment Condition Monitoring and Power Supply Security, Xi'an 710054, China
| | - Xuebin Qin
- College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China; Xi'an Key Laboratory of Electrical Equipment Condition Monitoring and Power Supply Security, Xi'an 710054, China
| |
Collapse
|
7
|
Schmidt R, Rose J, Muralidharan V. Transient oscillations as computations for cognition: Analysis, modeling and function. Curr Opin Neurobiol 2023; 83:102796. [PMID: 37804772 DOI: 10.1016/j.conb.2023.102796] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 08/31/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Our view of neural oscillations is currently changing. The dominant picture of sustained oscillations is now often replaced by transient oscillations occurring in bursts. This phenomenon seems to be quite comprehensive, as it has been reported for different oscillation frequencies, including the theta, beta, and gamma bands, as well as cortical and subcortical regions in a variety of cognitive tasks and species. Here we review recent developments in their analysis, computational modeling, and functional roles. For the analysis of transient oscillations methods using lagged coherence and Hidden Markov Models have been developed and applied in recent studies to ascertain their transient nature and study their contribution to cognitive functions. Furthermore, computational models have been developed that account for their stochastic nature, which poses interesting functional constraints. Finally, as transient oscillations have been observed across many species, they are likely of functional significance and we consider challenges in characterizing their function.
Collapse
Affiliation(s)
- Robert Schmidt
- Institute for Neural Computation, Faculty of Computer Science, Ruhr-University Bochum, Germany.
| | - Jonas Rose
- Neural Basis of Learning, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Germany
| | - Vignesh Muralidharan
- Center for Brain Science and Application, School of AI and Data Science, Indian Institute of Technology Jodhpur, India. https://twitter.com/vigmdhrn
| |
Collapse
|
8
|
Chinen K, Kawabata A, Tanaka H, Komura Y. Inaccessible time to visual awareness during attentional blinks in macaques and humans. iScience 2023; 26:108208. [PMID: 38223787 PMCID: PMC10784117 DOI: 10.1016/j.isci.2023.108208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Revised: 08/26/2023] [Accepted: 10/11/2023] [Indexed: 01/16/2024] Open
Abstract
Even when we attend to successive visual events, we often cannot notice an event occurring during a certain temporal window. Such an inaccessible time for visual awareness is known as "attentional blink" (AB). Whether AB is a phenomenon unique to humans or exists also in other animals is unclear. Using a dual-task paradigm shared between macaques and humans, we here demonstrate a nonhuman primate model of AB. Although macaques also showed behavioral signatures of AB, their AB effect lasted longer than that of humans. To map the relation between macaque and human ABs, we introduced a time warping analysis. The analysis revealed a formal structure behind the interspecies difference of AB; the temporal window of macaque AB was scaled from that of human AB. The present study opens the door to combining the approaches of neuroscience, psychophysics, and theoretical models to further identify a scale-invariant biological substrate of visual awareness.
Collapse
Affiliation(s)
- Koji Chinen
- Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan
| | - Akira Kawabata
- Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan
| | - Hitoshi Tanaka
- Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan
| | - Yutaka Komura
- Graduate School of Human and Environmental Studies, Kyoto University, Yoshida-Nihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan
| |
Collapse
|
9
|
Crone N, Candrea D, Shah S, Luo S, Angrick M, Rabbani Q, Coogan C, Milsap G, Nathan K, Wester B, Anderson W, Rosenblatt K, Clawson L, Maragakis N, Vansteensel M, Tenore F, Ramsey N, Fifer M, Uchil A. A click-based electrocorticographic brain-computer interface enables long-term high-performance switch-scan spelling. RESEARCH SQUARE 2023:rs.3.rs-3158792. [PMID: 37841873 PMCID: PMC10571601 DOI: 10.21203/rs.3.rs-3158792/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/17/2023]
Abstract
Background Brain-computer interfaces (BCIs) can restore communication in movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command "click" decoders provide a basic yet highly functional capability. Methods We sought to test the performance and long-term stability of click-decoding using a chronically implanted high density electrocorticographic (ECoG) BCI with coverage of the sensorimotor cortex in a human clinical trial participant (ClinicalTrials.gov, NCT03567213) with amyotrophic lateral sclerosis (ALS). We trained the participant's click decoder using a small amount of training data (< 44 minutes across four days) collected up to 21 days prior to BCI use, and then tested it over a period of 90 days without any retraining or updating. Results Using this click decoder to navigate a switch-scanning spelling interface, the study participant was able to maintain a median spelling rate of 10.2 characters per min. Though a transient reduction in signal power modulation interrupted testing with this fixed model, a new click decoder achieved comparable performance despite being trained with even less data (< 15 min, within one day). Conclusion These results demonstrate that a click decoder can be trained with a small ECoG dataset while retaining robust performance for extended periods, providing functional text-based communication to BCI users.
Collapse
|
10
|
Sadras N, Sani OG, Ahmadipour P, Shanechi MM. Post-stimulus encoding of decision confidence in EEG: toward a brain-computer interface for decision making. J Neural Eng 2023; 20:056012. [PMID: 37524073 DOI: 10.1088/1741-2552/acec14] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 07/31/2023] [Indexed: 08/02/2023]
Abstract
Objective.When making decisions, humans can evaluate how likely they are to be correct. If this subjective confidence could be reliably decoded from brain activity, it would be possible to build a brain-computer interface (BCI) that improves decision performance by automatically providing more information to the user if needed based on their confidence. But this possibility depends on whether confidence can be decoded right after stimulus presentation and before the response so that a corrective action can be taken in time. Although prior work has shown that decision confidence is represented in brain signals, it is unclear if the representation is stimulus-locked or response-locked, and whether stimulus-locked pre-response decoding is sufficiently accurate for enabling such a BCI.Approach.We investigate the neural correlates of confidence by collecting high-density electroencephalography (EEG) during a perceptual decision task with realistic stimuli. Importantly, we design our task to include a post-stimulus gap that prevents the confounding of stimulus-locked activity by response-locked activity and vice versa, and then compare with a task without this gap.Main results.We perform event-related potential and source-localization analyses. Our analyses suggest that the neural correlates of confidence are stimulus-locked, and that an absence of a post-stimulus gap could cause these correlates to incorrectly appear as response-locked. By preventing response-locked activity from confounding stimulus-locked activity, we then show that confidence can be reliably decoded from single-trial stimulus-locked pre-response EEG alone. We also identify a high-performance classification algorithm by comparing a battery of algorithms. Lastly, we design a simulated BCI framework to show that the EEG classification is accurate enough to build a BCI and that the decoded confidence could be used to improve decision making performance particularly when the task difficulty and cost of errors are high.Significance.Our results show feasibility of non-invasive EEG-based BCIs to improve human decision making.
Collapse
Affiliation(s)
- Nitin Sadras
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Omid G Sani
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Parima Ahmadipour
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Department of Computer Science, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Neuroscience Graduate Program University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
11
|
Odean NN, Sanayei M, Shadlen MN. Transient Oscillations of Neural Firing Rate Associated With Routing of Evidence in a Perceptual Decision. J Neurosci 2023; 43:6369-6383. [PMID: 37550053 PMCID: PMC10500999 DOI: 10.1523/jneurosci.2200-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 07/07/2023] [Accepted: 07/12/2023] [Indexed: 08/09/2023] Open
Abstract
To form a perceptual decision, the brain must acquire samples of evidence from the environment and incorporate them in computations that mediate choice behavior. While much is known about the neural circuits that process sensory information and those that form decisions, less is known about the mechanisms that establish the functional linkage between them. We trained monkeys of both sexes to make difficult decisions about the net direction of visual motion under conditions that required trial-by-trial control of functional connectivity. In one condition, the motion appeared at different locations on different trials. In the other, two motion patches appeared, only one of which was informative. Neurons in the parietal cortex produced brief oscillations in their firing rate at the time routing was established: upon onset of the motion display when its location was unpredictable across trials, and upon onset of an attention cue that indicated in which of two locations an informative patch of dots would appear. The oscillation was absent when the stimulus location was fixed across trials. We interpret the oscillation as a manifestation of the mechanism that establishes the source and destination of flexibly routed information, but not the transmission of the information per se Significance Statement It has often been suggested that oscillations in neural activity might serve a role in routing information appropriately. We observe an oscillation in neural firing rate in the lateral intraparietal area consistent with such a role. The oscillations are transient. They coincide with the establishment of routing, but they do not appear to play a role in the transmission (or conveyance) of the routed information itself.
Collapse
Affiliation(s)
- Naomi N Odean
- Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, New York 10025
| | - Mehdi Sanayei
- Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, New York 10025
| | - Michael N Shadlen
- Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, New York 10025
- Howard Hughes Medical Institute, Columbia University, New York, New York 10025
- Kavli Institute, New York, New York 10025
| |
Collapse
|
12
|
Stine GM, Trautmann EM, Jeurissen D, Shadlen MN. A neural mechanism for terminating decisions. Neuron 2023; 111:2601-2613.e5. [PMID: 37352857 PMCID: PMC10565788 DOI: 10.1016/j.neuron.2023.05.028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 03/20/2023] [Accepted: 05/30/2023] [Indexed: 06/25/2023]
Abstract
The brain makes decisions by accumulating evidence until there is enough to stop and choose. Neural mechanisms of evidence accumulation are established in association cortex, but the site and mechanism of termination are unknown. Here, we show that the superior colliculus (SC) plays a causal role in terminating decisions, and we provide evidence for a mechanism by which this occurs. We recorded simultaneously from neurons in the lateral intraparietal area (LIP) and SC while monkeys made perceptual decisions. Despite similar trial-averaged activity, we found distinct single-trial dynamics in the two areas: LIP displayed drift-diffusion dynamics and SC displayed bursting dynamics. We hypothesized that the bursts manifest a threshold mechanism applied to signals represented in LIP to terminate the decision. Consistent with this hypothesis, SC inactivation produced behavioral effects diagnostic of an impaired threshold sensor and prolonged the buildup of activity in LIP. The results reveal the transformation from deliberation to commitment.
Collapse
Affiliation(s)
- Gabriel M Stine
- Department of Neuroscience, Columbia University, New York, NY 10027, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Eric M Trautmann
- Department of Neuroscience, Columbia University, New York, NY 10027, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY 10027, USA
| | - Danique Jeurissen
- Department of Neuroscience, Columbia University, New York, NY 10027, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Howard Hughes Medical Institute, Columbia University, New York, NY 10027, USA
| | - Michael N Shadlen
- Department of Neuroscience, Columbia University, New York, NY 10027, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Howard Hughes Medical Institute, Columbia University, New York, NY 10027, USA; Kavli Institute for Brain Science, Columbia University, New York, NY 10027, USA.
| |
Collapse
|
13
|
Sotomayor-Gómez B, Battaglia FP, Vinck M. SpikeShip: A method for fast, unsupervised discovery of high-dimensional neural spiking patterns. PLoS Comput Biol 2023; 19:e1011335. [PMID: 37523401 PMCID: PMC10414626 DOI: 10.1371/journal.pcbi.1011335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 08/10/2023] [Accepted: 07/07/2023] [Indexed: 08/02/2023] Open
Abstract
Neural coding and memory formation depend on temporal spiking sequences that span high-dimensional neural ensembles. The unsupervised discovery and characterization of these spiking sequences requires a suitable dissimilarity measure to spiking patterns, which can then be used for clustering and decoding. Here, we present a new dissimilarity measure based on optimal transport theory called SpikeShip, which compares multi-neuron spiking patterns based on all the relative spike-timing relationships among neurons. SpikeShip computes the optimal transport cost to make all the relative spike-timing relationships (across neurons) identical between two spiking patterns. We show that this transport cost can be decomposed into a temporal rigid translation term, which captures global latency shifts, and a vector of neuron-specific transport flows, which reflect inter-neuronal spike timing differences. SpikeShip can be effectively computed for high-dimensional neuronal ensembles, has a low (linear) computational cost that has the same order as the spike count, and is sensitive to higher-order correlations. Furthermore, SpikeShip is binless, can handle any form of spike time distributions, is not affected by firing rate fluctuations, can detect patterns with a low signal-to-noise ratio, and can be effectively combined with a sliding window approach. We compare the advantages and differences between SpikeShip and other measures like SPIKE and Victor-Purpura distance. We applied SpikeShip to large-scale Neuropixel recordings during spontaneous activity and visual encoding. We show that high-dimensional spiking sequences detected via SpikeShip reliably distinguish between different natural images and different behavioral states. These spiking sequences carried complementary information to conventional firing rate codes. SpikeShip opens new avenues for studying neural coding and memory consolidation by rapid and unsupervised detection of temporal spiking patterns in high-dimensional neural ensembles.
Collapse
Affiliation(s)
- Boris Sotomayor-Gómez
- Donders Centre for Neuroscience, Department of Neurophysics, Radboud University Nijmegen, Nijmegen, Netherlands
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, Frankfurt, Germany
| | - Francesco P. Battaglia
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| | - Martin Vinck
- Donders Centre for Neuroscience, Department of Neurophysics, Radboud University Nijmegen, Nijmegen, Netherlands
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, Frankfurt, Germany
| |
Collapse
|
14
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
15
|
Koh TH, Bishop WE, Kawashima T, Jeon BB, Srinivasan R, Mu Y, Wei Z, Kuhlman SJ, Ahrens MB, Chase SM, Yu BM. Dimensionality reduction of calcium-imaged neuronal population activity. NATURE COMPUTATIONAL SCIENCE 2023; 3:71-85. [PMID: 37476302 PMCID: PMC10358781 DOI: 10.1038/s43588-022-00390-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 12/05/2022] [Indexed: 07/22/2023]
Abstract
Calcium imaging has been widely adopted for its ability to record from large neuronal populations. To summarize the time course of neural activity, dimensionality reduction methods, which have been applied extensively to population spiking activity, may be particularly useful. However, it is unclear if the dimensionality reduction methods applied to spiking activity are appropriate for calcium imaging. We thus carried out a systematic study of design choices based on standard dimensionality reduction methods. We also developed a method to perform deconvolution and dimensionality reduction simultaneously (Calcium Imaging Linear Dynamical System, CILDS). CILDS most accurately recovered the single-trial, low-dimensional time courses from simulated calcium imaging data. CILDS also outperformed the other methods on calcium imaging recordings from larval zebrafish and mice. More broadly, this study represents a foundation for summarizing calcium imaging recordings of large neuronal populations using dimensionality reduction in diverse experimental settings.
Collapse
Affiliation(s)
- Tze Hui Koh
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Center for the Neural Basis of Cognition, PA
| | - William E. Bishop
- Center for the Neural Basis of Cognition, PA
- Department of Machine Learning, Carnegie Mellon University, PA
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Takashi Kawashima
- Janelia Research Campus, Howard Hughes Medical Institute, VA
- Department of Brain Sciences, Weizmann Institute of Science, Israel
| | - Brian B. Jeon
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Center for the Neural Basis of Cognition, PA
| | - Ranjani Srinivasan
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Department of Electrical and Computer Engineering, Johns Hopkins University, MD
| | - Yu Mu
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, China
| | - Ziqiang Wei
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Sandra J. Kuhlman
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
- Department of Biological Sciences, Carnegie Mellon University, PA
| | - Misha B. Ahrens
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Steven M. Chase
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
| | - Byron M. Yu
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
- Department of Electrical and Computer Engineering, Carnegie Mellon University, PA
| |
Collapse
|
16
|
Jensen KT, Kadmon Harpaz N, Dhawale AK, Wolff SBE, Ölveczky BP. Long-term stability of single neuron activity in the motor system. Nat Neurosci 2022; 25:1664-1674. [PMID: 36357811 PMCID: PMC11152193 DOI: 10.1038/s41593-022-01194-3] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 10/03/2022] [Indexed: 11/12/2022]
Abstract
How an established behavior is retained and consistently produced by a nervous system in constant flux remains a mystery. One possible solution to ensure long-term stability in motor output is to fix the activity patterns of single neurons in the relevant circuits. Alternatively, activity in single cells could drift over time provided that the population dynamics are constrained to produce the same behavior. To arbitrate between these possibilities, we recorded single-unit activity in motor cortex and striatum continuously for several weeks as rats performed stereotyped motor behaviors-both learned and innate. We found long-term stability in single neuron activity patterns across both brain regions. A small amount of drift in neural activity, observed over weeks of recording, could be explained by concomitant changes in task-irrelevant aspects of the behavior. These results suggest that long-term stable behaviors are generated by single neuron activity patterns that are themselves highly stable.
Collapse
Affiliation(s)
- Kristopher T Jensen
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
| | - Naama Kadmon Harpaz
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Ashesh K Dhawale
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
- Centre for Neuroscience, Indian Institute of Science, Bangalore, India
| | - Steffen B E Wolff
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Bence P Ölveczky
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
17
|
Temporal scaling of human scalp-recorded potentials. Proc Natl Acad Sci U S A 2022; 119:e2214638119. [PMID: 36256817 PMCID: PMC9618087 DOI: 10.1073/pnas.2214638119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Neural activity is traditionally thought to occur over fixed timescales. However, recent animal work has suggested that some neural responses occur over varying timescales. We extended this animal result to humans by detecting temporally scaled signals noninvasively at the scalp in four different tasks. Our results suggest that temporal scaling is an important feature of cognitive processes known to unfold over varying timescales. Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp–recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.
Collapse
|
18
|
van Bree S, Melcón M, Kolibius LD, Kerrén C, Wimber M, Hanslmayr S. The brain time toolbox, a software library to retune electrophysiology data to brain dynamics. Nat Hum Behav 2022; 6:1430-1439. [PMID: 35726055 DOI: 10.1038/s41562-022-01386-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 05/12/2022] [Indexed: 12/21/2022]
Abstract
Human thought is highly flexible, achieved by evolving patterns of brain activity across groups of cells. Neuroscience aims to understand cognition in the brain by analysing these intricate patterns. We argue that this goal is impeded by the time format of our data-clock time. The brain is a system with its own dynamics and regime of time, with no intrinsic concern for the human-invented second. Here, we present the Brain Time Toolbox, a software library that retunes electrophysiology data in line with oscillations that orchestrate neural patterns of cognition. These oscillations continually slow down, speed up and undergo abrupt changes, introducing a disharmony between the brain's internal regime and clock time. The toolbox overcomes this disharmony by warping the data to the dynamics of coordinating oscillations, setting oscillatory cycles as the data's new time axis. This enables the study of neural patterns as they unfold in the brain, aiding neuroscientific enquiry into dynamic cognition. In support of this, we demonstrate that the toolbox can reveal results that are absent in a default clock time format.
Collapse
Affiliation(s)
- Sander van Bree
- Centre for Cognitive Neuroimaging, School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK.
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
| | - María Melcón
- Department of Biological and Health Psychology, Autónoma University of Madrid, Madrid, Spain
| | - Luca D Kolibius
- Centre for Cognitive Neuroimaging, School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Casper Kerrén
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| | - Maria Wimber
- Centre for Cognitive Neuroimaging, School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Simon Hanslmayr
- Centre for Cognitive Neuroimaging, School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| |
Collapse
|
19
|
Dorman DB, Blackwell KT. Synaptic Plasticity Is Predicted by Spatiotemporal Firing Rate Patterns and Robust to In Vivo-like Variability. Biomolecules 2022; 12:1402. [PMID: 36291612 PMCID: PMC9599115 DOI: 10.3390/biom12101402] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 09/13/2022] [Accepted: 09/28/2022] [Indexed: 11/22/2022] Open
Abstract
Synaptic plasticity, the experience-induced change in connections between neurons, underlies learning and memory in the brain. Most of our understanding of synaptic plasticity derives from in vitro experiments with precisely repeated stimulus patterns; however, neurons exhibit significant variability in vivo during repeated experiences. Further, the spatial pattern of synaptic inputs to the dendritic tree influences synaptic plasticity, yet is not considered in most synaptic plasticity rules. Here, we investigate how spatiotemporal synaptic input patterns produce plasticity with in vivo-like conditions using a data-driven computational model with a plasticity rule based on calcium dynamics. Using in vivo spike train recordings as inputs to different size clusters of spines, we show that plasticity is strongly robust to trial-to-trial variability of spike timing. In addition, we derive general synaptic plasticity rules describing how spatiotemporal patterns of synaptic inputs control the magnitude and direction of plasticity. Synapses that strongly potentiated have greater firing rates and calcium concentration later in the trial, whereas strongly depressing synapses have hiring firing rates early in the trial. The neighboring synaptic activity influences the direction and magnitude of synaptic plasticity, with small clusters of spines producing the greatest increase in synaptic strength. Together, our results reveal that calcium dynamics can unify diverse plasticity rules and reveal how spatiotemporal firing rate patterns control synaptic plasticity.
Collapse
Affiliation(s)
- Daniel B. Dorman
- Interdisciplinary Program in Neuroscience, George Mason University, Fairfax, VA 22030, USA
| | - Kim T. Blackwell
- Interdisciplinary Program in Neuroscience, George Mason University, Fairfax, VA 22030, USA
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, VA 22030, USA
| |
Collapse
|
20
|
Shafik W, Matinkhah SM, Shokoor F, Sharif L. A reawakening of Machine Learning Application in Unmanned Aerial Vehicle: Future Research Motivation. EAI ENDORSED TRANSACTIONS ON INTERNET OF THINGS 2022. [DOI: 10.4108/eetiot.v8i29.987] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Machine learning (ML) entails artificial procedures that improve robotically through experience and using data. Supervised, unsupervised, semi-supervised, and Reinforcement Learning (RL) are the main types of ML. This study mainly focuses on RL and Deep learning, since necessitates mainly sequential and consecutive decision-making context. This is a comparison to supervised and non-supervised learning due to the interactive nature of the environment. Exploiting a forthcoming accumulative compensation and its stimulus of machines, complex policy decisions. The study further analyses and presents ML perspectives depicting state-of-the-art developments with advancement, relatively depicting the future trend of RL based on its applicability in technology. It's a challenge to an Internet of Things (IoT) and demonstrates what possibly can be adopted as a solution. This study presented a summarized perspective on identified arenas on the analysis of RL. The study scrutinized that a reasonable number of the techniques engrossed in alternating policy values instead of modifying other gears in an exact state of intellectual. The study presented a strong foundation for the current studies to be adopted by the researchers from different research backgrounds to develop models, and architectures that are relevant.
Collapse
|
21
|
Wimalasena LN, Braun J, Keshtkaran MR, Hofmann D, Gallego JÁ, Alessandro C, Tresch M, Miller LE, Pandarinath C. Estimating muscle activation from EMG using deep learning-based dynamical systems models. J Neural Eng 2022; 19. [PMID: 35366649 DOI: 10.1088/1741-2552/ac6369] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Accepted: 04/01/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE To study the neural control of movement, it is often necessary to estimate how muscles are activated across a variety of behavioral conditions. One approach is to try extracting the underlying neural command signal to muscles by applying latent variable modeling methods to electromyographic (EMG) recordings. However, estimating the latent command signal that underlies muscle activation is challenging due to its complex relation with recorded EMG signals. Common approaches estimate each muscle activation independently or require manual tuning of model hyperparameters to preserve behaviorally-relevant features. APPROACH Here, we adapted AutoLFADS, a large-scale, unsupervised deep learning approach originally designed to de-noise cortical spiking data, to estimate muscle activation from multi-muscle EMG signals. AutoLFADS uses recurrent neural networks (RNNs) to model the spatial and temporal regularities that underlie multi-muscle activation. MAIN RESULTS We first tested AutoLFADS on muscle activity from the rat hindlimb during locomotion and found that it dynamically adjusts its frequency response characteristics across different phases of behavior. The model produced single-trial estimates of muscle activation that improved prediction of joint kinematics as compared to low-pass or Bayesian filtering. We also applied AutoLFADS to monkey forearm muscle activity recorded during an isometric wrist force task. AutoLFADS uncovered previously uncharacterized high-frequency oscillations in the EMG that enhanced the correlation with measured force. The AutoLFADS-inferred estimates of muscle activation were also more closely correlated with simultaneously-recorded motor cortical activity than were other tested approaches. SIGNIFICANCE This method leverages dynamical systems modeling and artificial neural networks to provide estimates of muscle activation for multiple muscles. Ultimately, the approach can be used for further studies of multi-muscle coordination and its control by upstream brain areas.
Collapse
Affiliation(s)
- Lahiru Neth Wimalasena
- Biomedical Engineering, Emory University, 101 Woodruff Circle NE, Atlanta, Georgia, 30322-1007, UNITED STATES
| | - Jonas Braun
- Electrical and Computer Engineering, Technical University of Munich, Arcisstraße 21, Munchen, Bayern, 80333, GERMANY
| | - Mohammad Reza Keshtkaran
- Biomedical Engineering, Emory University, 101 Woodruff Circle NE, Atlanta, Georgia, 30322-1007, UNITED STATES
| | - David Hofmann
- Physics, Emory University, Math & Science Center, 400 Dowman Drive, Atlanta, Georgia, 30322-1007, UNITED STATES
| | - Juan Álvaro Gallego
- Physiology, Northwestern University Feinberg School of Medicine, 303 East Chicago Ave, Chicago, Illinois, 60611-3008, UNITED STATES
| | - Cristiano Alessandro
- Physiology, Northwestern University Feinberg School of Medicine, 303 East Chicago Ave, Chicago, Illinois, 60611-3008, UNITED STATES
| | - Matthew Tresch
- Physiology, Northwestern University Feinberg School of Medicine, 303 East Chicago Ave, Chicago, Illinois, 60611-3008, UNITED STATES
| | - Lee E Miller
- Physiology, Northwestern University Feinberg School of Medicine, 303 East Chicago Ave, Chicago, Illinois, 60611-3008, UNITED STATES
| | - Chethan Pandarinath
- Biomedical Engineering, Emory University, 101 Woodruff Circle NE, Atlanta, Georgia, 30322-1007, UNITED STATES
| |
Collapse
|
22
|
Xu Z, Zhou X, Xu Y, Wu W. Removing nonlinear misalignment in neuronal spike trains using the Fisher-Rao registration framework. J Neurosci Methods 2022; 367:109436. [PMID: 34890697 DOI: 10.1016/j.jneumeth.2021.109436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Revised: 11/29/2021] [Accepted: 12/02/2021] [Indexed: 10/19/2022]
Abstract
BACKGROUND The temporal precision in neural spike train data is critically important for understanding functional mechanism in the nervous systems. However, the timing variability of spiking activity can be highly nonlinear in practical observations due to behavioral variability or unobserved/unobservable cognitive states. NEW METHOD In this study, we propose to adopt a powerful nonlinear method, referred to as the Fisher-Rao Registration (FRR), to remove such nonlinear phase variability in discrete neuronal spike trains. We also develop a smoothing procedure on the discrete spike train data in order to use the FRR framework. COMPARISON WITH EXISTING METHODS We systematically compare the FRR with the state-of-the-art linear and nonlinear methods in terms of model efficiency and effectiveness. RESULTS We show that the FRR has superior performance and the advantages are well illustrated with simulation and real experimental data. CONCLUSIONS It is found the FRR framework provides more appropriate alignment performance to understand the temporal variability in neuronal spike trains.
Collapse
Affiliation(s)
- Zishen Xu
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| | - Xinyu Zhou
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| | - Yiqi Xu
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| | - Wei Wu
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| |
Collapse
|
23
|
Porrmann F, Pilz S, Stella A, Kleinjohann A, Denker M, Hagemeyer J, Rückert U. Acceleration of the SPADE Method Using a Custom-Tailored FP-Growth Implementation. Front Neuroinform 2021; 15:723406. [PMID: 34603002 PMCID: PMC8483730 DOI: 10.3389/fninf.2021.723406] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Accepted: 08/16/2021] [Indexed: 11/13/2022] Open
Abstract
The SPADE (spatio-temporal Spike PAttern Detection and Evaluation) method was developed to find reoccurring spatio-temporal patterns in neuronal spike activity (parallel spike trains). However, depending on the number of spike trains and the length of recording, this method can exhibit long runtimes. Based on a realistic benchmark data set, we identified that the combination of pattern mining (using the FP-Growth algorithm) and the result filtering account for 85–90% of the method's total runtime. Therefore, in this paper, we propose a customized FP-Growth implementation tailored to the requirements of SPADE, which significantly accelerates pattern mining and result filtering. Our version allows for parallel and distributed execution, and due to the improvements made, an execution on heterogeneous and low-power embedded devices is now also possible. The implementation has been evaluated using a traditional workstation based on an Intel Broadwell Xeon E5-1650 v4 as a baseline. Furthermore, the heterogeneous microserver platform RECS|Box has been used for evaluating the implementation on two HiSilicon Hi1616 (Kunpeng 916), an Intel Coffee Lake-ER Xeon E-2276ME, an Intel Broadwell Xeon D-D1577, and three NVIDIA Tegra devices (Jetson AGX Xavier, Jetson Xavier NX, and Jetson TX2). Depending on the platform, our implementation is between 27 and 200 times faster than the original implementation. At the same time, the energy consumption was reduced by up to two orders of magnitude.
Collapse
Affiliation(s)
- Florian Porrmann
- Cognitronics and Sensor Systems, CITEC, Bielefeld University, Bielefeld, Germany
| | - Sarah Pilz
- Cognitronics and Sensor Systems, CITEC, Bielefeld University, Bielefeld, Germany
| | - Alessandra Stella
- Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Center, Jülich, Germany.,RWTH Aachen University, Aachen, Germany
| | - Alexander Kleinjohann
- Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Center, Jülich, Germany.,RWTH Aachen University, Aachen, Germany
| | - Michael Denker
- Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Center, Jülich, Germany
| | - Jens Hagemeyer
- Cognitronics and Sensor Systems, CITEC, Bielefeld University, Bielefeld, Germany
| | - Ulrich Rückert
- Cognitronics and Sensor Systems, CITEC, Bielefeld University, Bielefeld, Germany
| |
Collapse
|
24
|
Williams AH, Linderman SW. Statistical neuroscience in the single trial limit. Curr Opin Neurobiol 2021; 70:193-205. [PMID: 34861596 DOI: 10.1016/j.conb.2021.10.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Revised: 09/29/2021] [Accepted: 10/27/2021] [Indexed: 11/24/2022]
Abstract
Individual neurons often produce highly variable responses over nominally identical trials, reflecting a mixture of intrinsic 'noise' and systematic changes in the animal's cognitive and behavioral state. Disentangling these sources of variability is of great scientific interest in its own right, but it is also increasingly inescapable as neuroscientists aspire to study more complex and naturalistic animal behaviors. In these settings, behavioral actions never repeat themselves exactly and may rarely do so even approximately. Thus, new statistical methods that extract reliable features of neural activity using few, if any, repeated trials are needed. Accurate statistical modeling in this severely trial-limited regime is challenging, but still possible if simplifying structure in neural data can be exploited. We review recent works that have identified different forms of simplifying structure - including shared gain modulations across neural subpopulations, temporal smoothness in neural firing rates, and correlations in responses across behavioral conditions - and exploited them to reveal novel insights into the trial-by-trial operation of neural circuits.
Collapse
Affiliation(s)
- Alex H Williams
- Department of Statistics and Wu Tsai Neurosciences Institute, Stanford University, USA
| | - Scott W Linderman
- Department of Statistics and Wu Tsai Neurosciences Institute, Stanford University, USA.
| |
Collapse
|
25
|
Selection of Essential Neural Activity Timesteps for Intracortical Brain-Computer Interface Based on Recurrent Neural Network. SENSORS 2021; 21:s21196372. [PMID: 34640699 PMCID: PMC8512903 DOI: 10.3390/s21196372] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Revised: 09/15/2021] [Accepted: 09/20/2021] [Indexed: 11/29/2022]
Abstract
Intracortical brain–computer interfaces (iBCIs) translate neural activity into control commands, thereby allowing paralyzed persons to control devices via their brain signals. Recurrent neural networks (RNNs) are widely used as neural decoders because they can learn neural response dynamics from continuous neural activity. Nevertheless, excessively long or short input neural activity for an RNN may decrease its decoding performance. Based on the temporal attention module exploiting relations in features over time, we propose a temporal attention-aware timestep selection (TTS) method that improves the interpretability of the salience of each timestep in an input neural activity. Furthermore, TTS determines the appropriate input neural activity length for accurate neural decoding. Experimental results show that the proposed TTS efficiently selects 28 essential timesteps for RNN-based neural decoders, outperforming state-of-the-art neural decoders on two nonhuman primate datasets (R2=0.76±0.05 for monkey Indy and CC=0.91±0.01 for monkey N). In addition, it reduces the computation time for offline training (reducing 5–12%) and online prediction (reducing 16–18%). When visualizing the attention mechanism in TTS, the preparatory neural activity is consecutively highlighted during arm movement, and the most recent neural activity is highlighted during the resting state in nonhuman primates. Selecting only a few essential timesteps for an RNN-based neural decoder provides sufficient decoding performance and requires only a short computation time.
Collapse
|
26
|
Isbister JB, Reyes-Puerta V, Sun JJ, Horenko I, Luhmann HJ. Clustering and control for adaptation uncovers time-warped spike time patterns in cortical networks in vivo. Sci Rep 2021; 11:15066. [PMID: 34326363 PMCID: PMC8322153 DOI: 10.1038/s41598-021-94002-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 06/29/2021] [Indexed: 12/04/2022] Open
Abstract
How information in the nervous system is encoded by patterns of action potentials (i.e. spikes) remains an open question. Multi-neuron patterns of single spikes are a prime candidate for spike time encoding but their temporal variability requires further characterisation. Here we show how known sources of spike count variability affect stimulus-evoked spike time patterns between neurons separated over multiple layers and columns of adult rat somatosensory cortex in vivo. On subsets of trials (clusters) and after controlling for stimulus-response adaptation, spike time differences between pairs of neurons are “time-warped” (compressed/stretched) by trial-to-trial changes in shared excitability, explaining why fixed spike time patterns and noise correlations are seldom reported. We show that predicted cortical state is correlated between groups of 4 neurons, introducing the possibility of spike time pattern modulation by population-wide trial-to-trial changes in excitability (i.e. cortical state). Under the assumption of state-dependent coding, we propose an improved potential encoding capacity.
Collapse
Affiliation(s)
- James B Isbister
- Oxford Centre for Theoretical Neuroscience and Artificial Intelligence, Department of Experimental Psychology, University of Oxford, Oxford, UK. .,The Blue Brain Project, École Polytechnique Fédérale de Lausanne, 1202, Geneva, Switzerland.
| | - Vicente Reyes-Puerta
- Institute of Physiology, University Medical Center, Johannes Gutenberg University, Mainz, Germany
| | - Jyh-Jang Sun
- Institute of Physiology, University Medical Center, Johannes Gutenberg University, Mainz, Germany.,NERF, Kapeldreef 75, 3001, Leuven, Belgium.,imec, Remisebosweg 1, 3001, Leuven, Belgium
| | - Illia Horenko
- Faculty of Informatics, Universita della Svizzera Italiana, Via G. Buffi 13, 6900, Lugano, Switzerland
| | - Heiko J Luhmann
- Institute of Physiology, University Medical Center, Johannes Gutenberg University, Mainz, Germany
| |
Collapse
|
27
|
Stanford TR, Salinas E. Urgent Decision Making: Resolving Visuomotor Interactions at High Temporal Resolution. Annu Rev Vis Sci 2021; 7:323-348. [PMID: 34171199 DOI: 10.1146/annurev-vision-100419-103842] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Measuring when exactly perceptual decisions are made is crucial for defining how the activation of specific neurons contributes to behavior. However, in traditional, nonurgent visuomotor tasks, the uncertainty of this temporal measurement is very large. This is a problem not only for delimiting the capacity of perception, but also for correctly interpreting the functional roles ascribed to choice-related neuronal responses. In this article, we review psychophysical, neurophysiological, and modeling work based on urgent visuomotor tasks in which this temporal uncertainty can be effectively overcome. The cornerstone of this work is a novel behavioral metric that describes the evolution of the subject's perceptual judgment moment by moment, allowing us to resolve numerous perceptual events that unfold within a few tens of milliseconds. In this framework, the neural distinction between perceptual evaluation and motor selection processes becomes particularly clear, as the conclusion of one is not contingent on that of the other. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Terrence R Stanford
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, North Carolina 27157, USA; ,
| | - Emilio Salinas
- Department of Neurobiology & Anatomy, Wake Forest School of Medicine, Winston-Salem, North Carolina 27157, USA; ,
| |
Collapse
|
28
|
Noel JP, Caziot B, Bruni S, Fitzgerald NE, Avila E, Angelaki DE. Supporting generalization in non-human primate behavior by tapping into structural knowledge: Examples from sensorimotor mappings, inference, and decision-making. Prog Neurobiol 2021; 201:101996. [PMID: 33454361 PMCID: PMC8096669 DOI: 10.1016/j.pneurobio.2021.101996] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Revised: 12/15/2020] [Accepted: 01/12/2021] [Indexed: 02/05/2023]
Abstract
The complex behaviors we ultimately wish to understand are far from those currently used in systems neuroscience laboratories. A salient difference are the closed loops between action and perception prominently present in natural but not laboratory behaviors. The framework of reinforcement learning and control naturally wades across action and perception, and thus is poised to inform the neurosciences of tomorrow, not only from a data analyses and modeling framework, but also in guiding experimental design. We argue that this theoretical framework emphasizes active sensing, dynamical planning, and the leveraging of structural regularities as key operations for intelligent behavior within uncertain, time-varying environments. Similarly, we argue that we may study natural task strategies and their neural circuits without over-training animals when the tasks we use tap into our animal's structural knowledge. As proof-of-principle, we teach animals to navigate through a virtual environment - i.e., explore a well-defined and repetitive structure governed by the laws of physics - using a joystick. Once these animals have learned to 'drive', without further training they naturally (i) show zero- or one-shot learning of novel sensorimotor contingencies, (ii) infer the evolving path of dynamically changing latent variables, and (iii) make decisions consistent with maximizing reward rate. Such task designs allow for the study of flexible and generalizable, yet controlled, behaviors. In turn, they allow for the exploitation of pillars of intelligence - flexibility, prediction, and generalization -, properties whose neural underpinning have remained elusive.
Collapse
Affiliation(s)
- Jean-Paul Noel
- Center for Neural Science, New York University, New York, USA
| | - Baptiste Caziot
- Center for Neural Science, New York University, New York, USA
| | - Stefania Bruni
- Center for Neural Science, New York University, New York, USA
| | | | - Eric Avila
- Center for Neural Science, New York University, New York, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, USA; Tandon School of Engineering, New York University, New York, USA.
| |
Collapse
|
29
|
Goffinet J, Brudner S, Mooney R, Pearson J. Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires. eLife 2021; 10:e67855. [PMID: 33988503 PMCID: PMC8213406 DOI: 10.7554/elife.67855] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 05/12/2021] [Indexed: 11/16/2022] Open
Abstract
Increases in the scale and complexity of behavioral data pose an increasing challenge for data analysis. A common strategy involves replacing entire behaviors with small numbers of handpicked, domain-specific features, but this approach suffers from several crucial limitations. For example, handpicked features may miss important dimensions of variability, and correlations among them complicate statistical testing. Here, by contrast, we apply the variational autoencoder (VAE), an unsupervised learning method, to learn features directly from data and quantify the vocal behavior of two model species: the laboratory mouse and the zebra finch. The VAE converges on a parsimonious representation that outperforms handpicked features on a variety of common analysis tasks, enables the measurement of moment-by-moment vocal variability on the timescale of tens of milliseconds in the zebra finch, provides strong evidence that mouse ultrasonic vocalizations do not cluster as is commonly believed, and captures the similarity of tutor and pupil birdsong with qualitatively higher fidelity than previous approaches. In all, we demonstrate the utility of modern unsupervised learning approaches to the quantification of complex and high-dimensional vocal behavior.
Collapse
Affiliation(s)
- Jack Goffinet
- Department of Computer Science, Duke UniversityDurhamUnited States
- Center for Cognitive Neurobiology, Duke UniversityDurhamUnited States
- Department of Neurobiology, Duke UniversityDurhamUnited States
| | - Samuel Brudner
- Department of Neurobiology, Duke UniversityDurhamUnited States
| | - Richard Mooney
- Department of Neurobiology, Duke UniversityDurhamUnited States
| | - John Pearson
- Center for Cognitive Neurobiology, Duke UniversityDurhamUnited States
- Department of Neurobiology, Duke UniversityDurhamUnited States
- Department of Biostatistics & Bioinformatics, Duke UniversityDurhamUnited States
- Department of Electrical and Computer Engineering, Duke UniversityDurhamUnited States
| |
Collapse
|
30
|
High-performance brain-to-text communication via handwriting. Nature 2021; 593:249-254. [PMID: 33981047 DOI: 10.1038/s41586-021-03506-2] [Citation(s) in RCA: 248] [Impact Index Per Article: 82.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Accepted: 03/26/2021] [Indexed: 12/14/2022]
Abstract
Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. So far, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping1-5 or point-and-click typing with a computer cursor6,7. However, rapid sequences of highly dexterous behaviours, such as handwriting or touch typing, might enable faster rates of communication. Here we developed an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time, using a recurrent neural network decoding approach. With this BCI, our study participant, whose hand was paralysed from spinal cord injury, achieved typing speeds of 90 characters per minute with 94.1% raw accuracy online, and greater than 99% accuracy offline with a general-purpose autocorrect. To our knowledge, these typing speeds exceed those reported for any other BCI, and are comparable to typical smartphone typing speeds of individuals in the age group of our participant (115 characters per minute)8. Finally, theoretical considerations explain why temporally complex movements, such as handwriting, may be fundamentally easier to decode than point-to-point movements. Our results open a new approach for BCIs and demonstrate the feasibility of accurately decoding rapid, dexterous movements years after paralysis.
Collapse
|
31
|
Reisert J, Golden GJ, Dibattista M, Gelperin A. Odor sampling strategies in mice with genetically altered olfactory responses. PLoS One 2021; 16:e0249798. [PMID: 33939692 PMCID: PMC8092659 DOI: 10.1371/journal.pone.0249798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 03/25/2021] [Indexed: 12/04/2022] Open
Abstract
Peripheral sensory cells and the central neuronal circuits that monitor environmental changes to drive behaviors should be adapted to match the behaviorally relevant kinetics of incoming stimuli, be it the detection of sound frequencies, the speed of moving objects or local temperature changes. Detection of odorants begins with the activation of olfactory receptor neurons in the nasal cavity following inhalation of air and airborne odorants carried therein. Thus, olfactory receptor neurons are stimulated in a rhythmic and repeated fashion that is determined by the breathing or sniffing frequency that can be controlled and altered by the animal. This raises the question of how the response kinetics of olfactory receptor neurons are matched to the imposed stimulation frequency and if, vice versa, the kinetics of olfactory receptor neuron responses determine the sniffing frequency. We addressed this question by using a mouse model that lacks the K+-dependent Na+/Ca2+ exchanger 4 (NCKX4), which results in markedly slowed response termination of olfactory receptor neuron responses and hence changes the temporal response kinetics of these neurons. We monitored sniffing behaviors of freely moving wildtype and NCKX4 knockout mice while they performed olfactory Go/NoGo discrimination tasks. Knockout mice performed with similar or, surprisingly, better accuracy compared to wildtype mice, but chose, depending on the task, different odorant sampling durations depending on the behavioral demands of the odorant identification task. Similarly, depending on the demands of the behavioral task, knockout mice displayed a lower basal breathing frequency prior to odorant sampling, a possible mechanism to increase the dynamic range for changes in sniffing frequency during odorant sampling. Overall, changes in sniffing behavior between wildtype and NCKX4 knockout mice were subtle, suggesting that, at least for the particular odorant-driven task we used, slowed response termination of the odorant-induced receptor neuron response either has a limited detrimental effect on odorant-driven behavior or mice are able to compensate via an as yet unknown mechanism.
Collapse
Affiliation(s)
- Johannes Reisert
- Monell Chemical Senses Center, Philadelphia, PA, United States of America
- * E-mail: (JR); (AG)
| | - Glen J. Golden
- Monell Chemical Senses Center, Philadelphia, PA, United States of America
| | - Michele Dibattista
- Department of Basic Medical Sciences, Neuroscience and Sensory Organs, University of Bari “A. Moro”, Bari, Italy
| | - Alan Gelperin
- Princeton Neuroscience Program, Princeton University, Princeton, NJ, United States of America
- * E-mail: (JR); (AG)
| |
Collapse
|
32
|
Roth RH, Ding JB. From Neurons to Cognition: Technologies for Precise Recording of Neural Activity Underlying Behavior. BME FRONTIERS 2020; 2020:7190517. [PMID: 37849967 PMCID: PMC10521756 DOI: 10.34133/2020/7190517] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Accepted: 10/27/2020] [Indexed: 10/19/2023] Open
Abstract
Understanding how brain activity encodes information and controls behavior is a long-standing question in neuroscience. This complex problem requires converging efforts from neuroscience and engineering, including technological solutions to perform high-precision and large-scale recordings of neuronal activity in vivo as well as unbiased methods to reliably measure and quantify behavior. Thanks to advances in genetics, molecular biology, engineering, and neuroscience, in recent decades, a variety of optical imaging and electrophysiological approaches for recording neuronal activity in awake animals have been developed and widely applied in the field. Moreover, sophisticated computer vision and machine learning algorithms have been developed to analyze animal behavior. In this review, we provide an overview of the current state of technology for neuronal recordings with a focus on optical and electrophysiological methods in rodents. In addition, we discuss areas that future technological development will need to cover in order to further our understanding of the neural activity underlying behavior.
Collapse
Affiliation(s)
- Richard H Roth
- Department of Neurosurgery, Stanford University, Stanford, CA 94305, USA
| | - Jun B Ding
- Department of Neurosurgery, Stanford University, Stanford, CA 94305, USA
- Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA 94305, USA
| |
Collapse
|
33
|
Williams AH, Degleris A, Wang Y, Linderman SW. Point process models for sequence detection in high-dimensional neural spike trains. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 2020; 33:14350-14361. [PMID: 35002191 PMCID: PMC8734964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Sparse sequences of neural spikes are posited to underlie aspects of working memory [1], motor production [2], and learning [3, 4]. Discovering these sequences in an unsupervised manner is a longstanding problem in statistical neuroscience [5-7]. Promising recent work [4, 8] utilized a convolutive nonnegative matrix factorization model [9] to tackle this challenge. However, this model requires spike times to be discretized, utilizes a sub-optimal least-squares criterion, and does not provide uncertainty estimates for model predictions or estimated parameters. We address each of these shortcomings by developing a point process model that characterizes fine-scale sequences at the level of individual spikes and represents sequence occurrences as a small number of marked events in continuous time. This ultra-sparse representation of sequence events opens new possibilities for spike train modeling. For example, we introduce learnable time warping parameters to model sequences of varying duration, which have been experimentally observed in neural circuits [10]. We demonstrate these advantages on experimental recordings from songbird higher vocal center and rodent hippocampus.
Collapse
Affiliation(s)
- Alex H Williams
- Department of Statistics, Stanford University, Stanford, CA 94305
| | - Anthony Degleris
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305
| | - Yixin Wang
- Department of Statistics, Columbia University, New York NY 10027
| | | |
Collapse
|
34
|
Zhao W, Xu Z, Li W, Wu W. Modeling and analyzing neural signals with phase variability using Fisher-Rao registration. J Neurosci Methods 2020; 346:108954. [PMID: 32950555 DOI: 10.1016/j.jneumeth.2020.108954] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2020] [Revised: 09/11/2020] [Accepted: 09/16/2020] [Indexed: 01/02/2023]
Abstract
BACKGROUND The dynamic time warping (DTW) has recently been introduced to analyze neural signals such as EEG and fMRI where phase variability plays an important role in the data. NEW METHOD In this study, we propose to adopt a more powerful method, referred to as the Fisher-Rao Registration (FRR), to study the phase variability. COMPARISON WITH EXISTING METHODS We systematically compare FRR with DTW in three aspects: (1) basic framework, (2) mathematical properties, and (3) computational efficiency. RESULTS We show that FRR has superior performance in all these aspects and the advantages are well illustrated with simulation examples. CONCLUSIONS We then apply the FRR method to two real experimental recordings - one fMRI and one EEG data set. It is found the FRR method properly removes the phase variability in each set. Finally, we use the FRR framework to examine brain networks in these two data sets and the result demonstrates the effectiveness of the new method.
Collapse
Affiliation(s)
- Weilong Zhao
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| | - Zishen Xu
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| | - Wen Li
- Department of Psychology, Florida State University, 1107 W. Call St., Tallahassee, FL 32306-4301, USA
| | - Wei Wu
- Department of Statistics, Florida State University, 117 N Woodward Ave., Tallahassee, FL 32306-4330, USA
| |
Collapse
|
35
|
Torres JJ, Baroni F, Latorre R, Varona P. Temporal discrimination from the interaction between dynamic synapses and intrinsic subthreshold oscillations. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.07.031] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
36
|
Kawabata M, Soma S, Saiki-Ishikawa A, Nonomura S, Yoshida J, Ríos A, Sakai Y, Isomura Y. A spike analysis method for characterizing neurons based on phase locking and scaling to the interval between two behavioral events. J Neurophysiol 2020; 124:1923-1941. [PMID: 33085554 DOI: 10.1152/jn.00200.2020] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
Standard analysis of neuronal functions assesses the temporal correlation between animal behaviors and neuronal activity by aligning spike trains with the timing of a specific behavioral event, e.g., visual cue. However, spike activity is often involved in information processing dependent on a relative phase between two consecutive events rather than a single event. Nevertheless, less attention has so far been paid to such temporal features of spike activity in relation to two behavioral events. Here, we propose "Phase-Scaling analysis" to simultaneously evaluate the phase locking and scaling to the interval between two events in task-related spike activity of individual neurons. This analysis method can discriminate conceptual "scaled"-type neurons from "nonscaled"-type neurons using an activity variation map that combines phase locking with scaling to the interval. Its robustness was validated by spike simulation using different spike properties. Furthermore, we applied it to analyzing actual spike data from task-related neurons in the primary visual cortex (V1), posterior parietal cortex (PPC), primary motor cortex (M1), and secondary motor cortex (M2) of behaving rats. After hierarchical clustering of all neurons using their activity variation maps, we divided them objectively into four clusters corresponding to nonscaled-type sensory and motor neurons and scaled-type neurons including sustained and ramping activities, etc. Cluster/subcluster compositions for V1 differed from those of PPC, M1, and M2. The V1 neurons showed the fastest functional activities among those areas. Our method was also applicable to determine temporal "forms" and the latency of spike activity changes. These findings demonstrate its utility for characterizing neurons.NEW & NOTEWORTHY Phase-Scaling analysis is a novel technique to unbiasedly characterize the temporal dependency of functional neuron activity on two behavioral events and objectively determine the latency and form of the activity change. This powerful analysis can uncover several classes of latently functioning neurons that have thus far been overlooked, which may participate differently in intermediate processes of a brain function. The Phase-Scaling analysis will yield profound insights into neural mechanisms for processing internal information.
Collapse
Affiliation(s)
- Masanori Kawabata
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan.,Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan
| | - Shogo Soma
- Brain Science Institute, Tamagawa University, Tokyo, Japan.,Department of Molecular Cell Physiology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Akiko Saiki-Ishikawa
- Brain Science Institute, Tamagawa University, Tokyo, Japan.,Department of Neurobiology, Northwestern University, Evanston, Illinois
| | - Satoshi Nonomura
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan.,Brain Science Institute, Tamagawa University, Tokyo, Japan.,Systems Neuroscience Section, Primate Research Institute, Kyoto University, Aichi, Japan
| | - Junichi Yoshida
- Brain Science Institute, Tamagawa University, Tokyo, Japan.,Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, New York
| | - Alain Ríos
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan.,Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan
| | - Yutaka Sakai
- Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan.,Brain Science Institute, Tamagawa University, Tokyo, Japan
| | - Yoshikazu Isomura
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan.,Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan.,Brain Science Institute, Tamagawa University, Tokyo, Japan
| |
Collapse
|
37
|
Reisert J, Golden GJ, Dibattista M, Gelperin A. Dynamics of odor sampling strategies in mice. PLoS One 2020; 15:e0237756. [PMID: 32797072 PMCID: PMC7428156 DOI: 10.1371/journal.pone.0237756] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 07/31/2020] [Indexed: 11/18/2022] Open
Abstract
Mammalian olfactory receptor neurons in the nasal cavity are stimulated by odorants carried by the inhaled air and their activation is therefore tied to and driven by the breathing or sniffing frequency. Sniffing frequency can be deliberately modulated to alter how odorants stimulate olfactory receptor neurons, giving the animal control over the frequency of odorant exposure to potentially aid odorant detection and discrimination. We monitored sniffing behaviors and odorant discrimination ability of freely-moving mice while they sampled either decreasing concentrations of target odorants or sampled a fixed target odorant concentration in the presence of a background of increasing odorant concentrations, using a Go-NoGo behavioral paradigm. This allowed us to ask how mice alter their odorant sampling duration and sampling (sniffing) frequency depending on the demands of the task and its difficulty. Mice showed an anticipatory increase in sniffing rate prior to odorant exposure and chose to sample for longer durations when exposed to odorants as compared to the solvent control odorant. Similarly, mice also took more odorant sampling sniffs when exposed to target odorants compared to the solvent control odorant. In general, odorant sampling strategies became more similar the more difficult the task was, e.g. the lower the target odorant concentration or the lower the target odorant contrast relative to the background odorant, suggesting that sniffing patterns are not preset, but are dynamically modulated by the particular task and its difficulty.
Collapse
Affiliation(s)
- Johannes Reisert
- Monell Chemical Senses Center, Philadelphia, PA, United States of America
| | - Glen J. Golden
- Monell Chemical Senses Center, Philadelphia, PA, United States of America
| | - Michele Dibattista
- Department of Basic Medical Sciences, Neuroscience and Sensory Organs, University of Bari “A. Moro”, Bari, Italy
| | - Alan Gelperin
- Department of Neuroscience, Princeton University, Princeton, NJ, United States of America
| |
Collapse
|
38
|
Abstract
Neuronal population responses can vary across trials as a result of differences in the relative timing of internal brain processes. In this issue of Neuron, Williams et al. (2020) present an algorithm for inferring and inverting such trial-to-trial differences, thereby revealing an a priori hidden, precise temporal structure of population responses.
Collapse
|