1
|
Kuo YT, Wang HL, Chen BW, Wang CF, Lo YC, Lin SH, Chen PC, Chen YY. Degradation-aware neural imputation: Advancing decoding stability in brain machine interfaces. APL Bioeng 2025; 9:026106. [PMID: 40247859 PMCID: PMC12005879 DOI: 10.1063/5.0250296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2024] [Accepted: 04/08/2025] [Indexed: 04/19/2025] Open
Abstract
Neural signal degradation poses a significant challenge in maintaining stable performance when decoding motor tasks using multiunit activity (MUA) and local field potential (LFP) signals in the implantable brain machine interface (iBMI) applications. Effective methods for imputing degraded or missing signals are essential to restore neural signal integrity, thereby improving decoding accuracy and system robustness over long-term recordings with fluctuating signal quality. This study introduces a confidence-weighted Bayesian linear regression (CW-BLR) approach to impute neural signals affected by degradation, enhancing the robustness and consistency of decoding. The performance of CW-BLR was compared to traditional methods-mean imputation (Mean-imp) and Gaussian-mixture-model-based expectation-maximization (GMM-EM)-using a kernel-sliced inverse regression (kSIR) decoder to evaluate decoding outcomes. Four Wistar rats were trained to perform a forelimb-reaching task while neural activity (MUA and LFPs) was recorded over 27 days. CW-BLR imputed signals degraded during days 8-27. Decoding performance was evaluated using kSIR and compared with Mean-imp and GMM-EM. CW-BLR demonstrated superior performance by effectively preserving both temporal and spatial dependencies within the neural signals. CW-BLR-imputed data significantly improved decoding accuracy over traditional imputation methods, with the kSIR decoder showing consistently higher performance, particularly in maintaining signal quality from the degraded period. CW-BLR offers a robust and effective imputation framework for iBMI applications, addressing signal degradation challenges and maintaining accurate decoding over prolonged recordings. By utilizing confidence-based quality metrics, CW-BLR surpasses traditional methods, providing stable neural decoding across fluctuating signal quality scenarios.
Collapse
Affiliation(s)
- Yun-Ting Kuo
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2, Linong St., Taipei 112304, Taiwan
| | - Han-Lin Wang
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2, Linong St., Taipei 112304, Taiwan
| | - Bo-Wei Chen
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2, Linong St., Taipei 112304, Taiwan
| | | | - Yu-Chun Lo
- Ph.D. Program in Medical Neuroscience, College of Medical Science and Technology, Taipei Medical University, 12F, Education & Research Building, Shuang-Ho Campus, No. 301, Yuantong Rd., New Taipei City 23564, Taiwan
| | | | - Po-Chuan Chen
- Department of Mechanical Engineering, University of Minnesota, Minneapolis, Minnesota 55455, USA
| | - You-Yin Chen
- Author to whom correspondence should be addressed:
| |
Collapse
|
2
|
Li K, Liang H, Qiu J, Zhang X, Cai B, Wang D, Zhang D, Lin B, Han H, Yang G, Zhu Z. Reveal the mechanism of brain function with fluorescence microscopy at single-cell resolution: from neural decoding to encoding. J Neuroeng Rehabil 2025; 22:118. [PMID: 40426214 PMCID: PMC12107988 DOI: 10.1186/s12984-025-01655-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2025] [Accepted: 05/17/2025] [Indexed: 05/29/2025] Open
Abstract
As a key pathway for understanding behavior, cognition, and emotion, neural decoding and encoding provide effective tools to bridge the gap between neural mechanisms and imaging recordings, especially at single-cell resolution. While neural decoding aims to establish an interpretable theory of how complex biological behaviors are represented in neural activities, neural encoding focuses on manipulating behaviors through the stimulation of specific neurons. We thoroughly analyze the application of fluorescence imaging techniques, particularly two-photon fluorescence imaging, in decoding neural activities, showcasing the theoretical analysis and technological advancements from imaging recording to behavioral manipulation. For decoding models, we compared linear and nonlinear methods, including independent component analysis, random forests, and support vector machines, highlighting their capabilities to reveal the intricate mapping between neural activity and behavior. By employing synthetic stimuli via optogenetics, fundamental principles of neural encoding are further explored. We elucidate various encoding types based on different stimulus paradigms-quantity encoding, spatial encoding, temporal encoding, and frequency encoding-enhancing our understanding of how the brain represents and processes information. We believe that fluorescence imaging-based neural decoding and encoding techniques have deepened our understanding of the brain, and hold great potential in paving the way for future neuroscience research and clinical applications.
Collapse
Affiliation(s)
- Kangchen Li
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China
- Department of Critical Care Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Huanwei Liang
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China
- Department of Nephrology, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, China
| | - Jialing Qiu
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China
- Department of Hematology, The Third Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Xulan Zhang
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China
- Department of Gastroenterology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Bobo Cai
- Zhejiang Hospital, Hangzhou, China
| | - Depeng Wang
- College of Energy and Power Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Diming Zhang
- Research Center for Intelligent Sensing Systems, Zhejiang Laboratory, Hangzhou, China
| | - Bingzhi Lin
- College of Energy and Power Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Haijun Han
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China
| | - Geng Yang
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China.
| | - Zhijing Zhu
- Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, Institute of Brain and Cognitive Science, School of Medicine, Hangzhou City University, Hangzhou, China.
| |
Collapse
|
3
|
Zhang Y, Chen Y, Wang T, Cui H. Neural geometry from mixed sensorimotor selectivity for predictive sensorimotor control. eLife 2025; 13:RP100064. [PMID: 40310450 PMCID: PMC12045623 DOI: 10.7554/elife.100064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2025] Open
Abstract
Although recent studies suggest that activity in the motor cortex, in addition to generating motor outputs, receives substantial information regarding sensory inputs, it is still unclear how sensory context adjusts the motor commands. Here, we recorded population neural activity in the motor cortex via microelectrode arrays while monkeys performed flexible manual interceptions of moving targets. During this task, which requires predictive sensorimotor control, the activity of most neurons in the motor cortex encoding upcoming movements was influenced by ongoing target motion. Single-trial neural states at the movement onset formed staggered orbital geometries, suggesting that target motion modulates peri-movement activity in an orthogonal manner. This neural geometry was further evaluated with a representational model and recurrent neural networks (RNNs) with task-specific input-output mapping. We propose that the sensorimotor dynamics can be derived from neuronal mixed sensorimotor selectivity and dynamic interaction between modulations.
Collapse
Affiliation(s)
- Yiheng Zhang
- Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of SciencesShanghaiChina
- Chinese Institute for Brain ResearchBeijingChina
- University of Chinese Academy of SciencesBeijingChina
| | - Yun Chen
- Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of SciencesShanghaiChina
- Chinese Institute for Brain ResearchBeijingChina
- University of Chinese Academy of SciencesBeijingChina
| | - Tianwei Wang
- Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of SciencesShanghaiChina
- Chinese Institute for Brain ResearchBeijingChina
| | - He Cui
- Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of SciencesShanghaiChina
- Chinese Institute for Brain ResearchBeijingChina
| |
Collapse
|
4
|
Smoulder AL, Marino PJ, Oby ER, Snyder SE, Batista AP, Chase SM. Reward influences movement vigor through multiple motor cortical mechanisms. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.04.09.648001. [PMID: 40291660 PMCID: PMC12027334 DOI: 10.1101/2025.04.09.648001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/30/2025]
Abstract
The prospect of greater rewards often invigorates movements. What neural mechanisms support this increase of movement vigor for greater rewards? We had three rhesus monkeys perform reaching movements to targets worth different magnitudes of reward. We recorded neural population activity from primary motor and dorsal premotor cortex, brain areas at the output of cortical processing for voluntary movements, and asked how neural activity mediated the translation of reward into increased vigor. We identified features of neural activity during movement preparation, initiation, and execution that were both correlated with vigor and modulated by reward. We also found that the neural metrics that correlate with different aspects of movement vigor exhibit only limited correlation with one another, suggesting that there are multiple mechanisms through which reward modulates vigor. Finally, we note that the majority of reward's modulation of motor cortical activity cannot be accounted for by reward-mediated vigor differences in behavior, indicating that reward modulations within motor cortex may serve roles in addition to affecting vigor. Overall, our results provide insight into the neural mechanisms that link reward-driven motivation to the modulation of the details of movement.
Collapse
|
5
|
Qi Y, Zhu X, Xiong X, Yang X, Ding N, Wu H, Xu K, Zhu J, Zhang J, Wang Y. Human motor cortex encodes complex handwriting through a sequence of stable neural states. Nat Hum Behav 2025:10.1038/s41562-025-02157-x. [PMID: 40175631 DOI: 10.1038/s41562-025-02157-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2024] [Accepted: 02/28/2025] [Indexed: 04/04/2025]
Abstract
How the human motor cortex (MC) orchestrates sophisticated sequences of fine movements such as handwriting remains a puzzle. Here we investigate this question through Utah array recordings from human MC during attempted handwriting of Chinese characters (n = 306, each consisting of 6.3 ± 2.0 strokes). We find that MC activity evolves through a sequence of states corresponding to the writing of stroke fragments during complicated handwriting. The directional tuning curve of MC neurons remains stable within states, but its gain or preferred direction strongly varies across states. By building models that can automatically infer the neural states and implement state-dependent directional tuning, we can significantly better explain the firing pattern of individual neurons and reconstruct recognizable handwriting trajectories with 69% improvement compared with baseline models. Our findings unveil that skilled and sophisticated movements are encoded through state-specific neural configurations.
Collapse
Affiliation(s)
- Yu Qi
- Affiliated Mental Health Center and Hangzhou Seventh People's Hospital, MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China.
- NANHU Brain-Computer Interface Institute, Hangzhou, China.
- State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China.
- College of Computer Science, Zhejiang University, Hangzhou, China.
| | - Xinyun Zhu
- College of Computer Science, Zhejiang University, Hangzhou, China
| | - Xinzhu Xiong
- College of Computer Science, Zhejiang University, Hangzhou, China
| | - Xiaomeng Yang
- College of Computer Science, Zhejiang University, Hangzhou, China
| | - Nai Ding
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
| | - Hemmings Wu
- Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, China
| | - Kedi Xu
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
| | - Junming Zhu
- Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, China.
| | - Jianmin Zhang
- Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, China
| | - Yueming Wang
- NANHU Brain-Computer Interface Institute, Hangzhou, China.
- College of Computer Science, Zhejiang University, Hangzhou, China.
| |
Collapse
|
6
|
Moreno A, de Lafuente V, Merchant H. Time Varying Encoding of Grasping Type and Force in the Primate Motor Cortex. eNeuro 2025; 12:ENEURO.0010-25.2025. [PMID: 40246551 PMCID: PMC12037165 DOI: 10.1523/eneuro.0010-25.2025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2025] [Revised: 04/07/2025] [Accepted: 04/11/2025] [Indexed: 04/19/2025] Open
Abstract
The primary motor cortex (M1) is strongly engaged by movement planning and execution. However, the role of M1 activity in voluntary grasping is still not completely understood. Here we analyze recordings of M1 neurons during the execution of a delayed reach-to-grasp task, where monkeys had to actively grasp an object with either a side or a precision grip, and then pull it with a low or high amount of force. Single cell and neural populations analyses showed that grip type was robustly and specifically encoded by a large population of neurons, while force level was weakly and transiently encoded within mixed-selective neurons that also encoded grip type. Notably, the grip type was stably decoded from motor cortical populations during the preparation and execution epochs of the task. Our results are consistent with the idea that planning and performing specific grasping movements are high-level skills that strongly engage M1 neurons, while the execution of pulling force might be prominently encoded at lower stages of the motor system.
Collapse
Affiliation(s)
- Adriana Moreno
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro 76230, México
| | - Victor de Lafuente
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro 76230, México
| | - Hugo Merchant
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro 76230, México
| |
Collapse
|
7
|
Tessari F, West AM, Hogan N. Explaining human motor coordination via the synergy expansion hypothesis. Proc Natl Acad Sci U S A 2025; 122:e2501705122. [PMID: 40146855 PMCID: PMC12002196 DOI: 10.1073/pnas.2501705122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2025] [Accepted: 02/18/2025] [Indexed: 03/29/2025] Open
Abstract
The search for an answer to Bernstein's degrees of freedom problem has propelled a large portion of research studies in human motor control over the past six decades. Different theories have been developed to explain how humans might use their incredibly complex neuro-musculo-skeletal system with astonishing ease. Among these theories, motor synergies appeared as one possible explanation. In this work, the authors investigate the nature and role of synergies and propose a theoretical framework, namely the "expansion hypothesis," to answer Bernstein's problem. The expansion hypothesis is articulated in three propositions: mechanical, developmental, and behavioral. Each proposition addresses a different question on the nature of synergies: i) How many synergies can humans have? ii) How do we learn and develop synergies? iii) How do we use synergies? An example numerical simulation is presented and analyzed to clarify the hypothesis propositions. The expansion hypothesis is contextualized with respect to the existing literature on motor synergies both in healthy and impaired individuals, as well as other prominent theories in human motor control and development. The expansion hypothesis provides a framework to better comprehend and explain the nature, use, and evolution of human motor skills.
Collapse
Affiliation(s)
- Federico Tessari
- Newman Laboratory for Biomechanics and Human Rehabilitation, Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA02139
| | - A. Michael West
- Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD21218
| | - Neville Hogan
- Newman Laboratory for Biomechanics and Human Rehabilitation, Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA02139
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA02139
| |
Collapse
|
8
|
Alcolea PI, Ma X, Bodkin K, Miller LE, Danziger ZC. Less is more: selection from a small set of options improves BCI velocity control. J Neural Eng 2025; 22:10.1088/1741-2552/adbcd9. [PMID: 40043320 PMCID: PMC12051477 DOI: 10.1088/1741-2552/adbcd9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2024] [Accepted: 03/05/2025] [Indexed: 03/12/2025]
Abstract
Objective.Decoding algorithms used in invasive brain-computer interfaces (iBCIs) typically convert neural activity into continuously varying velocity commands. We hypothesized that putting constraints on which decoded velocity commands are permissible could improve user performance. To test this hypothesis, we designed the discrete direction selection (DDS) decoder, which uses neural activity to select among a small menu of preset cursor velocities.Approach. We tested DDS in a closed-loop cursor control task against many common continuous velocity decoders in both a human-operated real-time iBCI simulator (the jaBCI) and in a monkey using an iBCI. In the jaBCI, we compared performance across four visits by each of 48 naïve, able-bodied human subjects using either DDS, direct regression with assist (an affine map from neural activity to cursor velocity, DR-A), ReFIT, or the velocity Kalman Filter (vKF). In a follow up study to verify the jaBCI results, we compared a monkey's performance using an iBCI with either DDS or the Wiener filter decoder (a direct regression decoder that includes time history, WF).Main Result. In the jaBCI, DDS substantially outperformed all other decoders with 93% mean targets hit per visit compared to DR-A, ReFIT, and vKF with 56%, 39%, and 26% mean targets hit, respectively. With the iBCI, the monkey achieved a 61% success rate with DDS and a 37% success rate with WF.Significance. Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of discretization in simplifying online BCI control.
Collapse
Affiliation(s)
- Pedro I Alcolea
- Department of Biomedical Engineering, Florida International University, Miami, FL 33199, United States of America
| | - Xuan Ma
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, United States of America
| | - Kevin Bodkin
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, United States of America
| | - Lee E Miller
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, United States of America
- Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, United States of America
- Department of Biomedical Engineering, McCormick School of Engineering, Northwestern University, Evanston, IL 60208, United States of America
- Shirley Ryan AbilityLab, Chicago, IL 60611, United States of America
| | - Zachary C Danziger
- Department of Biomedical Engineering, Florida International University, Miami, FL 33199, United States of America
- Department of Rehabilitation Medicine—Division of Physical Therapy, Emory University, Atlanta, GA 30322, United States of America
- W.H. Coulter Department of Biomedical Engineering, Emory University Atlanta, Atlanta, GA 30322, United States of America
| |
Collapse
|
9
|
Mao H, Hasse BA, Schwartz AB. Hybrid Neural Network Models Explain Cortical Neuronal Activity During Volitional Movement. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.02.20.636945. [PMID: 40027649 PMCID: PMC11870545 DOI: 10.1101/2025.02.20.636945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Massive interconnectivity in large-scale neural networks is the key feature underlying their powerful and complex functionality. We have developed hybrid neural network (HNN) models that allow us to find statistical structure in this connectivity. Describing this structure is critical for understanding biological and artificial neural networks. The HNNs are composed of artificial neurons, a subset of which are trained to reproduce the responses of individual neurons recorded experimentally. The experimentally observed firing rates came from populations of neurons recorded in the motor cortices of monkeys performing a reaching task. After training, these networks (recurrent and spiking) underwent the same state transitions as those observed in the empirical data, a result that helps resolve a long-standing question of prescribed vs ongoing control of volitional movement. Because all aspects of the models are exposed, we were able to analyze the dynamic statistics of the connections between neurons. Our results show that the dynamics of extrinsic input to the network changed this connectivity to cause the state transitions. Two processes at the synaptic level were recognized: one in which many different neurons contributed to a buildup of membrane potential and another in which more specific neurons triggered an action potential. HNNs facilitate modeling of realistic neuron-neuron connectivity and provide foundational descriptions of large-scale network functionality.
Collapse
Affiliation(s)
- Hongwei Mao
- Department of Neurobiology, University of Pittsburgh School of Medicine
| | - Brady A. Hasse
- Department of Neurobiology, University of Pittsburgh School of Medicine
| | - Andrew B. Schwartz
- Department of Neurobiology, University of Pittsburgh School of Medicine
- Systems Neuroscience Center, University of Pittsburgh School of Medicine
- Department of Bioengineering, University of Pittsburgh School of Engineering
| |
Collapse
|
10
|
Agudelo-Toro A, Michaels JA, Sheng WA, Scherberger H. Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit. Neuron 2024; 112:4115-4129.e8. [PMID: 39419024 DOI: 10.1016/j.neuron.2024.09.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 03/15/2024] [Accepted: 09/19/2024] [Indexed: 10/19/2024]
Abstract
Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.
Collapse
Affiliation(s)
- Andres Agudelo-Toro
- Neurobiology Laboratory, Deutsches Primatenzentrum GmbH, Göttingen 37077, Germany.
| | - Jonathan A Michaels
- Neurobiology Laboratory, Deutsches Primatenzentrum GmbH, Göttingen 37077, Germany; School of Kinesiology and Health Science, Faculty of Health, York University, Toronto, ON M3J 1P3, Canada
| | - Wei-An Sheng
- Neurobiology Laboratory, Deutsches Primatenzentrum GmbH, Göttingen 37077, Germany; Institute of Biomedical Sciences, Academia Sinica, Taipei 115, Taiwan
| | - Hansjörg Scherberger
- Neurobiology Laboratory, Deutsches Primatenzentrum GmbH, Göttingen 37077, Germany; Faculty of Biology and Psychology, University of Göttingen, Göttingen 37073, Germany.
| |
Collapse
|
11
|
Haghi B, Aflalo T, Kellis S, Guan C, Gamez de Leon JA, Huang AY, Pouratian N, Andersen RA, Emami A. Enhanced control of a brain-computer interface by tetraplegic participants via neural-network-mediated feature extraction. Nat Biomed Eng 2024:10.1038/s41551-024-01297-1. [PMID: 39643728 DOI: 10.1038/s41551-024-01297-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 10/27/2024] [Indexed: 12/09/2024]
Abstract
To infer intent, brain-computer interfaces must extract features that accurately estimate neural activity. However, the degradation of signal quality over time hinders the use of feature-engineering techniques to recover functional information. By using neural data recorded from electrode arrays implanted in the cortices of three human participants, here we show that a convolutional neural network can be used to map electrical signals to neural features by jointly optimizing feature extraction and decoding under the constraint that all the electrodes must use the same neural-network parameters. In all three participants, the neural network led to offline and online performance improvements in a cursor-control task across all metrics, outperforming the rate of threshold crossings and wavelet decomposition of the broadband neural data (among other feature-extraction techniques). We also show that the trained neural network can be used without modification for new datasets, brain areas and participants.
Collapse
Affiliation(s)
- Benyamin Haghi
- Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA.
| | - Tyson Aflalo
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
| | - Spencer Kellis
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
- Neurostimulation Center and Neurosurgery, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
- Blackrock Microsystems, Salt Lake City, UT, USA
| | - Charles Guan
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Jorge A Gamez de Leon
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Albert Yan Huang
- Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Nader Pouratian
- UT Southern Medical Center, Dallas, TX, USA
- UCLA Neurosurgery, University of California Los Angeles, Los Angeles, CA, USA
| | - Richard A Andersen
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Azita Emami
- Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA.
- Andrew and Peggy Cherng Department of Medical Engineering, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
12
|
Chang YJ, Chen YI, Stealey HM, Zhao Y, Lu HY, Contreras-Hernandez E, Baker MN, Castillo E, Yeh HC, Santacruz SR. Multiscale effective connectivity analysis of brain activity using neural ordinary differential equations. PLoS One 2024; 19:e0314268. [PMID: 39630698 PMCID: PMC11616886 DOI: 10.1371/journal.pone.0314268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2024] [Accepted: 11/07/2024] [Indexed: 12/07/2024] Open
Abstract
Neural mechanisms and underlying directionality of signaling among brain regions depend on neural dynamics spanning multiple spatiotemporal scales of population activity. Despite recent advances in multimodal measurements of brain activity, there is no broadly accepted multiscale dynamical models for the collective activity represented in neural signals. Here we introduce a neurobiological-driven deep learning model, termed multiscale neural dynamics neural ordinary differential equation (msDyNODE), to describe multiscale brain communications governing cognition and behavior. We demonstrate that msDyNODE successfully captures multiscale activity using both simulations and electrophysiological experiments. The msDyNODE-derived causal interactions between recording locations and scales not only aligned well with the abstraction of the hierarchical neuroanatomy of the mammalian central nervous system but also exhibited behavioral dependences. This work offers a new approach for mechanistic multiscale studies of neural processes.
Collapse
Affiliation(s)
- Yin-Jui Chang
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Yuan-I Chen
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Hannah M. Stealey
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Yi Zhao
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Hung-Yun Lu
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | | | - Megan N. Baker
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Edward Castillo
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
| | - Hsin-Chih Yeh
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
- Texas Materials Institute, University of Texas at Austin, Austin, TX, United States of America
| | - Samantha R. Santacruz
- Biomedical Engineering, University of Texas at Austin, Austin, TX, United States of America
- Electrical & Computer Engineering, University of Texas at Austin, Austin, TX, United States of America
- Institute for Neuroscience, University of Texas at Austin, Austin, TX, United States of America
| |
Collapse
|
13
|
Park S, Lipton M, Dadarlat MC. Decoding multi-limb movements from two-photon calcium imaging of neuronal activity using deep learning. J Neural Eng 2024; 21:066006. [PMID: 39508456 DOI: 10.1088/1741-2552/ad83c0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Accepted: 09/26/2024] [Indexed: 11/15/2024]
Abstract
Objective.Brain-machine interfaces (BMIs) aim to restore sensorimotor function to individuals suffering from neural injury and disease. A critical step in implementing a BMI is to decode movement intention from recorded neural activity patterns in sensorimotor areas. Optical imaging, including two-photon (2p) calcium imaging, is an attractive approach for recording large-scale neural activity with high spatial resolution using a minimally-invasive technique. However, relating slow two-photon calcium imaging data to fast behaviors is challenging due to the relatively low optical imaging sampling rates. Nevertheless, neural activity recorded with 2p calcium imaging has been used to decode information about stereotyped single-limb movements and to control BMIs. Here, we expand upon prior work by applying deep learning to decode multi-limb movements of running mice from 2p calcium imaging data.Approach.We developed a recurrent encoder-decoder network (LSTM-encdec) in which the output is longer than the input.Main results.LSTM-encdec could accurately decode information about all four limbs (contralateral and ipsilateral front and hind limbs) from calcium imaging data recorded in a single cortical hemisphere.Significance.Our approach provides interpretability measures to validate decoding accuracy and expands the utility of BMIs by establishing the groundwork for control of multiple limbs. Our work contributes to the advancement of neural decoding techniques and the development of next-generation optical BMIs.
Collapse
Affiliation(s)
- Seungbin Park
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN 47906, United States of America
| | - Megan Lipton
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN 47906, United States of America
| | - Maria C Dadarlat
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN 47906, United States of America
| |
Collapse
|
14
|
Zhang H, Jiao L, Yang S, Li H, Jiang X, Feng J, Zou S, Xu Q, Gu J, Wang X, Wei B. Brain-computer interfaces: the innovative key to unlocking neurological conditions. Int J Surg 2024; 110:5745-5762. [PMID: 39166947 PMCID: PMC11392146 DOI: 10.1097/js9.0000000000002022] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Accepted: 07/30/2024] [Indexed: 08/23/2024]
Abstract
Neurological disorders such as Parkinson's disease, stroke, and spinal cord injury can pose significant threats to human mortality, morbidity, and functional independence. Brain-Computer Interface (BCI) technology, which facilitates direct communication between the brain and external devices, emerges as an innovative key to unlocking neurological conditions, demonstrating significant promise in this context. This comprehensive review uniquely synthesizes the latest advancements in BCI research across multiple neurological disorders, offering an interdisciplinary perspective on both clinical applications and emerging technologies. We explore the progress in BCI research and its applications in addressing various neurological conditions, with a particular focus on recent clinical studies and prospective developments. Initially, the review provides an up-to-date overview of BCI technology, encompassing its classification, operational principles, and prevalent paradigms. It then critically examines specific BCI applications in movement disorders, disorders of consciousness, cognitive and mental disorders, as well as sensory disorders, highlighting novel approaches and their potential impact on patient care. This review reveals emerging trends in BCI applications, such as the integration of artificial intelligence and the development of closed-loop systems, which represent significant advancements over previous technologies. The review concludes by discussing the prospects and directions of BCI technology, underscoring the need for interdisciplinary collaboration and ethical considerations. It emphasizes the importance of prioritizing bidirectional and high-performance BCIs, areas that have been underexplored in previous reviews. Additionally, we identify crucial gaps in current research, particularly in long-term clinical efficacy and the need for standardized protocols. The role of neurosurgery in spearheading the clinical translation of BCI research is highlighted. Our comprehensive analysis presents BCI technology as an innovative key to unlocking neurological disorders, offering a transformative approach to diagnosing, treating, and rehabilitating neurological conditions, with substantial potential to enhance patients' quality of life and advance the field of neurotechnology.
Collapse
Affiliation(s)
- Hongyu Zhang
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
- Harbin Medical University, Harbin
| | - Le Jiao
- Department of Neurosurgery, The First Hospital of Qiqihar, Qiqihar, Heilongjiang Province
| | | | | | | | - Jing Feng
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
- Harbin Medical University, Harbin
| | - Shuhuai Zou
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
- Harbin Medical University, Harbin
| | - Qiang Xu
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
- Harbin Medical University, Harbin
| | - Jianheng Gu
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
- Harbin Medical University, Harbin
| | - Xuefeng Wang
- Department of Neurosurgery, The Fourth Affiliated Hospital of Harbin Medical University
| | - Baojian Wei
- School of Nursing, Shandong First Medical University & Shandong Academy of Medical Sciences, Taian, Shandong, People's Republic of China
| |
Collapse
|
15
|
Li Y, Zhu X, Qi Y, Wang Y. Revealing unexpected complex encoding but simple decoding mechanisms in motor cortex via separating behaviorally relevant neural signals. eLife 2024; 12:RP87881. [PMID: 39120996 PMCID: PMC11315449 DOI: 10.7554/elife.87881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/11/2024] Open
Abstract
In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.
Collapse
Affiliation(s)
- Yangang Li
- Qiushi Academy for Advanced Studies, Zhejiang UniversityHangzhouChina
- Nanhu Brain-Computer Interface InstituteHangzhouChina
- College of Computer Science and Technology, Zhejiang UniversityHangzhouChina
- The State Key Lab of Brain-Machine Intelligence, Zhejiang UniversityHangzhouChina
| | - Xinyun Zhu
- Qiushi Academy for Advanced Studies, Zhejiang UniversityHangzhouChina
- Nanhu Brain-Computer Interface InstituteHangzhouChina
- College of Computer Science and Technology, Zhejiang UniversityHangzhouChina
- The State Key Lab of Brain-Machine Intelligence, Zhejiang UniversityHangzhouChina
| | - Yu Qi
- Nanhu Brain-Computer Interface InstituteHangzhouChina
- College of Computer Science and Technology, Zhejiang UniversityHangzhouChina
- The State Key Lab of Brain-Machine Intelligence, Zhejiang UniversityHangzhouChina
- Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital and the MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University School of MedicineHangzhouChina
| | - Yueming Wang
- Qiushi Academy for Advanced Studies, Zhejiang UniversityHangzhouChina
- Nanhu Brain-Computer Interface InstituteHangzhouChina
- College of Computer Science and Technology, Zhejiang UniversityHangzhouChina
- The State Key Lab of Brain-Machine Intelligence, Zhejiang UniversityHangzhouChina
- Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital and the MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University School of MedicineHangzhouChina
| |
Collapse
|
16
|
Schwartze KC, Lee WH, Rouse AG. Initial and corrective submovement encoding differences within primary motor cortex during precision reaching. J Neurophysiol 2024; 132:433-445. [PMID: 38985937 PMCID: PMC11427045 DOI: 10.1152/jn.00269.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 06/28/2024] [Accepted: 07/01/2024] [Indexed: 07/12/2024] Open
Abstract
Precision reaching often requires corrective submovements to obtain the desired goal. Most studies of reaching have focused on single initial movements, and implied the cortical encoding model was the same for all submovements. However, corrective submovements may show different encoding patterns from the initial submovement with distinct patterns of activation across the population. Two rhesus macaques performed a precision center-out-task with small targets. Neural activity from single units in the primary motor cortex and associated behavioral data were recorded to evaluate movement characteristics. Neural population data and individual neuronal firing rates identified with a peak finding algorithm to identify peaks in hand speed were examined for encoding differences between initial and corrective submovements. Individual neurons were fitted with a regression model that included the reach vector, position, and speed to predict firing rate. For both initial and corrective submovements, the largest effect remained movement direction. We observed a large subset changed their preferred direction greater than 45° between initial and corrective submovements. Neuronal depth of modulation also showed considerable variation when adjusted for movement speed. By using principal component analysis, neural trajectories of initial and corrective submovements progressed through different neural subspaces. These findings all suggest that different neural encoding patterns exist for initial and corrective submovements within the cortex. We hypothesize that this variation in how neurons change to encode small, corrective submovements might allow for a larger portion of the neural space being used to encode a greater range of movements with varying amplitudes and levels of precision.NEW & NOTEWORTHY Neuronal recordings matched with kinematic behavior were collected in a precision center-out task that often required corrective movements. We reveal large differences in preferred direction and depth of modulation between initial and corrective submovements across the neural population. We then present a model of the neural population describing how these shifts in tuning create different subspaces for signaling initial and corrective movements likely to improve motor precision.
Collapse
Affiliation(s)
- Kevin C Schwartze
- Department of Neurosurgery, University of Kansas Medical Center, Kansas City, Kansas, United States
- Department of Cell Biology and Physiology, University of Kansas Medical Center, Kansas City, Kansas, United States
| | - Wei-Hsien Lee
- Department of Neurosurgery, University of Kansas Medical Center, Kansas City, Kansas, United States
- Bioengineering Program, University of Kansas, Lawrence, Kansas, United States
| | - Adam G Rouse
- Department of Neurosurgery, University of Kansas Medical Center, Kansas City, Kansas, United States
- Department of Cell Biology and Physiology, University of Kansas Medical Center, Kansas City, Kansas, United States
- Bioengineering Program, University of Kansas, Lawrence, Kansas, United States
- Department of Electrical Engineering and Computer Science, University of Kansas, Lawrence, Kansas, United States
| |
Collapse
|
17
|
Jiang H, Bu X, Sui X, Tang H, Pan X, Chen Y. Spike Neural Network of Motor Cortex Model for Arm Reaching Control. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2024; 2024:1-4. [PMID: 40039622 DOI: 10.1109/embc53108.2024.10781802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/06/2025]
Abstract
Motor cortex modeling is crucial for understanding movement planning and execution. While interconnected recurrent neural networks have successfully described the dynamics of neural population activity, most existing methods utilize continuous signal-based neural networks, which do not reflect the biological spike neural signal. To address this limitation, we propose a recurrent spike neural network to simulate motor cortical activity during an arm-reaching task. Specifically, our model is built upon integrate-and-fire spiking neurons with conductance-based synapses. We carefully designed the interconnections of neurons with two different firing time scales - "fast" and "slow" neurons. Experimental results demonstrate the effectiveness of our method, with the model's neuronal activity in good agreement with monkey's motor cortex data at both single-cell and population levels. Quantitative analysis reveals a correlation coefficient 0.89 between the model's and real data. These results suggest the possibility of multiple timescales in motor cortical control.
Collapse
|
18
|
Alcolea P, Ma X, Bodkin K, Miller LE, Danziger ZC. Less is more: selection from a small set of options improves BCI velocity control. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.03.596241. [PMID: 38895473 PMCID: PMC11185569 DOI: 10.1101/2024.06.03.596241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
We designed the discrete direction selection (DDS) decoder for intracortical brain computer interface (iBCI) cursor control and showed that it outperformed currently used decoders in a human-operated real-time iBCI simulator and in monkey iBCI use. Unlike virtually all existing decoders that map between neural activity and continuous velocity commands, DDS uses neural activity to select among a small menu of preset cursor velocities. We compared closed-loop cursor control across four visits by each of 48 naïve, able-bodied human subjects using either DDS or one of three common continuous velocity decoders: direct regression with assist (an affine map from neural activity to cursor velocity), ReFIT, and the velocity Kalman Filter. DDS outperformed all three by a substantial margin. Subsequently, a monkey using an iBCI also had substantially better performance with DDS than with the Wiener filter decoder (direct regression decoder that includes time history). Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of simplifying online iBCI control.
Collapse
Affiliation(s)
- Pedro Alcolea
- Department of Biomedical Engineering, Florida International University, Miami, FL 33199, USA
| | - Xuan Ma
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Kevin Bodkin
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Lee E. Miller
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
- Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
- Department of Biomedical Engineering, McCormick School of Engineering, Northwestern University, Evanston, IL 60208, USA
- Shirley Ryan AbilityLab, Chicago, IL 60611, USA
| | - Zachary C. Danziger
- Department of Biomedical Engineering, Florida International University, Miami, FL 33199, USA
- Department of Rehabilitation Medicine - Division of Physical Therapy, Emory University, Atlanta, GA 30322, USA
- W.H. Coulter Department of Biomedical Engineering, Emory University Atlanta, GA 30322, USA
| |
Collapse
|
19
|
Rodriguez AC, Perich MG, Miller L, Humphries MD. Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.05.26.542452. [PMID: 37292834 PMCID: PMC10246015 DOI: 10.1101/2023.05.26.542452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.
Collapse
Affiliation(s)
| | - Matthew G. Perich
- Département de neurosciences, Faculté de médecine, Université de Montréal, Montréal, Canada
- Québec Artificial Intelligence Institute (Mila), Québec, Canada
| | - Lee Miller
- Northwestern University, Department of Biomedical Engineering, Chicago, USA
| | - Mark D. Humphries
- School of Psychology, University of Nottingham, Nottingham, United Kingdom
| |
Collapse
|
20
|
Tankus A, Rosenberg N, Ben-Hamo O, Stern E, Strauss I. Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces. J Neural Eng 2024; 21:036009. [PMID: 38648783 DOI: 10.1088/1741-2552/ad4179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 04/22/2024] [Indexed: 04/25/2024]
Abstract
Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding.Approach. We intraoperatively recorded single neuron activity in the left Vim of eight neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space.Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other.Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.
Collapse
Affiliation(s)
- Ariel Tankus
- Functional Neurosurgery Unit, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel
- Department of Neurology and Neurosurgery, School of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Noam Rosenberg
- School of Electrical Engineering, Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Oz Ben-Hamo
- School of Electrical Engineering, Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Einat Stern
- Department of Neurology and Neurosurgery, School of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Ido Strauss
- Functional Neurosurgery Unit, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel
- Department of Neurology and Neurosurgery, School of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
| |
Collapse
|
21
|
Wei 魏赣超 G, Tajik Mansouri زینب تاجیک منصوری Z, Wang 王晓婧 X, Stevenson IH. Calibrating Bayesian Decoders of Neural Spiking Activity. J Neurosci 2024; 44:e2158232024. [PMID: 38538143 PMCID: PMC11063820 DOI: 10.1523/jneurosci.2158-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 01/29/2024] [Accepted: 03/11/2024] [Indexed: 05/03/2024] Open
Abstract
Accurately decoding external variables from observations of neural activity is a major challenge in systems neuroscience. Bayesian decoders, which provide probabilistic estimates, are some of the most widely used. Here we show how, in many common settings, the probabilistic predictions made by traditional Bayesian decoders are overconfident. That is, the estimates for the decoded stimulus or movement variables are more certain than they should be. We then show how Bayesian decoding with latent variables, taking account of low-dimensional shared variability in the observations, can improve calibration, although additional correction for overconfidence is still needed. Using data from males, we examine (1) decoding the direction of grating stimuli from spike recordings in the primary visual cortex in monkeys, (2) decoding movement direction from recordings in the primary motor cortex in monkeys, (3) decoding natural images from multiregion recordings in mice, and (4) decoding position from hippocampal recordings in rats. For each setting, we characterize the overconfidence, and we describe a possible method to correct miscalibration post hoc. Properly calibrated Bayesian decoders may alter theoretical results on probabilistic population coding and lead to brain-machine interfaces that more accurately reflect confidence levels when identifying external variables.
Collapse
Affiliation(s)
- Ganchao Wei 魏赣超
- Department of Statistical Science, Duke University, Durham, North Carolina 27708
| | | | | | - Ian H Stevenson
- Departments of Biomedical Engineering, University of Connecticut, Storrs, Connecticut 06269
- Psychological Sciences, University of Connecticut, Storrs, Connecticut 06269
- Connecticut Institute for Brain and Cognitive Science, University of Connecticut, Storrs, Connecticut 06269
| |
Collapse
|
22
|
Wang T, Chen Y, Zhang Y, Cui H. Multiplicative joint coding in preparatory activity for reaching sequence in macaque motor cortex. Nat Commun 2024; 15:3153. [PMID: 38605030 PMCID: PMC11009282 DOI: 10.1038/s41467-024-47511-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 04/02/2024] [Indexed: 04/13/2024] Open
Abstract
Although the motor cortex has been found to be modulated by sensory or cognitive sequences, the linkage between multiple movement elements and sequence-related responses is not yet understood. Here, we recorded neuronal activity from the motor cortex with implanted micro-electrode arrays and single electrodes while monkeys performed a double-reach task that was instructed by simultaneously presented memorized cues. We found that there existed a substantial multiplicative component jointly tuned to impending and subsequent reaches during preparation, then the coding mechanism transferred to an additive manner during execution. This multiplicative joint coding, which also spontaneously emerged in recurrent neural networks trained for double reach, enriches neural patterns for sequential movement, and might explain the linear readout of elemental movements.
Collapse
Affiliation(s)
- Tianwei Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - Yun Chen
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - Yiheng Zhang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - He Cui
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.
- University of Chinese Academy of Sciences, Beijing, 100049, China.
- Chinese Institute for Brain Research, Beijing, 102206, China.
- Shanghai Center for Brain Science and Brain-inspired Technology, Shanghai, 200031, China.
| |
Collapse
|
23
|
Tessari F, Hermus J, Sugimoto-Dimitrova R, Hogan N. Brownian processes in human motor control support descending neural velocity commands. Sci Rep 2024; 14:8341. [PMID: 38594312 PMCID: PMC11004188 DOI: 10.1038/s41598-024-58380-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 03/28/2024] [Indexed: 04/11/2024] Open
Abstract
The motor neuroscience literature suggests that the central nervous system may encode some motor commands in terms of velocity. In this work, we tackle the question: what consequences would velocity commands produce at the behavioral level? Considering the ubiquitous presence of noise in the neuromusculoskeletal system, we predict that velocity commands affected by stationary noise would produce "random walks", also known as Brownian processes, in position. Brownian motions are distinctively characterized by a linearly growing variance and a power spectral density that declines in inverse proportion to frequency. This work first shows that these Brownian processes are indeed observed in unbounded motion tasks e.g., rotating a crank. We further predict that such growing variance would still be present, but bounded, in tasks requiring a constant posture e.g., maintaining a static hand position or quietly standing. This hypothesis was also confirmed by experimental observations. A series of descriptive models are investigated to justify the observed behavior. Interestingly, one of the models capable of accounting for all the experimental results must feature forward-path velocity commands corrupted by stationary noise. The results of this work provide behavioral support for the hypothesis that humans plan the motion components of their actions in terms of velocity.
Collapse
Affiliation(s)
- Federico Tessari
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA.
| | - James Hermus
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Rika Sugimoto-Dimitrova
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Neville Hogan
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
24
|
Khazali MF, Wong YT, Dean HL, Hagan MA, Fabiszak MM, Pesaran B. Putative cell-type-specific multiregional mode in posterior parietal cortex during coordinated visual behavior. Neuron 2023; 111:1979-1992.e7. [PMID: 37044088 PMCID: PMC10935574 DOI: 10.1016/j.neuron.2023.03.023] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Revised: 01/09/2023] [Accepted: 03/16/2023] [Indexed: 04/14/2023]
Abstract
In the reach and saccade regions of the posterior parietal cortex (PPC), multiregional communication depends on the timing of neuronal activity with respect to beta-frequency (10-30 Hz) local field potential (LFP) activity, termed dual coherence. Neural coherence is believed to reflect neural excitability, whereby spiking tends to occur at a particular phase of LFP activity, but the mechanisms of multiregional dual coherence remain unknown. Here, we investigate dual coherence in the PPC of non-human primates performing eye-hand movements. We computationally model dual coherence in terms of multiregional neural excitability and show that one latent component, a multiregional mode, reflects shared excitability across distributed PPC populations. Analyzing the power in the multiregional mode with respect to different putative cell types reveals significant modulations with the spiking of putative pyramidal neurons and not inhibitory interneurons. These results suggest a specific role for pyramidal neurons in dual coherence supporting multiregional communication in PPC.
Collapse
Affiliation(s)
- Mohammad Farhan Khazali
- Center for Neural Science, New York University, New York, NY 10003, USA; Freiburg Epilepsy Center, Medical Center - University of Freiburg, 79106 Freiburg, Germany
| | - Yan T Wong
- Department of Physiology and Neuroscience Program, Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia; Electrical and Computer Systems Engineering, Monash University, Clayton, VIC 3800, Australia
| | - Heather L Dean
- Center for Neural Science, New York University, New York, NY 10003, USA
| | - Maureen A Hagan
- Department of Physiology and Neuroscience Program, Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia
| | | | - Bijan Pesaran
- Center for Neural Science, New York University, New York, NY 10003, USA; Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 190104, USA; Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 190104, USA; Department of Bioengineering, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 190104, USA.
| |
Collapse
|
25
|
Guan C, Aflalo T, Kadlec K, Gámez de Leon J, Rosario ER, Bari A, Pouratian N, Andersen RA. Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex. J Neural Eng 2023; 20:036020. [PMID: 37160127 PMCID: PMC10209510 DOI: 10.1088/1741-2552/acd3b1] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 03/24/2023] [Accepted: 05/09/2023] [Indexed: 05/11/2023]
Abstract
Objective. Enable neural control of individual prosthetic fingers for participants with upper-limb paralysis.Approach. Two tetraplegic participants were each implanted with a 96-channel array in the left posterior parietal cortex (PPC). One of the participants was additionally implanted with a 96-channel array near the hand knob of the left motor cortex (MC). Across tens of sessions, we recorded neural activity while the participants attempted to move individual fingers of the right hand. Offline, we classified attempted finger movements from neural firing rates using linear discriminant analysis with cross-validation. The participants then used the neural classifier online to control individual fingers of a brain-machine interface (BMI). Finally, we characterized the neural representational geometry during individual finger movements of both hands.Main Results. The two participants achieved 86% and 92% online accuracy during BMI control of the contralateral fingers (chance = 17%). Offline, a linear decoder achieved ten-finger decoding accuracies of 70% and 66% using respective PPC recordings and 75% using MC recordings (chance = 10%). In MC and in one PPC array, a factorized code linked corresponding finger movements of the contralateral and ipsilateral hands.Significance. This is the first study to decode both contralateral and ipsilateral finger movements from PPC. Online BMI control of contralateral fingers exceeded that of previous finger BMIs. PPC and MC signals can be used to control individual prosthetic fingers, which may contribute to a hand restoration strategy for people with tetraplegia.
Collapse
Affiliation(s)
- Charles Guan
- California Institute of Technology, Pasadena, CA, United States of America
| | - Tyson Aflalo
- California Institute of Technology, Pasadena, CA, United States of America
- T&C Chen Brain-Machine Interface Center at Caltech, Pasadena, CA, United States of America
| | - Kelly Kadlec
- California Institute of Technology, Pasadena, CA, United States of America
| | | | - Emily R Rosario
- Casa Colina Hospital and Centers for Healthcare, Pomona, CA, United States of America
| | - Ausaf Bari
- David Geffen School of Medicine at UCLA, Los Angeles, CA, United States of America
| | - Nader Pouratian
- University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| | - Richard A Andersen
- California Institute of Technology, Pasadena, CA, United States of America
- T&C Chen Brain-Machine Interface Center at Caltech, Pasadena, CA, United States of America
| |
Collapse
|
26
|
Wan Z, Liu T, Ran X, Liu P, Chen W, Zhang S. The influence of non-stationarity of spike signals on decoding performance in intracortical brain-computer interface: a simulation study. Front Comput Neurosci 2023; 17:1135783. [PMID: 37251598 PMCID: PMC10213332 DOI: 10.3389/fncom.2023.1135783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2023] [Accepted: 04/28/2023] [Indexed: 05/31/2023] Open
Abstract
Introduction Intracortical Brain-Computer Interfaces (iBCI) establish a new pathway to restore motor functions in individuals with paralysis by interfacing directly with the brain to translate movement intention into action. However, the development of iBCI applications is hindered by the non-stationarity of neural signals induced by the recording degradation and neuronal property variance. Many iBCI decoders were developed to overcome this non-stationarity, but its effect on decoding performance remains largely unknown, posing a critical challenge for the practical application of iBCI. Methods To improve our understanding on the effect of non-stationarity, we conducted a 2D-cursor simulation study to examine the influence of various types of non-stationarities. Concentrating on spike signal changes in chronic intracortical recording, we used the following three metrics to simulate the non-stationarity: mean firing rate (MFR), number of isolated units (NIU), and neural preferred directions (PDs). MFR and NIU were decreased to simulate the recording degradation while PDs were changed to simulate the neuronal property variance. Performance evaluation based on simulation data was then conducted on three decoders and two different training schemes. Optimal Linear Estimation (OLE), Kalman Filter (KF), and Recurrent Neural Network (RNN) were implemented as decoders and trained using static and retrained schemes. Results In our evaluation, RNN decoder and retrained scheme showed consistent better performance under small recording degradation. However, the serious signal degradation would cause significant performance to drop eventually. On the other hand, RNN performs significantly better than the other two decoders in decoding simulated non-stationary spike signals, and the retrained scheme maintains the decoders' high performance when changes are limited to PDs. Discussion Our simulation work demonstrates the effects of neural signal non-stationarity on decoding performance and serves as a reference for selecting decoders and training schemes in chronic iBCI. Our result suggests that comparing to KF and OLE, RNN has better or equivalent performance using both training schemes. Performance of decoders under static scheme is influenced by recording degradation and neuronal property variation while decoders under retrained scheme are only influenced by the former one.
Collapse
Affiliation(s)
- Zijun Wan
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| | - Tengjun Liu
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| | - Xingchen Ran
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| | - Pengfu Liu
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| | - Weidong Chen
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| | - Shaomin Zhang
- Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Department of Biomedical Engineering, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| |
Collapse
|
27
|
Bachschmid-Romano L, Hatsopoulos NG, Brunel N. Interplay between external inputs and recurrent dynamics during movement preparation and execution in a network model of motor cortex. eLife 2023; 12:77690. [PMID: 37166452 PMCID: PMC10174693 DOI: 10.7554/elife.77690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 03/09/2023] [Indexed: 05/12/2023] Open
Abstract
The primary motor cortex has been shown to coordinate movement preparation and execution through computations in approximately orthogonal subspaces. The underlying network mechanisms, and the roles played by external and recurrent connectivity, are central open questions that need to be answered to understand the neural substrates of motor control. We develop a recurrent neural network model that recapitulates the temporal evolution of neuronal activity recorded from the primary motor cortex of a macaque monkey during an instructed delayed-reach task. In particular, it reproduces the observed dynamic patterns of covariation between neural activity and the direction of motion. We explore the hypothesis that the observed dynamics emerges from a synaptic connectivity structure that depends on the preferred directions of neurons in both preparatory and movement-related epochs, and we constrain the strength of both synaptic connectivity and external input parameters from data. While the model can reproduce neural activity for multiple combinations of the feedforward and recurrent connections, the solution that requires minimum external inputs is one where the observed patterns of covariance are shaped by external inputs during movement preparation, while they are dominated by strong direction-specific recurrent connectivity during movement execution. Our model also demonstrates that the way in which single-neuron tuning properties change over time can explain the level of orthogonality of preparatory and movement-related subspaces.
Collapse
Affiliation(s)
| | - Nicholas G Hatsopoulos
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, United States
- Committee on Computational Neuroscience, University of Chicago, Chicago, United States
| | - Nicolas Brunel
- Department of Neurobiology, Duke University, Durham, United States
- Department of Physics, Duke University, Durham, United States
- Duke Institute for Brain Sciences, Duke University, Durham, United States
- Center for Cognitive Neuroscience, Duke University, Durham, United States
| |
Collapse
|
28
|
Wang T, Chen Y, Cui H. From Parametric Representation to Dynamical System: Shifting Views of the Motor Cortex in Motor Control. Neurosci Bull 2022; 38:796-808. [PMID: 35298779 PMCID: PMC9276910 DOI: 10.1007/s12264-022-00832-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Accepted: 11/29/2021] [Indexed: 11/01/2022] Open
Abstract
In contrast to traditional representational perspectives in which the motor cortex is involved in motor control via neuronal preference for kinetics and kinematics, a dynamical system perspective emerging in the last decade views the motor cortex as a dynamical machine that generates motor commands by autonomous temporal evolution. In this review, we first look back at the history of the representational and dynamical perspectives and discuss their explanatory power and controversy from both empirical and computational points of view. Here, we aim to reconcile the above perspectives, and evaluate their theoretical impact, future direction, and potential applications in brain-machine interfaces.
Collapse
Affiliation(s)
- Tianwei Wang
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China.,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China.,University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Yun Chen
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China.,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China.,University of Chinese Academy of Sciences, Beijing, 100049, China
| | - He Cui
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China. .,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China. .,University of Chinese Academy of Sciences, Beijing, 100049, China.
| |
Collapse
|
29
|
Tankus A, Solomon L, Aharony Y, Faust-Socher A, Strauss I. Machine learning algorithm for decoding multiple subthalamic spike trains for speech brain-machine interfaces. J Neural Eng 2021; 18. [PMID: 34695815 DOI: 10.1088/1741-2552/ac3315] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2021] [Accepted: 10/25/2021] [Indexed: 11/11/2022]
Abstract
Objective. The goal of this study is to decode the electrical activity of single neurons in the human subthalamic nucleus (STN) to infer the speech features that a person articulated, heard or imagined. We also aim to evaluate the amount of subthalamic neurons required for high accuracy decoding suitable for real-life speech brain-machine interfaces (BMI).Approach. We intraoperatively recorded single-neuron activity in the STN of 21 neurosurgical patients with Parkinson's disease undergoing implantation of deep brain stimulator while patients produced, perceived or imagined the five monophthongal vowel sounds. Our decoder is based on machine learning algorithms that dynamically learn specific features of the speech-related firing patterns.Main results. In an extensive comparison of algorithms, our sparse decoder ('SpaDe'), based on sparse decomposition of the high dimensional neuronal feature space, outperformed the other algorithms in all three conditions: production, perception and imagery. For speech production, our algorithm, Spade, predicted all vowels correctly (accuracy: 100%; chance level: 20%). For perception accuracy was 96%, and for imagery: 88%. The accuracy of Spade showed a linear behavior in the amount of neurons for the perception data, and even faster for production or imagery.Significance. Our study demonstrates that the information encoded by single neurons in the STN about the production, perception and imagery of speech is suitable for high-accuracy decoding. It is therefore an important step towards BMIs for restoration of speech faculties that bears an enormous potential to alleviate the suffering of completely paralyzed ('locked-in') patients and allow them to communicate again with their environment. Moreover, our research indicates how many subthalamic neurons may be necessary to achieve each level of decoding accuracy, which is of supreme importance for a neurosurgeon planning the implantation of a speech BMI.
Collapse
Affiliation(s)
- Ariel Tankus
- Functional Neurosurgery Unit, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel.,Department of Neurology and Neurosurgery, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel.,Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Lior Solomon
- School of Electrical Engineering, Faculty of Engineering, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Yotam Aharony
- School of Electrical Engineering, Faculty of Engineering, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Achinoam Faust-Socher
- Movement Disorders Unit, Department of Neurology, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel
| | - Ido Strauss
- Functional Neurosurgery Unit, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel.,Department of Neurology and Neurosurgery, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
| |
Collapse
|
30
|
Matić A, Valerjev P, Gomez-Marin A. Hierarchical Control of Visually-Guided Movements in a 3D-Printed Robot Arm. Front Neurorobot 2021; 15:755723. [PMID: 34776921 PMCID: PMC8589028 DOI: 10.3389/fnbot.2021.755723] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Accepted: 10/08/2021] [Indexed: 11/13/2022] Open
Abstract
The control architecture guiding simple movements such as reaching toward a visual target remains an open problem. The nervous system needs to integrate different sensory modalities and coordinate multiple degrees of freedom in the human arm to achieve that goal. The challenge increases due to noise and transport delays in neural signals, non-linear and fatigable muscles as actuators, and unpredictable environmental disturbances. Here we examined the capabilities of hierarchical feedback control models proposed by W. T. Powers, so far only tested in silico. We built a robot arm system with four degrees of freedom, including a visual system for locating the planar position of the hand, joint angle proprioception, and pressure sensing in one point of contact. We subjected the robot to various human-inspired reaching and tracking tasks and found features of biological movement, such as isochrony and bell-shaped velocity profiles in straight-line movements, and the speed-curvature power law in curved movements. These behavioral properties emerge without trajectory planning or explicit optimization algorithms. We then applied static structural perturbations to the robot: we blocked the wrist joint, tilted the writing surface, extended the hand with a tool, and rotated the visual system. For all of them, we found that the arm in machina adapts its behavior without being reprogrammed. In sum, while limited in speed and precision (by the nature of the do-it-yourself inexpensive components we used to build the robot from scratch), when faced with the noise, delays, non-linearities, and unpredictable disturbances of the real world, the embodied control architecture shown here balances biological realism with design simplicity.
Collapse
Affiliation(s)
- Adam Matić
- Behavior of Organisms Laboratory, Instituto de Neurociencias CSIC-UMH, Alicante, Spain
| | - Pavle Valerjev
- Department of Psychology, University of Zadar, Zadar, Croatia
| | - Alex Gomez-Marin
- Behavior of Organisms Laboratory, Instituto de Neurociencias CSIC-UMH, Alicante, Spain
| |
Collapse
|
31
|
Johnson T, Taylor D. Improving reaching with functional electrical stimulation by incorporating stiffness modulation. J Neural Eng 2021; 18. [PMID: 34644693 DOI: 10.1088/1741-2552/ac2f7a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Accepted: 10/13/2021] [Indexed: 11/12/2022]
Abstract
Objective.Intracortical recordings have now been combined with functional electrical stimulation (FES) of arm/hand muscles to demonstrate restoration of upper-limb function after spinal cord injury. However, for each desired limb position decoded from the brain, there are multiple combinations of muscle stimulation levels that can produce that position. The objective of this simulation study is to explore how modulating the amount of coactivation of antagonist muscles during FES can impact reaching performance and energy usage. Stiffening the limb by cocontracting antagonist muscles makes the limb more resistant to perturbation. Minimizing cocontraction saves energy and reduces fatigue.Approach.Prior demonstrations of reaching via FES used a fixed empirically-derived lookup table for each joint that defined the muscle stimulation levels that would position the limb at the desired joint angle decoded from the brain at each timestep. This study expands on that previous work by using simulations to: (a) test the feasibility of controlling arm reaching using asuiteof lookup tables with varying levels of cocontraction instead of a single fixed lookup table for each joint, (b) optimize a simple function for automatically switching between these different cocontraction tables using only the desired kinematic information already being decoded from the brain, and (c) compare energy savings and movement performance when using the optimized function to automatically modulate cocontraction during reaching versus using the best fixed level of cocontraction.Main results.Our data suggests energy usage and/or movement performance can be significantly improved by dynamically modulating limb stiffness using our multi-table method and a simple function that determines cocontraction level based on decoded endpoint speed and its derivative.Significance.By demonstrating how modulating cocontraction can reduce energy usage while maintaining or even improving movement performance, this study makes brain-controlled FES a more viable option for restoration of reaching after paralysis.
Collapse
Affiliation(s)
- Tyler Johnson
- Cleveland Clinic, Cleveland, OH, United States of America.,Case Western Reserve University, Cleveland, OH, United States of America.,Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States of America
| | - Dawn Taylor
- Cleveland Clinic, Cleveland, OH, United States of America.,Case Western Reserve University, Cleveland, OH, United States of America.,Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States of America
| |
Collapse
|
32
|
Lopes-Dias C, Sburlea AI, Breitegger K, Wyss D, Drescher H, Wildburger R, Müller-Putz GR. Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier. J Neural Eng 2021; 18:046022. [PMID: 33779576 DOI: 10.1088/1741-2552/abd1eb] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
For brain-computer interface (BCI) users, the awareness of an error is associated with a cortical signature known as an error-related potential (ErrP). The incorporation of ErrP detection into BCIs can improve their performance. OBJECTIVE This work has three main aims. First, we investigate whether an ErrP classifier is transferable from able-bodied participants to participants with a spinal cord injury (SCI). Second, we test this generic ErrP classifier with SCI and control participants, in an online experiment without offline calibration. Third, we investigate the morphology of ErrPs in both groups of participants. APPROACH We used previously recorded electroencephalographic data from able-bodied participants to train an ErrP classifier. We tested the classifier asynchronously, in an online experiment with 16 new participants: 8 participants with SCI and 8 able-bodied control participants. The experiment had no offline calibration and participants received feedback regarding the ErrP detections from the start. To increase the fluidity of the experiment, feedback regarding false positive ErrP detections was not presented to the participants, but these detections were taken into account in the evaluation of the classifier. The generic classifier was not trained with the user's brain signals. However, its performance was optimized during the online experiment by the use of personalized decision thresholds. The classifier's performance was evaluated using trial-based metrics, which considered the asynchronous detection of ErrPs during the entire trial's duration. MAIN RESULTS Participants with SCI presented a non-homogenous ErrP morphology, and four of them did not present clear ErrP signals. The generic classifier performed better than chance in participants with clear ErrP signals, independently of the SCI (11 out of 16 participants). Three out of the five participants that obtained chance level results with the generic classifier would have not benefitted from the use of a personalized classifier. SIGNIFICANCE This work shows the feasibility of transferring an ErrP classifier from able-bodied participants to participants with SCI, for asynchronous detection of ErrPs in an online experiment without offline calibration, which provided immediate feedback to the users.
Collapse
Affiliation(s)
- Catarina Lopes-Dias
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | | | | | | | | | | | | |
Collapse
|
33
|
Sosnik R, Li Z. Reconstruction of hand, elbow and shoulder actual and imagined trajectories in 3D space using EEG current source dipoles. J Neural Eng 2021; 18. [PMID: 33752186 DOI: 10.1088/1741-2552/abf0d7] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2020] [Accepted: 03/22/2021] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Growing evidence suggests that EEG electrode (sensor) potential time series (PTS) of slow cortical potentials (SCPs) hold motor neural correlates that can be used for motion trajectory prediction (MTP), commonly by multiple linear regression (mLR). It is not yet known whether arm-joint trajectories can be reliably decoded from current sources, computed from sensor data, from which brain areas they can be decoded and using which neural features. APPROACH In this study, the PTS of 44 sensors were fed into sLORETA source localization software to compute current source activity in 30 regions of interest (ROIs) found in a recent meta-analysis to be engaged in action execution, motor imagery and motor preparation. The current sources PTS and band-power time series (BTS) in several frequency bands and time lags were used to predict actual and imagined trajectories in 3D space of the three velocity components of the hand, elbow and shoulder of nine subjects using an mLR model. MAIN RESULTS For all arm joints and movement types, current source SCPs PTS contributed most to trajectory reconstruction with time lags 150ms, 116ms and 84ms providing the highest contribution, and current source BTS in any of the tested frequency bands was not informative. Person's correlation coefficient (r) averaged across movement types, arm joints and velocity components using source data was slightly lower than using sensor data (r=0.25 and r=0.28, respectively). For each ROI, the three current source dipoles had different contribution to the reconstruction of each of the three velocity components. SIGNIFICANCE Overall, our results demonstrate the feasibility of predicting of actual and imagined 3D trajectories of all arm joints from current sources, computed from scalp EEG. These findings may be used by developers of a future BCI as a validated set of contributing ROIs.
Collapse
Affiliation(s)
- Ronen Sosnik
- Electrical, Electronics and Communication Engineering, Holon Institute of Technology, 52 Golomb St., Holon, 5810201, ISRAEL
| | - Zheng Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Yingdong Building, Xinjiekouwai Street 19, Beijing Haidian, Beijing, 100875, CHINA
| |
Collapse
|
34
|
Kobler RJ, Sburlea AI, Mondini V, Hirata M, Müller-Putz GR. Distance- and speed-informed kinematics decoding improves M/EEG based upper-limb movement decoder accuracy. J Neural Eng 2020; 17:056027. [PMID: 33146148 DOI: 10.1088/1741-2552/abb3b3] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g. velocity) and nondirectional (e.g. speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy. APPROACH In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters which did not combine both types of kinematics. MAIN RESULTS At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories. SIGNIFICANCE We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.
Collapse
Affiliation(s)
- Reinmar J Kobler
- Institute of Neural Engineering, Graz University of Technology, Graz 8010, Styria, Austria
| | | | | | | | | |
Collapse
|
35
|
Nguyen AT, Xu J, Jiang M, Luu DK, Wu T, Tam WK, Zhao W, Drealan MW, Overstreet CK, Zhao Q, Cheng J, Keefer E, Yang Z. A bioelectric neural interface towards intuitive prosthetic control for amputees. J Neural Eng 2020; 17. [PMID: 33091891 DOI: 10.1088/1741-2552/abc3d3] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2020] [Accepted: 10/22/2020] [Indexed: 01/17/2023]
Abstract
OBJECTIVE While prosthetic hands with independently actuated digits have become commercially available, state-of-the-art human-machine interfaces (HMI) only permit control over a limited set of grasp patterns, which does not enable amputees to experience sufficient improvement in their daily activities to make an active prosthesis useful. APPROACH Here we present a technology platform combining fully-integrated bioelectronics, implantable intrafascicular microelectrodes and deep learning-based artificial intelligence (AI) to facilitate this missing bridge by tapping into the intricate motor control signals of peripheral nerves. The bioelectric neural interface includes an ultra-low-noise neural recording system to sense electroneurography (ENG) signals from microelectrode arrays implanted in the residual nerves, and AI models employing the recurrent neural network (RNN) architecture to decode the subject's motor intention. MAIN RESULTS A pilot human study has been carried out on a transradial amputee. We demonstrate that the information channel established by the proposed neural interface is sufficient to provide high accuracy control of a prosthetic hand up to 15 degrees of freedom (DOF). The interface is intuitive as it directly maps complex prosthesis movements to the patient's true intention. SIGNIFICANCE Our study layouts the foundation towards not only a robust and dexterous control strategy for modern neuroprostheses at a near-natural level approaching that of the able hand, but also an intuitive conduit for connecting human minds and machines through the peripheral neural pathways. (Clinical trial identifier: NCT02994160).
Collapse
Affiliation(s)
- Anh Tuan Nguyen
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Jian Xu
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Ming Jiang
- Computer Science and Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Diu Khue Luu
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Tong Wu
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Wing-Kin Tam
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Wenfeng Zhao
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | - Markus W Drealan
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | | | - Qi Zhao
- Computer Science and Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| | | | | | - Zhi Yang
- Biomedical Engineering, University of Minnesota Twin Cities, Minneapolis, Minnesota, UNITED STATES
| |
Collapse
|
36
|
Suway SB, Schwartz AB. Activity in Primary Motor Cortex Related to Visual Feedback. Cell Rep 2020; 29:3872-3884.e4. [PMID: 31851920 DOI: 10.1016/j.celrep.2019.11.069] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2019] [Revised: 09/16/2019] [Accepted: 11/15/2019] [Indexed: 01/06/2023] Open
Abstract
Neural modulation in primate motor cortex exhibits complex patterns. We found that modulation during reaching could be separated into discrete temporal epochs. To determine if these epochs are driven by behavioral events, monkeys performed variations of a center-out reaching task. Monkeys viewed a computer cursor matched to hand position and a radial target at 1 of 16 locations. In some trials, they performed a visuomotor rotation (the cursor moved at an angle to the hand). After adaptation, encoding changes for single units are temporally structured: adaptation could affect one temporal component of a unit's response but not another. In half the normal and perturbed trials, we removed visual feedback before movement. Adaptation-sensitive firing components toward the end of movement are often weak or absent during reaches without feedback. These results show that temporal structure in motor cortical activity is driven by behavior, with a discrete component related to visual feedback.
Collapse
Affiliation(s)
- Steven B Suway
- Center for Neuroscience, University of Pittsburgh, Pittsburgh, PA 15260, USA; Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Andrew B Schwartz
- Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, PA 15213, USA; Systems Neuroscience Center, University of Pittsburgh, Pittsburgh, PA 15260, USA; Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA 15260, USA.
| |
Collapse
|
37
|
Abstract
Basic neurophysiological research with monkeys has shown how neurons in the motor cortex have firing rates tuned to movement direction. This original finding would have been difficult to uncover without the use of a behaving primate paradigm in which subjects grasped a handle and moved purposefully to targets in different directions. Subsequent research, again using behaving primate models, extended these findings to continuous drawing and to arm and hand movements encompassing action across multiple joints. This research also led to robust extraction algorithms in which information from neuronal populations is used to decode movement intent. The ability to decode intended movement provided the foundation for neural prosthetics in which brain-controlled interfaces are used by paralyzed human subjects to control computer cursors or high-performance motorized prosthetic arms and hands. This translation of neurophysiological laboratory findings to therapy is a clear example of why using nonhuman primates for basic research is valuable for advancing treatment of neurological disorders. Recent research emphasizes the distribution of intention signaling through neuronal populations and shows how many movement parameters are encoded simultaneously. In addition to direction and velocity, the arm's impedance has now been found to be encoded as well. The ability to decode motion and force from neural populations will make it possible to extend neural prosthetic paradigms to precise interaction with objects, enabling paralyzed individuals to perform many tasks of daily living.
Collapse
Affiliation(s)
- Scott D. Kennedy
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15260
- Center for the Neural Basis of Cognition, University of Pittsburgh and Carnegie Mellon University, Pittsburgh, PA 15261
- Systems Neuroscience Center, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213
| | - Andrew B. Schwartz
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15260
- Center for the Neural Basis of Cognition, University of Pittsburgh and Carnegie Mellon University, Pittsburgh, PA 15261
- Systems Neuroscience Center, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213
| |
Collapse
|
38
|
Willett FR, Young DR, Murphy BA, Memberg WD, Blabe CH, Pandarinath C, Stavisky SD, Rezaii P, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Bolu Ajiboye A. Principled BCI Decoder Design and Parameter Selection Using a Feedback Control Model. Sci Rep 2019; 9:8881. [PMID: 31222030 PMCID: PMC6586941 DOI: 10.1038/s41598-019-44166-7] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Accepted: 03/04/2019] [Indexed: 02/01/2023] Open
Abstract
Decoders optimized offline to reconstruct intended movements from neural recordings sometimes fail to achieve optimal performance online when they are used in closed-loop as part of an intracortical brain-computer interface (iBCI). This is because typical decoder calibration routines do not model the emergent interactions between the decoder, the user, and the task parameters (e.g. target size). Here, we investigated the feasibility of simulating online performance to better guide decoder parameter selection and design. Three participants in the BrainGate2 pilot clinical trial controlled a computer cursor using a linear velocity decoder under different gain (speed scaling) and temporal smoothing parameters and acquired targets with different radii and distances. We show that a user-specific iBCI feedback control model can predict how performance changes under these different decoder and task parameters in held-out data. We also used the model to optimize a nonlinear speed scaling function for the decoder. When used online with two participants, it increased the dynamic range of decoded speeds and decreased the time taken to acquire targets (compared to an optimized standard decoder). These results suggest that it is feasible to simulate iBCI performance accurately enough to be useful for quantitative decoder optimization and design.
Collapse
Affiliation(s)
- Francis R Willett
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA. .,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA. .,Department of Neurosurgery, Stanford University, Stanford, California, USA. .,Department of Electrical Engineering, Stanford University, Stanford, California, USA.
| | - Daniel R Young
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Brian A Murphy
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - William D Memberg
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Christine H Blabe
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Chethan Pandarinath
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Paymon Rezaii
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Jad Saab
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA
| | - Benjamin L Walter
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurology, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jennifer A Sweet
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jonathan P Miller
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA.,Department of Bioengineering, Stanford University, Stanford, California, 94305, USA.,Department of Neurobiology, Stanford University, Stanford, California, 94305, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, California, 94305, USA.,Neurosciences Program, Stanford University, Stanford, California, 94305, USA.,Bio-X Program, Stanford University, Stanford, California, 94305, USA
| | - John D Simeral
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Carney Institute for Brain Science, Brown University, Providence, Rhode Island, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Beata Jarosiewicz
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Leigh R Hochberg
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA.,Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
| | - Robert F Kirsch
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - A Bolu Ajiboye
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| |
Collapse
|