1
|
Sadras N, Pesaran B, Shanechi MM. Event detection and classification from multimodal time series with application to neural data. J Neural Eng 2024; 21:026049. [PMID: 38513289 DOI: 10.1088/1741-2552/ad3678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 03/21/2024] [Indexed: 03/23/2024]
Abstract
The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.
Collapse
Affiliation(s)
- Nitin Sadras
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Alfred E. Mann Department of Biomedical Engineering, and the Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
2
|
Combrisson E, Di Rienzo F, Saive AL, Perrone-Bertolotti M, Soto JLP, Kahane P, Lachaux JP, Guillot A, Jerbi K. Human local field potentials in motor and non-motor brain areas encode upcoming movement direction. Commun Biol 2024; 7:506. [PMID: 38678058 PMCID: PMC11055917 DOI: 10.1038/s42003-024-06151-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 04/05/2024] [Indexed: 04/29/2024] Open
Abstract
Limb movement direction can be inferred from local field potentials in motor cortex during movement execution. Yet, it remains unclear to what extent intended hand movements can be predicted from brain activity recorded during movement planning. Here, we set out to probe the directional-tuning of oscillatory features during motor planning and execution, using a machine learning framework on multi-site local field potentials (LFPs) in humans. We recorded intracranial EEG data from implanted epilepsy patients as they performed a four-direction delayed center-out motor task. Fronto-parietal LFP low-frequency power predicted hand-movement direction during planning while execution was largely mediated by higher frequency power and low-frequency phase in motor areas. By contrast, Phase-Amplitude Coupling showed uniform modulations across directions. Finally, multivariate classification led to an increase in overall decoding accuracy (>80%). The novel insights revealed here extend our understanding of the role of neural oscillations in encoding motor plans.
Collapse
Affiliation(s)
- Etienne Combrisson
- Psychology Department, University of Montreal, Montreal, QC, Canada.
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France.
- Institut de Neurosciences de la Timone, Aix Marseille Université, UMR 7289 CNRS, 13005, Marseille, France.
| | - Franck Di Rienzo
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France
| | - Anne-Lise Saive
- Psychology Department, University of Montreal, Montreal, QC, Canada
- Cognitive Science Department, Lyfe Research and Innovation Center, Ecully, France
| | | | - Juan L P Soto
- Telecommunications and Control Engineering Department, University of Sao Paulo, Sao Paulo, Brazil
| | - Philippe Kahane
- Université Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, GIN, Grenoble, France
| | - Jean-Philippe Lachaux
- Lyon Neuroscience Research Center, EDUWELL team, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, F-69000, Lyon, France
| | - Aymeric Guillot
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France
| | - Karim Jerbi
- Psychology Department, University of Montreal, Montreal, QC, Canada.
- Mila (Quebec AI Institute), montreal, QC, Canada.
- UNIQUE Centre (Quebec Neuro-AI research Center), Montreal, QC, Canada.
| |
Collapse
|
3
|
Merrick CM, Doyle ON, Gallegos NE, Irwin ZT, Olson JW, Gonzalez CL, Knight RT, Ivry RB, Walker HC. Differential contribution of sensorimotor cortex and subthalamic nucleus to unimanual and bimanual hand movements. Cereb Cortex 2024; 34:bhad492. [PMID: 38124548 PMCID: PMC10793582 DOI: 10.1093/cercor/bhad492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 11/18/2023] [Accepted: 11/19/2023] [Indexed: 12/23/2023] Open
Abstract
Why does unilateral deep brain stimulation improve motor function bilaterally? To address this clinical observation, we collected parallel neural recordings from sensorimotor cortex (SMC) and the subthalamic nucleus (STN) during repetitive ipsilateral, contralateral, and bilateral hand movements in patients with Parkinson's disease. We used a cross-validated electrode-wise encoding model to map electromyography data to the neural signals. Electrodes in the STN encoded movement at a comparable level for both hands, whereas SMC electrodes displayed a strong contralateral bias. To examine representational overlap across the two hands, we trained the model with data from one condition (contralateral hand) and used the trained weights to predict neural activity for movements produced with the other hand (ipsilateral hand). Overall, between-hand generalization was poor, and this limitation was evident in both regions. A similar method was used to probe representational overlap across different task contexts (unimanual vs. bimanual). Task context was more important for the STN compared to the SMC indicating that neural activity in the STN showed greater divergence between the unimanual and bimanual conditions. These results indicate that SMC activity is strongly lateralized and relatively context-free, whereas the STN integrates contextual information with the ongoing behavior.
Collapse
Affiliation(s)
- Christina M Merrick
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
| | - Owen N Doyle
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, United States
| | - Natali E Gallegos
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, United States
| | - Zachary T Irwin
- Department of Neurosurgery, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Joseph W Olson
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Christopher L Gonzalez
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Robert T Knight
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720, United States
| | - Richard B Ivry
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720, United States
| | - Harrison C Walker
- Department of Neurosurgery, University of Alabama at Birmingham, Birmingham, AL 35294, United States
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
- Department of Biomedical Engineering, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| |
Collapse
|
4
|
Schalk G, Shao S, Xiao K, Wu Z. Detection of common EEG phenomena using individual electrodes placed outside the hair. Biomed Phys Eng Express 2023; 10:015015. [PMID: 38055994 DOI: 10.1088/2057-1976/ad12f9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 12/06/2023] [Indexed: 12/08/2023]
Abstract
Many studies over the past decades have provided exciting evidence that electrical signals recorded from the scalp (electroencephalogram, EEG) hold meaningful information about the brain's function or dysfunction. This information is used routinely in research laboratories to test specific hypotheses and in clinical settings to aid in diagnoses (such as during polysomnography evaluations). Unfortunately, with very few exceptions, such meaningful information about brain function has not yet led to valuable solutions that can address the needs of many people outside such research laboratories or clinics. One of the major hurdles to practical application of EEG-based neurotechnologies is the current predominant requirement to use electrodes that are placed in the hair, which greatly reduces practicality and cosmesis. While several studies reported results using one specific combination of signal/reference electrode outside the hair in one specific context (such as a brain-computer interface experiment), it has been unclear what information about brain function can be acquired using different signal/referencing locations placed outside the hair. To address this issue, in this study, we set out to determine to what extent EEG phenomena related to auditory, visual, cognitive, motor, and sleep function can be detected from different combinations of individual signal/referencing electrodes that are placed outside the hair. The results of our study from 15 subjects suggest that only a few EEG electrodes placed in locations on the forehead or around the ear can provide substantial task-related information in 6 of 7 tasks. Thus, the results of our study provide encouraging evidence and guidance that should invigorate and facilitate the translation of laboratory experiments into practical, useful, and valuable EEG-based neurotechnology solutions.
Collapse
Affiliation(s)
- Gerwin Schalk
- Chen Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai, People's Republic of China
- Department of Neurosurgery, Huashan Hospital / Fudan University, Shanghai, People's Republic of China
| | - Shiyun Shao
- Chen Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai, People's Republic of China
| | - Kewei Xiao
- Chen Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai, People's Republic of China
| | - Zehan Wu
- Chen Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai, People's Republic of China
- Department of Neurosurgery, Huashan Hospital / Fudan University, Shanghai, People's Republic of China
| |
Collapse
|
5
|
Borra D, Mondini V, Magosso E, Müller-Putz GR. Decoding movement kinematics from EEG using an interpretable convolutional neural network. Comput Biol Med 2023; 165:107323. [PMID: 37619325 DOI: 10.1016/j.compbiomed.2023.107323] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 07/28/2023] [Accepted: 08/07/2023] [Indexed: 08/26/2023]
Abstract
Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.
Collapse
Affiliation(s)
- Davide Borra
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, Italy.
| | - Valeria Mondini
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Elisa Magosso
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, Italy; Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, Bologna, Italy
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria; BioTechMed, Graz, Austria
| |
Collapse
|
6
|
Qiao N, Ma L, Zhang Y, Wang L. Update on Nonhuman Primate Models of Brain Disease and Related Research Tools. Biomedicines 2023; 11:2516. [PMID: 37760957 PMCID: PMC10525665 DOI: 10.3390/biomedicines11092516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 08/19/2023] [Accepted: 09/06/2023] [Indexed: 09/29/2023] Open
Abstract
The aging of the population is an increasingly serious issue, and many age-related illnesses are on the rise. These illnesses pose a serious threat to the health and safety of elderly individuals and create a serious economic and social burden. Despite substantial research into the pathogenesis of these diseases, their etiology and pathogenesis remain unclear. In recent decades, rodent models have been used in attempts to elucidate these disorders, but such models fail to simulate the full range of symptoms. Nonhuman primates (NHPs) are the most ideal neuroscientific models for studying the human brain and are more functionally similar to humans because of their high genetic similarities and phenotypic characteristics in comparison with humans. Here, we review the literature examining typical NHP brain disease models, focusing on NHP models of common diseases such as dementia, Parkinson's disease, and epilepsy. We also explore the application of electroencephalography (EEG), magnetic resonance imaging (MRI), and optogenetic study methods on NHPs and neural circuits associated with cognitive impairment.
Collapse
Affiliation(s)
- Nan Qiao
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Lizhen Ma
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Yi Zhang
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Lifeng Wang
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| |
Collapse
|
7
|
Li G, Jiang S, Meng J, Wu Z, Jiang H, Fan Z, Hu J, Sheng X, Zhang D, Schalk G, Chen L, Zhu X. Spatio-temporal evolution of human neural activity during visually cued hand movements. Cereb Cortex 2023; 33:9764-9777. [PMID: 37464883 DOI: 10.1093/cercor/bhad242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 06/14/2023] [Accepted: 06/15/2023] [Indexed: 07/20/2023] Open
Abstract
Making hand movements in response to visual cues is common in daily life. It has been well known that this process activates multiple areas in the brain, but how these neural activations progress across space and time remains largely unknown. Taking advantage of intracranial electroencephalographic (iEEG) recordings using depth and subdural electrodes from 36 human subjects using the same task, we applied single-trial and cross-trial analyses to high-frequency iEEG activity. The results show that the neural activation was widely distributed across the human brain both within and on the surface of the brain, and focused specifically on certain areas in the parietal, frontal, and occipital lobes, where parietal lobes present significant left lateralization on the activation. We also demonstrate temporal differences across these brain regions. Finally, we evaluated the degree to which the timing of activity within these regions was related to sensory or motor function. The findings of this study promote the understanding of task-related neural processing of the human brain, and may provide important insights for translational applications.
Collapse
Affiliation(s)
- Guangye Li
- Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Shize Jiang
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Jianjun Meng
- Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Zehan Wu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Haiteng Jiang
- Department of Neurobiology, Affiliated Mental Health Center & Hangzhou Seventh People's Hospital, Zhejiang University School of Medicine, Hangzhou 310013, China
- MOE Frontier Science Center for Brain Science & Brain-Machine Integration, Zhejiang University, Hangzhou 310058, China
| | - Zhen Fan
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Jie Hu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Xinjun Sheng
- Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Dingguo Zhang
- Department of Electronic and Electrical Engineering, University of Bath, Bath BA2 7AY, United Kingdom
| | - Gerwin Schalk
- Chen Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai 200052, China
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Liang Chen
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Xiangyang Zhu
- Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| |
Collapse
|
8
|
Natraj N, Seko S, Abiri R, Yan H, Graham Y, Tu-Chan A, Chang EF, Ganguly K. Flexible regulation of representations on a drifting manifold enables long-term stable complex neuroprosthetic control. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.11.551770. [PMID: 37645922 PMCID: PMC10462094 DOI: 10.1101/2023.08.11.551770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
The nervous system needs to balance the stability of neural representations with plasticity. It is unclear what is the representational stability of simple actions, particularly those that are well-rehearsed in humans, and how it changes in new contexts. Using an electrocorticography brain-computer interface (BCI), we found that the mesoscale manifold and relative representational distances for a repertoire of simple imagined movements were remarkably stable. Interestingly, however, the manifold's absolute location demonstrated day-to-day drift. Strikingly, representational statistics, especially variance, could be flexibly regulated to increase discernability during BCI control without somatotopic changes. Discernability strengthened with practice and was specific to the BCI, demonstrating remarkable contextual specificity. Accounting for drift, and leveraging the flexibility of representations, allowed neuroprosthetic control of a robotic arm and hand for over 7 months without recalibration. Our study offers insight into how electrocorticography can both track representational statistics across long periods and allow long-term complex neuroprosthetic control.
Collapse
Affiliation(s)
- Nikhilesh Natraj
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| | - Sarah Seko
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| | - Reza Abiri
- Electrical, Computer and Biomedical Engineering, University of Rhode Island, Rhode Island, USA
| | - Hongyi Yan
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| | - Yasmin Graham
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| | - Adelyn Tu-Chan
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| | - Edward F Chang
- Department of Neurological Surgery, Weill Institute for Neuroscience, University of California-San Francisco, San Francisco, California, USA
| | - Karunesh Ganguly
- Dept. of Neurology, Weill Institute for Neurosciences, University of California San Francisco, San Francisco, California, USA
- UCSF - Veteran Affairs Medical Center, San Francisco, California, USA
| |
Collapse
|
9
|
Dreyer AM, Michalke L, Perry A, Chang EF, Lin JJ, Knight RT, Rieger JW. Grasp-specific high-frequency broadband mirror neuron activity during reach-and-grasp movements in humans. Cereb Cortex 2023; 33:6291-6298. [PMID: 36562997 PMCID: PMC10183732 DOI: 10.1093/cercor/bhac504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Revised: 11/30/2022] [Accepted: 12/01/2022] [Indexed: 12/24/2022] Open
Abstract
Broadly congruent mirror neurons, responding to any grasp movement, and strictly congruent mirror neurons, responding only to specific grasp movements, have been reported in single-cell studies with primates. Delineating grasp properties in humans is essential to understand the human mirror neuron system with implications for behavior and social cognition. We analyzed electrocorticography data from a natural reach-and-grasp movement observation and delayed imitation task with 3 different natural grasp types of everyday objects. We focused on the classification of grasp types from high-frequency broadband mirror activation patterns found in classic mirror system areas, including sensorimotor, supplementary motor, inferior frontal, and parietal cortices. Classification of grasp types was successful during movement observation and execution intervals but not during movement retention. Our grasp type classification from combined and single mirror electrodes provides evidence for grasp-congruent activity in the human mirror neuron system potentially arising from strictly congruent mirror neurons.
Collapse
Affiliation(s)
- Alexander M Dreyer
- Department of Psychology, Carl von Ossietzky University Oldenburg, Oldenburg 26129, Germany
| | - Leo Michalke
- Department of Psychology, Carl von Ossietzky University Oldenburg, Oldenburg 26129, Germany
| | - Anat Perry
- Department of Psychology, Hebrew University of Jerusalem, Jerusalem 91905, Israel
| | - Edward F Chang
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States
| | - Jack J Lin
- Department of Biomedical Engineering and the Comprehensive Epilepsy Program, Department of Neurology, University of California, Irvine, CA 92868, United States
| | - Robert T Knight
- Department of Psychology and the Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, United States
| | - Jochem W Rieger
- Department of Psychology, Carl von Ossietzky University Oldenburg, Oldenburg 26129, Germany
| |
Collapse
|
10
|
Rao N, Paek A, Contreras-Vidal JL, Parikh PJ. Lateralized Neural Entropy modulates with Grip Force during Precision Grasping. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.07.539751. [PMID: 37214821 PMCID: PMC10197571 DOI: 10.1101/2023.05.07.539751] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
When holding a coffee mug filled to the brim, we strive to avoid spilling the coffee. This ability relies on the neural processes underlying the control of finger forces on a moment-to-moment basis. The brain activity lateralized to the contralateral hemisphere averaged over a trial and across the trials is known to be associated with the magnitude of grip force applied on an object. However, the mechanistic involvement of the variability in neural signals during grip force control remains unclear. In this study, we examined the dependence of neural variability over the frontal, central, and parietal regions assessed using noninvasive electroencephalography (EEG) on grip force magnitude during an isometric force control task. We hypothesized laterally specific modulation in EEG variability with higher magnitude of the grip force exerted during grip force control. We utilized an existing EEG dataset (64 channel) comprised of healthy young adults, who performed an isometric force control task while receiving visual feedback of the force applied. The force magnitude to be exerted on the instrumented object was cued to participants during the task, and varied pseudorandomly among 5, 10, and 15% of their maximum voluntary contraction (MVC) across the trials. We quantified neural variability via sample entropy (sequence-dependent measure) and standard deviation (sequence-independent measure) of the temporal EEG signal over the frontal, central, and parietal electrodes. The EEG sample entropy over the central electrodes showed lateralized, nonlinear, localized, modulation with force magnitude. Similar modulation was not observed over frontal or parietal EEG activity, nor for standard deviation in the EEG activity. Our findings highlight specificity in neural control of grip forces by demonstrating the modulation in sequence-dependent but not sequence-independent component of EEG variability. This modulation appeared to be lateralized, spatially constrained, and functionally dependent on the grip force magnitude. We discuss the relevance of these findings in scenarios where a finer precision is essential to enable grasp application, such as prosthesis and associated neural signal integration, and propose directions for future studies investigating the mechanistic role of neural entropy in grip force control.
Collapse
|
11
|
Mintz Hemed N, Melosh NA. An integrated perspective for the diagnosis and therapy of neurodevelopmental disorders - From an engineering point of view. Adv Drug Deliv Rev 2023; 194:114723. [PMID: 36746077 DOI: 10.1016/j.addr.2023.114723] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 12/14/2022] [Accepted: 01/29/2023] [Indexed: 02/05/2023]
Abstract
Neurodevelopmental disorders (NDDs) are complex conditions with largely unknown pathophysiology. While many NDD symptoms are familiar, the cause of these disorders remains unclear and may involve a combination of genetic, biological, psychosocial, and environmental risk factors. Current diagnosis relies heavily on behaviorally defined criteria, which may be biased by the clinical team's professional and cultural expectations, thus a push for new biological-based biomarkers for NDDs diagnosis is underway. Emerging new research technologies offer an unprecedented view into the electrical, chemical, and physiological activity in the brain and with further development in humans may provide clinically relevant diagnoses. These could also be extended to new treatment options, which can start to address the underlying physiological issues. When combined with current speech, language, occupational therapy, and pharmacological treatment these could greatly improve patient outcomes. The current review will discuss the latest technologies that are being used or may be used for NDDs diagnosis and treatment. The aim is to provide an inspiring and forward-looking view for future research in the field.
Collapse
Affiliation(s)
- Nofar Mintz Hemed
- Department of Materials Science and Engineering, Stanford University, Stanford, CA 94305, USA.
| | - Nicholas A Melosh
- Department of Materials Science and Engineering, Stanford University, Stanford, CA 94305, USA
| |
Collapse
|
12
|
Ganguly K, Khanna P, Morecraft R, Lin DJ. Modulation of neural co-firing to enhance network transmission and improve motor function after stroke. Neuron 2022; 110:2363-2385. [PMID: 35926452 PMCID: PMC9366919 DOI: 10.1016/j.neuron.2022.06.024] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2022] [Revised: 05/15/2022] [Accepted: 06/28/2022] [Indexed: 01/28/2023]
Abstract
Stroke is a leading cause of disability. While neurotechnology has shown promise for improving upper limb recovery after stroke, efficacy in clinical trials has been variable. Our central thesis is that to improve clinical translation, we need to develop a common neurophysiological framework for understanding how neurotechnology alters network activity. Our perspective discusses principles for how motor networks, both healthy and those recovering from stroke, subserve reach-to-grasp movements. We focus on neural processing at the resolution of single movements, the timescale at which neurotechnologies are applied, and discuss how this activity might drive long-term plasticity. We propose that future studies should focus on cross-area communication and bridging our understanding of timescales ranging from single trials within a session to across multiple sessions. We hope that this perspective establishes a combined path forward for preclinical and clinical research with the goal of more robust clinical translation of neurotechnology.
Collapse
Affiliation(s)
- Karunesh Ganguly
- Department of Neurology, Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA,Neurology Service, SFVAHCS, San Francisco, CA, USA,
| | - Preeya Khanna
- Department of Neurology, Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA,Neurology Service, SFVAHCS, San Francisco, CA, USA
| | - Robert Morecraft
- Laboratory of Neurological Sciences, Division of Basic Biomedical Sciences, Sanford School of Medicine, The University of South Dakota, Vermillion, SD, 57069 USA
| | - David J. Lin
- Center for Neurotechnology and Neurorecovery, Division of Neurocritical Care and Emergency Neurology, Department of Neurology, Massachusetts General Hospital, Boston, MA,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI
| |
Collapse
|
13
|
Xie T, Wu Z, Schalk G, Tong Y, Vato A, Raviv N, Guo Q, Ye H, Sheng X, Zhu X, Brunner P, Chen L. Automated intraoperative central sulcus localization and somatotopic mapping using median nerve stimulation. J Neural Eng 2022; 19. [PMID: 35785769 PMCID: PMC9534515 DOI: 10.1088/1741-2552/ac7dfd] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 07/04/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Accurate identification of functional cortical regions is essential in neurological resection. The central sulcus (CS) is an important landmark that delineates functional cortical regions. Median nerve stimulation (MNS) is a standard procedure to identify the position of the CS intraoperatively. In this paper, we introduce an automated procedure that uses MNS to rapidly localize the CS and create functional somatotopic maps. APPROACH We recorded electrocorticographic signals from 13 patients who underwent MNS in the course of an awake craniotomy. We analyzed these signals to develop an automated procedure that determines the location of the CS and that also produces functional somatotopic maps. MAIN RESULTS The comparison between our automated method and visual inspection performed by the neurosurgeon shows that our procedure has a high sensitivity (89%) in identifying the CS. Further, we found substantial concordance between the functional somatotopic maps generated by our method and passive functional mapping (92% sensitivity). SIGNIFICANCE Our automated MNS-based method can rapidly localize the CS and create functional somatotopic maps without imposing additional burden on the clinical procedure. With additional development and validation, our method may lead to a diagnostic tool that guides neurosurgeon and reduces postoperative morbidity in patients undergoing resective brain surgery.
Collapse
Affiliation(s)
- Tao Xie
- Department of Neurosurgery, Washington University School of Medicine in Saint Louis, 660 S. Euclid Avenue, St Louis, Missouri, 63110-1010, UNITED STATES
| | - Zehan Wu
- Dept. of Neurosurgery, Huashan Hospital Fudan University, 12 Wulumuqi Middle Rd, Shanghai, 200040, CHINA
| | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, 113 Holland Avenue, Albany, New York, 12208, UNITED STATES
| | - Yusheng Tong
- Dept. of Neurosurgery, Huashan Hospital Fudan University, 12 Wulumuqi Middle Rd, Shanghai, 200040, CHINA
| | - Alessandro Vato
- National Center for Adaptive Neurotechnologies, 113 Holland Avenue, Albany, New York, 12208, UNITED STATES
| | - Nataly Raviv
- National Center for Adaptive Neurotechnologies, 113 Holland Avenue, Albany, New York, 12208, UNITED STATES
| | - Qinglong Guo
- Dept. of Neurosurgery, Huashan Hospital Fudan University, 12 Wulumuqi Middle Rd, Shanghai, 200040, CHINA
| | - Huanpeng Ye
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, CHINA
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, CHINA
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration , Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, CHINA
| | - Peter Brunner
- Department of Neurosurgery, Washington University School of Medicine in Saint Louis, 660 S. Euclid Avenue, St Louis, Missouri, 63110-1010, UNITED STATES
| | - Liang Chen
- Dept. of Neurosurgery, Huashan Hospital Fudan University, 12 Wulumuqi Middle Rd, Shanghai, 200040, CHINA
| |
Collapse
|
14
|
Ye H, Li G, Sheng X, Zhu X. Phase-amplitude coupling between low-frequency scalp EEG and high-frequency intracranial EEG during working memory task. J Neural Eng 2022; 19. [PMID: 35441594 DOI: 10.1088/1741-2552/ac63e9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 04/04/2022] [Indexed: 11/12/2022]
Abstract
Objective. Revealing the relationship between simultaneous scalp electroencephalography (EEG) and intracranial electroencephalography (iEEG) is of great importance for both neuroscientific research and translational applications. However, whether prominent iEEG features in the high-gamma band can be reflected by scalp EEG is largely unknown. To address this, we investigated the phase-amplitude coupling (PAC) phenomenon between the low-frequency band of scalp EEG and the high-gamma band of iEEG.Approach. We analyzed a simultaneous iEEG and scalp EEG dataset acquired under a verbal working memory paradigm from nine epilepsy subjects. The PAC values between pairs of scalp EEG channel and identified iEEG channel were explored. After identifying the frequency combinations and electrode locations that generated the most significant PAC values, we compared the PAC values of different task periods (encoding, maintenance, and retrieval) and memory loads.Main results. We demonstrated that the amplitude of high-gamma activities in the entorhinal cortex, hippocampus, and amygdala was correlated to the delta or theta phase at scalp locations such as Cz and Pz. In particular, the frequency bin that generated the maximum PAC value centered at 3.16-3.84 Hz for the phase and 50-85 Hz for the amplitude. Moreover, our results showed that PAC values for the retrieval period were significantly higher than those of the encoding and maintenance periods, and the PAC was also influenced by the memory load.Significance. This is the first human simultaneous iEEG and scalp EEG study demonstrating that the amplitude of iEEG high-gamma components is associated with the phase of low-frequency components in scalp EEG. These findings enhance our understanding of multiscale neural interactions during working memory, and meanwhile, provide a new perspective to estimate intracranial high-frequency features with non-invasive neural recordings.
Collapse
Affiliation(s)
- Huanpeng Ye
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Guangye Li
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
15
|
Hammer J, Schirrmeister RT, Hartmann K, Marusic P, Schulze-Bonhage A, Ball T. Interpretable functional specialization emerges in deep convolutional networks trained on brain signals. J Neural Eng 2022; 19. [PMID: 35421857 DOI: 10.1088/1741-2552/ac6770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 04/14/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Functional specialization is fundamental to neural information processing. Here, we study whether and how functional specialization emerges in artificial deep convolutional neural networks (CNNs) during a brain-computer interfacing (BCI) task. APPROACH We trained CNNs to predict hand movement speed from intracranial EEG (iEEG) and delineated how units across the different CNN hidden layers learned to represent the iEEG signal. MAIN RESULTS We show that distinct, functionally interpretable neural populations emerged as a result of the training process. While some units became sensitive to either iEEG amplitude or phase, others showed bimodal behavior with significant sensitivity to both features. Pruning of highly-sensitive units resulted in a steep drop of decoding accuracy not observed for pruning of less sensitive units, highlighting the functional relevance of the amplitude- and phase-specialized populations. SIGNIFICANCE We anticipate that emergent functional specialization as uncovered here will become a key concept in research towards interpretable deep learning for neuroscience and BCI applications.
Collapse
Affiliation(s)
- Jiri Hammer
- Neuromedical AI Lab, Department of Neurosurgery, University of Freiburg, Engelbergerstraße 21, Freiburg, 79106, GERMANY
| | | | - Kay Hartmann
- Neuromedical AI Lab, Department of Neurosurgery, University of Freiburg, Engelbergerstraße 21, Freiburg, 79106, GERMANY
| | - Petr Marusic
- Department of Neurology, Motol University Hospital, V Úvalu 84, Prague, 150 06, CZECH REPUBLIC
| | - Andreas Schulze-Bonhage
- Epilepsy Center, University Clinics, Albert-Ludwigs-Universitaet Freiburg, Albert-Ludwigs-University,, 79095 Freiburg, Germany, Freiburg, 79095, GERMANY
| | - Tonio Ball
- Epilepsy Center, University Clinics, Albert-Ludwigs-Universitaet Freiburg, Albert-Ludwigs-University,, 79095 Freiburg, Germany, Freiburg, 79106, GERMANY
| |
Collapse
|
16
|
Parmigiani S, Mikulan EP, Russo S, Sarasso S, Zauli FM, Rubino A, Cattani A, Fecchio M, Giampiccolo D, Lanzone J, D'Orio P, Del Vecchio M, Avanzini P, Nobili L, Sartori I, Massimini M, Pigorini A. Simultaneous stereo-EEG and high-density scalp EEG recordings to study the effects of intracerebral stimulation parameters. Brain Stimul 2022; 15:664-675. [PMID: 35421585 DOI: 10.1016/j.brs.2022.04.007] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 04/06/2022] [Accepted: 04/06/2022] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND Cortico-cortical evoked potentials (CCEPs) recorded by stereo-electroencephalography (SEEG) are a valuable tool to investigate brain reactivity and effective connectivity. However, invasive recordings are spatially sparse since they depend on clinical needs. This sparsity hampers systematic comparisons across-subjects, the detection of the whole-brain effects of intracortical stimulation, as well as their relationships to the EEG responses evoked by non-invasive stimuli. OBJECTIVE To demonstrate that CCEPs recorded by high-density electroencephalography (hd-EEG) provide additional information with respect SEEG alone and to provide an open, curated dataset to allow for further exploration of their potential. METHODS The dataset encompasses SEEG and hd-EEG recordings simultaneously acquired during Single Pulse Electrical Stimulation (SPES) in drug-resistant epileptic patients (N = 36) in whom stimulations were delivered with different physical, geometrical, and topological parameters. Differences in CCEPs were assessed by amplitude, latency, and spectral measures. RESULTS While invasively and non-invasively recorded CCEPs were generally correlated, differences in pulse duration, angle and stimulated cortical area were better captured by hd-EEG. Further, intracranial stimulation evoked site-specific hd-EEG responses that reproduced the spectral features of EEG responses to transcranial magnetic stimulation (TMS). Notably, SPES, albeit unperceived by subjects, elicited scalp responses that were up to one order of magnitude larger than the responses typically evoked by sensory stimulation in awake humans. CONCLUSIONS CCEPs can be simultaneously recorded with SEEG and hd-EEG and the latter provides a reliable descriptor of the effects of SPES as well as a common reference to compare the whole-brain effects of intracortical stimulation to those of non-invasive transcranial or sensory stimulations in humans.
Collapse
Affiliation(s)
- S Parmigiani
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy
| | - E P Mikulan
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy
| | - S Russo
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy; Department of Philosophy "Piero Martinetti", Università degli Studi di Milano, Milan, Italy
| | - S Sarasso
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy
| | - F M Zauli
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy; Department of Philosophy "Piero Martinetti", Università degli Studi di Milano, Milan, Italy
| | - A Rubino
- "C. Munari" Epilepsy Surgery Centre, Department of Neuroscience, Niguarda Hospital, Milan, Italy
| | - A Cattani
- Department of Mathematics & Statistics, Boston University, Boston, MA, USA
| | - M Fecchio
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | - D Giampiccolo
- Department of Neurosurgery, Department of Neuroscience, Biomedicine and Movement Sciences, University of Verona, Verona, Italy; Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, University College London, London, UK; Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK; Institute of Neurosciences, Cleveland Clinic London, London, UK
| | - J Lanzone
- Department of Systems Medicine, Neuroscience, University of Rome Tor Vergata, Rome, Italy; Istituti Clinici Scientifici Maugeri, IRCCS, Neurorehabilitation Department of Milano Institute, Milan, Italy
| | - P D'Orio
- "C. Munari" Epilepsy Surgery Centre, Department of Neuroscience, Niguarda Hospital, Milan, Italy; Istituto di Neuroscienze, Consiglio Nazionale delle Ricerche, Parma, Italy
| | - M Del Vecchio
- Istituto di Neuroscienze, Consiglio Nazionale delle Ricerche, Parma, Italy
| | - P Avanzini
- Istituto di Neuroscienze, Consiglio Nazionale delle Ricerche, Parma, Italy
| | - L Nobili
- Child Neuropsychiatry, IRCCS Istituto G. Gaslini, Genova, Italy
| | - I Sartori
- "C. Munari" Epilepsy Surgery Centre, Department of Neuroscience, Niguarda Hospital, Milan, Italy
| | - M Massimini
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy; Istituto Di Ricovero e Cura a Carattere Scientifico, Fondazione Don Carlo Gnocchi, Milan, Italy; Azrieli Program in Brain, Mind and Consciousness, Canadian Institute for Advanced Research, Toronto, Canada
| | - A Pigorini
- Department of Biomedical and Clinical Sciences "L. Sacco" Università degli Studi di Milano, Milan, Italy; Department of Biomedical, V, Università degli Studi di Milano, Milan, Italy.
| |
Collapse
|
17
|
Śliwowski M, Martin M, Souloumiac A, Blanchart P, Aksenova T. Decoding ECoG signal into 3D hand translation using deep learning. J Neural Eng 2022; 19. [PMID: 35287119 DOI: 10.1088/1741-2552/ac5d69] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 03/14/2022] [Indexed: 12/29/2022]
Abstract
Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.
Collapse
Affiliation(s)
- Maciej Śliwowski
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France.,Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France
| | - Matthieu Martin
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| | | | | | - Tetiana Aksenova
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| |
Collapse
|
18
|
Srisrisawang N, Müller-Putz GR. Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories. Front Hum Neurosci 2022; 16:830221. [PMID: 35399364 PMCID: PMC8988304 DOI: 10.3389/fnhum.2022.830221] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 02/28/2022] [Indexed: 11/13/2022] Open
Abstract
Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.
Collapse
Affiliation(s)
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed Graz, Graz, Austria
- *Correspondence: Gernot R. Müller-Putz,
| |
Collapse
|
19
|
Ye H, Fan Z, Li G, Wu Z, Hu J, Sheng X, Chen L, Zhu X. Spontaneous State Detection Using Time-Frequency and Time-Domain Features Extracted From Stereo-Electroencephalography Traces. Front Neurosci 2022; 16:818214. [PMID: 35368269 PMCID: PMC8968069 DOI: 10.3389/fnins.2022.818214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 02/15/2022] [Indexed: 11/23/2022] Open
Abstract
As a minimally invasive recording technique, stereo-electroencephalography (SEEG) measures intracranial signals directly by inserting depth electrodes shafts into the human brain, and thus can capture neural activities in both cortical layers and subcortical structures. Despite gradually increasing SEEG-based brain-computer interface (BCI) studies, the features utilized were usually confined to the amplitude of the event-related potential (ERP) or band power, and the decoding capabilities of other time-frequency and time-domain features have not been demonstrated for SEEG recordings yet. In this study, we aimed to verify the validity of time-domain and time-frequency features of SEEG, where classification performances served as evaluating indicators. To do this, using SEEG signals under intermittent auditory stimuli, we extracted features including the average amplitude, root mean square, slope of linear regression, and line-length from the ERP trace and three traces of band power activities (high-gamma, beta, and alpha). These features were used to detect the active state (including activations to two types of names) against the idle state. Results suggested that valid time-domain and time-frequency features distributed across multiple regions, including the temporal lobe, parietal lobe, and deeper structures such as the insula. Among all feature types, the average amplitude, root mean square, and line-length extracted from high-gamma (60–140 Hz) power and the line-length extracted from ERP were the most informative. Using a hidden Markov model (HMM), we could precisely detect the onset and the end of the active state with a sensitivity of 95.7 ± 1.3% and a precision of 91.7 ± 1.6%. The valid features derived from high-gamma power and ERP in this work provided new insights into the feature selection procedure for further SEEG-based BCI applications.
Collapse
Affiliation(s)
- Huanpeng Ye
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Zhen Fan
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Guangye Li
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Zehan Wu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Jie Hu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Liang Chen
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
- *Correspondence: Liang Chen
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
- Xiangyang Zhu
| |
Collapse
|
20
|
Moly A, Costecalde T, Martel F, Martin M, Larzabal C, Karakas S, Verney A, Charvet G, Chabardès S, Benabid AL, Aksenova T. An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic. J Neural Eng 2022; 19. [PMID: 35234665 DOI: 10.1088/1741-2552/ac59a0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 03/01/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The article aims at addressing 2 challenges to step motor BCI out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration. APPROACH Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based Recursive Exponentially Weighted Markov-Switching multi-Linear Model (REW-MSLM) decoder is proposed. REW-MSLM uses a Mixture of Expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a "gating" model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action. MAIN RESULTS Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of 6 months (without decoder recalibration) 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated. SIGNIFICANCE Based on the long-term (>36 months) chronic bilateral epidural ECoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behaviour (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.
Collapse
Affiliation(s)
- Alexandre Moly
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Thomas Costecalde
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Félix Martel
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Matthieu Martin
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des Martyrs, Grenoble, 38000, FRANCE
| | - Christelle Larzabal
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Serpil Karakas
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Alexandre Verney
- Université Paris-Saclay, Palaiseau, Palaiseau, Île-de-France, 91120, FRANCE
| | - Guillaume Charvet
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Stephan Chabardès
- CHU Grenoble Alpes, Boulevard de la Chantourne, La Tronche, Auvergne-Rhône-Alpes, 38700, FRANCE
| | - Alim-Louis Benabid
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, 17, avenue des Martyrs, Grenoble, 38000, FRANCE
| | - Tatiana Aksenova
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| |
Collapse
|
21
|
Kim HH, Jeong J. An electrocorticographic decoder for arm movement for brain–machine interface using an echo state network and Gaussian readout. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2021.108393] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
|
22
|
Bauernfeind G, Teschner MJ, Wriessnegger S, Büchner A, Lenarz T, Haumann S. Towards single-trial classification of invasively recorded auditory evoked potentials in cochlear implant users. J Neural Eng 2022; 19. [PMID: 35189612 DOI: 10.1088/1741-2552/ac572d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 02/21/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE One promising approach towards further improving cochlear implants (CIs) is to use brain signals controlling the device in order to close the auditory loop. Initial electroencephalography (EEG) studies have already shown promising results. However, they are based on noninvasive measurements, whereas implanted electrodes are expected to be more convenient in terms of everyday-life usability. If additional measurement electrodes were implanted during CI surgery, then invasive recordings should be possible. Furthermore, implantation will provide better signal quality, greater robustness to artefacts, and thus enhanced classification accuracy. APPROACH In an initial project, three additional epidural electrodes were temporarily implanted during the surgical procedure. After surgery, different auditory evoked potentials (AEPs) were recorded both invasively (epidural) and using surface electrodes, with invasively recorded signals demonstrated as being markedly superior. In this present analysis, cortical evoked response audiometry (CERA) signals recorded in seven patients were used for single-trial classification of sounds with different intensities. For classification purposes, we used shrinkage-regularized linear discriminant analysis (sLDA). Clinical speech perception scores were also investigated. MAIN RESULTS Analysis of CERA data from different subjects showed single-trial classification accuracies of up to 99.2% for perceived vs. non-perceived sounds. Accuracies of up to 89.1% were achieved in classification of sounds perceived at different intensities. Highest classification accuracies were achieved by means of epidural recordings. Required loudness differences seemed to correspond to speech perception in noise. Significance: The proposed epidural recording approach showed good classification accuracy into sound perceived and not perceived when the best-performing electrodes were selected. Classifying different levels of sound stimulation accurately proved more challenging. At present, the methods explored in this study would not be sufficiently reliable to allow automated closed-loop control of CI parameters. However, our findings are an important initial contribution towards improving applicability of closed auditory loops and for next-generation automatic fitting approaches.
Collapse
Affiliation(s)
- Guenther Bauernfeind
- Independent researcher; Former member of the Department of Otolaryngology, Hannover Medical School, Carl-Neuberg-Str. 1, Hannover, 30625, GERMANY
| | - Magnus Johannes Teschner
- Department of Otolaryngology, Hannover Medical School, Carl-Neuberg-Str. 1, Hannover, Niedersachsen, 30625, GERMANY
| | - Selina Wriessnegger
- Department of Psychology, Karl Franzens Universitaet Graz, Universitätsplatz 2 / III, A-8010 Graz, Austria, Graz, A-8010, AUSTRIA
| | - Andreas Büchner
- Department of Otolaryngology, Hannover Medical School, Carl-Neuberg-Str.1, Hannover, Niedersachsen, 30625, GERMANY
| | - Thomas Lenarz
- Department of Otolaryngology, Hannover Medical School, Carl-Neubergstr. 1, 30625 Hannover, Germany, Hannover, Niedersachsen, 30625, GERMANY
| | - Sabine Haumann
- Department of Otolaryngology, Hannover Medical School, Carl-Neuberg-Str.1, Hannover, Niedersachsen, 30625, GERMANY
| |
Collapse
|
23
|
Yao L, Zhu B, Shoaran M. Fast and accurate decoding of finger movements from ECoG through Riemannian features and modern machine learning techniques. J Neural Eng 2022; 19. [DOI: 10.1088/1741-2552/ac4ed1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Accepted: 01/25/2022] [Indexed: 11/11/2022]
Abstract
Abstract
Objective: Accurate decoding of individual finger movements is crucial for advanced prosthetic control. In this work, we introduce the use of Riemannian-space features and temporal dynamics of electrocorticography (ECoG) signal combined with modern machine learning tools to improve the motor decoding accuracy at the level of individual fingers. Approach: We selected a set of informative biomarkers that correlated with finger movements and evaluated the performance of state-of-the-art machine learning algorithms on the BCI competition IV dataset (ECoG, three subjects) and a second ECoG dataset with a similar recording paradigm (Stanford, 9 subjects). We further explored the temporal concatenation of features to effectively capture the history of ECoG signal, which led to a significant improvement over single-epoch decoding in both classification (p<0.01) and regression tasks (p<0.01). Main results: Using feature concatenation and gradient boosted trees (the top-performing model), we achieved a classification accuracy of 77.0% in detecting individual finger movements (6-class task, including rest state), improving over the state-of-the-art conditional random fields (CRF) by 11.7% on the 3 BCI competition subjects. In continuous decoding of movement trajectory, our approach resulted in an average Pearson's correlation coefficient (r) of 0.537 across subjects and fingers, outperforming both the BCI competition winner and the state-of-the-art approach reported on the same dataset (CNN+LSTM). Furthermore, our proposed method features a low time complexity, with only <17.2s required for training and <50ms for inference. This enables about 250× speed-up in training compared to previously reported deep learning method with state-of-the-art performance. Significance: The proposed techniques enable fast, reliable, and high-performance prosthetic control through minimally-invasive cortical signals.
Collapse
|
24
|
Kim H, Im CH. Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces. Front Neuroinform 2021; 15:750839. [PMID: 34744677 PMCID: PMC8569408 DOI: 10.3389/fninf.2021.750839] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Accepted: 09/21/2021] [Indexed: 11/13/2022] Open
Abstract
There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.
Collapse
Affiliation(s)
- Hodam Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea.,Department of Electronic Engineering, Hanyang University, Seoul, South Korea
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea.,Department of Electronic Engineering, Hanyang University, Seoul, South Korea.,Department of HY-KIST Bioconvergence, Hanyang University, Seoul, South Korea.,Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
| |
Collapse
|
25
|
Liu S, Li G, Jiang S, Wu X, Hu J, Zhang D, Chen L. Investigating Data Cleaning Methods to Improve Performance of Brain-Computer Interfaces Based on Stereo-Electroencephalography. Front Neurosci 2021; 15:725384. [PMID: 34690673 PMCID: PMC8528199 DOI: 10.3389/fnins.2021.725384] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Accepted: 09/01/2021] [Indexed: 11/13/2022] Open
Abstract
Stereo-electroencephalography (SEEG) utilizes localized and penetrating depth electrodes to directly measure electrophysiological brain activity. The implanted electrodes generally provide a sparse sampling of multiple brain regions, including both cortical and subcortical structures, making the SEEG neural recordings a potential source for the brain–computer interface (BCI) purpose in recent years. For SEEG signals, data cleaning is an essential preprocessing step in removing excessive noises for further analysis. However, little is known about what kinds of effect that different data cleaning methods may exert on BCI decoding performance and, moreover, what are the reasons causing the differentiated effects. To address these questions, we adopted five different data cleaning methods, including common average reference, gray–white matter reference, electrode shaft reference, bipolar reference, and Laplacian reference, to process the SEEG data and evaluated the effect of these methods on improving BCI decoding performance. Additionally, we also comparatively investigated the changes of SEEG signals induced by these different methods from multiple-domain (e.g., spatial, spectral, and temporal domain). The results showed that data cleaning methods could improve the accuracy of gesture decoding, where the Laplacian reference produced the best performance. Further analysis revealed that the superiority of the data cleaning method with excellent performance might be attributed to the increased distinguishability in the low-frequency band. The findings of this work highlighted the importance of applying proper data clean methods for SEEG signals and proposed the application of Laplacian reference for SEEG-based BCI.
Collapse
Affiliation(s)
- Shengjie Liu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Guangye Li
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Shize Jiang
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Xiaolong Wu
- Department of Electronic and Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Jie Hu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Dingguo Zhang
- Department of Electronic and Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Liang Chen
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| |
Collapse
|
26
|
Inferring entire spiking activity from local field potentials. Sci Rep 2021; 11:19045. [PMID: 34561480 PMCID: PMC8463692 DOI: 10.1038/s41598-021-98021-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2020] [Accepted: 09/01/2021] [Indexed: 11/29/2022] Open
Abstract
Extracellular recordings are typically analysed by separating them into two distinct signals: local field potentials (LFPs) and spikes. Previous studies have shown that spikes, in the form of single-unit activity (SUA) or multiunit activity (MUA), can be inferred solely from LFPs with moderately good accuracy. SUA and MUA are typically extracted via threshold-based technique which may not be reliable when the recordings exhibit a low signal-to-noise ratio (SNR). Another type of spiking activity, referred to as entire spiking activity (ESA), can be extracted by a threshold-less, fast, and automated technique and has led to better performance in several tasks. However, its relationship with the LFPs has not been investigated. In this study, we aim to address this issue by inferring ESA from LFPs intracortically recorded from the motor cortex area of three monkeys performing different tasks. Results from long-term recording sessions and across subjects revealed that ESA can be inferred from LFPs with good accuracy. On average, the inference performance of ESA was consistently and significantly higher than those of SUA and MUA. In addition, local motor potential (LMP) was found to be the most predictive feature. The overall results indicate that LFPs contain substantial information about spiking activity, particularly ESA. This could be useful for understanding LFP-spike relationship and for the development of LFP-based BMIs.
Collapse
|
27
|
Salahuddin U, Gao PX. Signal Generation, Acquisition, and Processing in Brain Machine Interfaces: A Unified Review. Front Neurosci 2021; 15:728178. [PMID: 34588951 PMCID: PMC8475516 DOI: 10.3389/fnins.2021.728178] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Accepted: 08/18/2021] [Indexed: 11/13/2022] Open
Abstract
Brain machine interfaces (BMIs), or brain computer interfaces (BCIs), are devices that act as a medium for communications between the brain and the computer. It is an emerging field with numerous applications in domains of prosthetic devices, robotics, communication technology, gaming, education, and security. It is noted in such a multidisciplinary field, many reviews have surveyed on various focused subfields of interest, such as neural signaling, microelectrode fabrication, and signal classification algorithms. A unified review is lacking to cover and link all the relevant areas in this field. Herein, this review intends to connect on the relevant areas that circumscribe BMIs to present a unified script that may help enhance our understanding of BMIs. Specifically, this article discusses signal generation within the cortex, signal acquisition using invasive, non-invasive, or hybrid techniques, and the signal processing domain. The latest development is surveyed in this field, particularly in the last decade, with discussions regarding the challenges and possible solutions to allow swift disruption of BMI products in the commercial market.
Collapse
Affiliation(s)
- Usman Salahuddin
- Institute of Materials Science, University of Connecticut, Storrs, CT, United States
| | - Pu-Xian Gao
- Institute of Materials Science, University of Connecticut, Storrs, CT, United States
- Department of Materials Science and Engineering, University of Connecticut, Storrs, CT, United States
| |
Collapse
|
28
|
Li G, Jiang S, Paraskevopoulou SE, Chai G, Wei Z, Liu S, Wang M, Xu Y, Fan Z, Wu Z, Chen L, Zhang D, Zhu X. Detection of human white matter activation and evaluation of its function in movement decoding using stereo-electroencephalography (SEEG). J Neural Eng 2021; 18. [PMID: 34284361 DOI: 10.1088/1741-2552/ac160e] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 07/20/2021] [Indexed: 11/11/2022]
Abstract
Objective. White matter tissue takes up approximately 50% of the human brain volume and it is widely known as a messenger conducting information between areas of the central nervous system. However, the characteristics of white matter neural activity and whether white matter neural recordings can contribute to movement decoding are often ignored and still remain largely unknown. In this work, we make quantitative analyses to investigate these two important questions using invasive neural recordings.Approach. We recorded stereo-electroencephalography (SEEG) data from 32 human subjects during a visually-cued motor task, where SEEG recordings can tap into gray and white matter electrical activity simultaneously. Using the proximal tissue density method, we identified the location (i.e. gray or white matter) of each SEEG contact. Focusing on alpha oscillatory and high gamma activities, we compared the activation patterns between gray matter and white matter. Then, we evaluated the performance of such white matter activation in movement decoding.Main results. The results show that white matter also presents activation under the task, in a similar way with the gray matter but at a significantly lower amplitude. Additionally, this work also demonstrates that combing white matter neural activities together with that of gray matter significantly promotes the movement decoding accuracy than using gray matter signals only.Significance. Taking advantage of SEEG recordings from a large number of subjects, we reveal the response characteristics of white matter neural signals under the task and demonstrate its enhancing function in movement decoding. This study highlights the importance of taking white matter activities into consideration in further scientific research and translational applications.
Collapse
Affiliation(s)
- Guangye Li
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China.,These authors contributed to this paper equally and should be considered as co-first authors
| | - Shize Jiang
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, People's Republic of China.,These authors contributed to this paper equally and should be considered as co-first authors
| | - Sivylla E Paraskevopoulou
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America.,These authors contributed to this paper equally and should be considered as co-first authors
| | - Guohong Chai
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zixuan Wei
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Shengjie Liu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Meng Wang
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Yang Xu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zhen Fan
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Zehan Wu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Liang Chen
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Dingguo Zhang
- Department of Electronic and Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
29
|
Frey M, Tanni S, Perrodin C, O'Leary A, Nau M, Kelly J, Banino A, Bendor D, Lefort J, Doeller CF, Barry C. Interpreting wide-band neural activity using convolutional neural networks. eLife 2021; 10:e66551. [PMID: 34338632 PMCID: PMC8328518 DOI: 10.7554/elife.66551] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Accepted: 07/13/2021] [Indexed: 11/29/2022] Open
Abstract
Rapid progress in technologies such as calcium imaging and electrophysiology has seen a dramatic increase in the size and extent of neural recordings. Even so, interpretation of this data requires considerable knowledge about the nature of the representation and often depends on manual operations. Decoding provides a means to infer the information content of such recordings but typically requires highly processed data and prior knowledge of the encoding scheme. Here, we developed a deep-learning framework able to decode sensory and behavioral variables directly from wide-band neural data. The network requires little user input and generalizes across stimuli, behaviors, brain regions, and recording techniques. Once trained, it can be analyzed to determine elements of the neural code that are informative about a given variable. We validated this approach using electrophysiological and calcium-imaging data from rodent auditory cortex and hippocampus as well as human electrocorticography (ECoG) data. We show successful decoding of finger movement, auditory stimuli, and spatial behaviors - including a novel representation of head direction - from raw neural activity.
Collapse
Affiliation(s)
- Markus Frey
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and TechnologyTrondheimNorway
- Max-Planck-Insitute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Sander Tanni
- Cell & Developmental Biology, UCLLondonUnited Kingdom
| | | | - Alice O'Leary
- Cell & Developmental Biology, UCLLondonUnited Kingdom
| | - Matthias Nau
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and TechnologyTrondheimNorway
- Max-Planck-Insitute for Human Cognitive and Brain SciencesLeipzigGermany
| | | | | | - Daniel Bendor
- Institute of Behavioural Neuroscience, UCLLondonUnited Kingdom
| | - Julie Lefort
- Cell & Developmental Biology, UCLLondonUnited Kingdom
| | - Christian F Doeller
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and TechnologyTrondheimNorway
- Max-Planck-Insitute for Human Cognitive and Brain SciencesLeipzigGermany
- Institute of Psychology, Leipzig UniversityLeipzigGermany
| | - Caswell Barry
- Cell & Developmental Biology, UCLLondonUnited Kingdom
| |
Collapse
|
30
|
Camarrone F, Branco MP, Ramsey NF, Van Hulle MM. Accurate Offline Asynchronous Detection of Individual Finger Movement From Intracranial Brain Signals Using a Novel Multiway Approach. IEEE Trans Biomed Eng 2021; 68:2176-2187. [PMID: 33186097 DOI: 10.1109/tbme.2020.3037934] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Asynchronous motor Brain Computer Interfacing (BCI) is characterized by the continuous decoding of intended muscular activity from brain signals. Such applications have gained widespread interest for enabling users to issue commands volitionally. In conventional motor BCIs features extracted from brain signals are concatenated into vector- or matrix-based (or one-/two-way) representations. Nevertheless, when accounting for the original multimodal or multiway signal structure, decoding performance has been shown to improve jointly with result interpretability. However, as multiway decoders are notorious for the extensive computational cost to train them, conventional ones are still preferred. To curb this limitation, we introduce a novel multiway classifier, called Block-Term Tensor Classifier that inherits the improved accuracy of multiway methods while providing fast training. We show that it can outperform state-of-the-art multiway and two-way Linear Discriminant Analysis classifiers in asynchronous detection of individual finger movements from intracranial recordings, an essential feature to achieve a sense of dexterity with hand prosthetics and exoskeletons.
Collapse
|
31
|
Ye H, Fan Z, Chai G, Li G, Wei Z, Hu J, Sheng X, Chen L, Zhu X. Self-Related Stimuli Decoding With Auditory and Visual Modalities Using Stereo-Electroencephalography. Front Neurosci 2021; 15:653965. [PMID: 34017235 PMCID: PMC8129191 DOI: 10.3389/fnins.2021.653965] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Accepted: 04/06/2021] [Indexed: 11/29/2022] Open
Abstract
Name recognition plays important role in self-related cognitive processes and also contributes to a variety of clinical applications, such as autism spectrum disorder diagnosis and consciousness disorder analysis. However, most previous name-related studies usually adopted noninvasive EEG or fMRI recordings, which were limited by low spatial resolution and temporal resolution, respectively, and thus millisecond-level response latencies in precise brain regions could not be measured using these noninvasive recordings. By invasive stereo-electroencephalography (SEEG) recordings that have high resolution in both the spatial and temporal domain, the current study distinguished the neural response to one's own name or a stranger's name, and explored common active brain regions in both auditory and visual modalities. The neural activities were classified using spatiotemporal features of high-gamma, beta, and alpha band. Results showed that different names could be decoded using multi-region SEEG signals, and the best classification performance was achieved at high gamma (60–145 Hz) band. In this case, auditory and visual modality-based name classification accuracies were 84.5 ± 8.3 and 79.9 ± 4.6%, respectively. Additionally, some single regions such as the supramarginal gyrus, middle temporal gyrus, and insula could also achieve remarkable accuracies for both modalities, supporting their roles in the processing of self-related information. The average latency of the difference between the two responses in these precise regions was 354 ± 63 and 285 ± 59 ms in the auditory and visual modality, respectively. This study suggested that name recognition was attributed to a distributed brain network, and the subsets with decoding capabilities might be potential implanted regions for awareness detection and cognition evaluation.
Collapse
Affiliation(s)
- Huanpeng Ye
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Zhen Fan
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Guohong Chai
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Guangye Li
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Zixuan Wei
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Jie Hu
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Liang Chen
- Department of Neurosurgery of Huashan Hospital, Fudan University, Shanghai, China
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
32
|
Singh SH, Peterson SM, Rao RPN, Brunton BW. Mining naturalistic human behaviors in long-term video and neural recordings. J Neurosci Methods 2021; 358:109199. [PMID: 33910024 DOI: 10.1016/j.jneumeth.2021.109199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Revised: 04/07/2021] [Accepted: 04/19/2021] [Indexed: 10/21/2022]
Abstract
BACKGROUND Recent technological advances in brain recording and machine learning algorithms are enabling the study of neural activity underlying spontaneous human behaviors, beyond the confines of cued, repeated trials. However, analyzing such unstructured data lacking a priori experimental design remains a significant challenge, especially when the data is multi-modal and long-term. NEW METHOD Here we describe an automated, behavior-first approach for analyzing simultaneously recorded long-term, naturalistic electrocorticography (ECoG) and behavior video data. We identify and characterize spontaneous human upper-limb movements by combining computer vision, discrete latent-variable modeling, and string pattern-matching on the video. RESULTS Our pipeline discovers and annotates over 40,000 instances of naturalistic arm movements in long term (7-9 day) behavioral videos, across 12 subjects. Analysis of the simultaneously recorded brain data reveals neural signatures of movement that corroborate previous findings. Our pipeline produces large training datasets for brain-computer interfacing applications, and we show decoding results from a movement initiation detection task. COMPARISON WITH EXISTING METHODS Spontaneous movements capture real-world neural and behavior variability that is missing from traditional cued tasks. Building beyond window-based movement detection metrics, our unsupervised discretization scheme produces a queryable pose representation, allowing localization of movements with finer temporal resolution. CONCLUSIONS Our work addresses the unique analytic challenges of studying naturalistic human behaviors and contributes methods that may generalize to other neural recording modalities beyond ECoG. We publish our curated dataset and believe that it will be a valuable resource for future studies of naturalistic movements.
Collapse
Affiliation(s)
- Satpreet H Singh
- Department of Electrical and Computer Engineering, University of Washington, Seattle, USA
| | - Steven M Peterson
- Department of Biology, University of Washington, Seattle, USA; eScience Institute, University of Washington, Seattle, USA
| | - Rajesh P N Rao
- Department of Electrical and Computer Engineering, University of Washington, Seattle, USA; Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, USA; Center for Neurotechnology, University of Washington, Seattle, USA; University of Washington Institute for Neuroengineering, Seattle, USA
| | - Bingni W Brunton
- Department of Biology, University of Washington, Seattle, USA; eScience Institute, University of Washington, Seattle, USA; University of Washington Institute for Neuroengineering, Seattle, USA.
| |
Collapse
|
33
|
Zaer H, Deshmukh A, Orlowski D, Fan W, Prouvot PH, Glud AN, Jensen MB, Worm ES, Lukacova S, Mikkelsen TW, Fitting LM, Adler JR, Schneider MB, Jensen MS, Fu Q, Go V, Morizio J, Sørensen JCH, Stroh A. An Intracortical Implantable Brain-Computer Interface for Telemetric Real-Time Recording and Manipulation of Neuronal Circuits for Closed-Loop Intervention. Front Hum Neurosci 2021; 15:618626. [PMID: 33613212 PMCID: PMC7887289 DOI: 10.3389/fnhum.2021.618626] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2020] [Accepted: 01/14/2021] [Indexed: 11/13/2022] Open
Abstract
Recording and manipulating neuronal ensemble activity is a key requirement in advanced neuromodulatory and behavior studies. Devices capable of both recording and manipulating neuronal activity brain-computer interfaces (BCIs) should ideally operate un-tethered and allow chronic longitudinal manipulations in the freely moving animal. In this study, we designed a new intracortical BCI feasible of telemetric recording and stimulating local gray and white matter of visual neural circuit after irradiation exposure. To increase the translational reliance, we put forward a Göttingen minipig model. The animal was stereotactically irradiated at the level of the visual cortex upon defining the target by a fused cerebral MRI and CT scan. A fully implantable neural telemetry system consisting of a 64 channel intracortical multielectrode array, a telemetry capsule, and an inductive rechargeable battery was then implanted into the visual cortex to record and manipulate local field potentials, and multi-unit activity. We achieved a 3-month stability of the functionality of the un-tethered BCI in terms of telemetric radio-communication, inductive battery charging, and device biocompatibility for 3 months. Finally, we could reliably record the local signature of sub- and suprathreshold neuronal activity in the visual cortex with high bandwidth without complications. The ability to wireless induction charging combined with the entirely implantable design, the rather high recording bandwidth, and the ability to record and stimulate simultaneously put forward a wireless BCI capable of long-term un-tethered real-time communication for causal preclinical circuit-based closed-loop interventions.
Collapse
Affiliation(s)
- Hamed Zaer
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Ashlesha Deshmukh
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, NC, United States
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University, Durham, NC, United States
| | - Dariusz Orlowski
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Wei Fan
- Leibniz Institute for Resilience Research, Mainz, Germany
| | | | - Andreas Nørgaard Glud
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Morten Bjørn Jensen
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Oncology, Radiation Therapy, and Clinical Medicine, Aarhus University Hospital, Aarhus University, Aarhus, Denmark
| | - Esben Schjødt Worm
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Oncology, Radiation Therapy, and Clinical Medicine, Aarhus University Hospital, Aarhus University, Aarhus, Denmark
| | - Slávka Lukacova
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Oncology, Radiation Therapy, and Clinical Medicine, Aarhus University Hospital, Aarhus University, Aarhus, Denmark
| | - Trine Werenberg Mikkelsen
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Lise Moberg Fitting
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - John R. Adler
- Zap Surgical Systems, Inc., San Carlos, CA, United States
- Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, United States
| | - M. Bret Schneider
- Zap Surgical Systems, Inc., San Carlos, CA, United States
- Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, United States
- Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, United States
| | - Martin Snejbjerg Jensen
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Nuclear Medicine and PET Center, Institute of Clinical Medicine, Aarhus University and Hospital, Aarhus, Denmark
| | - Quanhai Fu
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, NC, United States
| | - Vinson Go
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, NC, United States
| | - James Morizio
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, NC, United States
| | - Jens Christian Hedemann Sørensen
- Department of Neurosurgery, Center for Experimental Neuroscience (CENSE), Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Albrecht Stroh
- Leibniz Institute for Resilience Research, Mainz, Germany
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany
| |
Collapse
|
34
|
Modelling and prediction of the dynamic responses of large-scale brain networks during direct electrical stimulation. Nat Biomed Eng 2021; 5:324-345. [PMID: 33526909 DOI: 10.1038/s41551-020-00666-w] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Accepted: 11/24/2020] [Indexed: 01/19/2023]
Abstract
Direct electrical stimulation can modulate the activity of brain networks for the treatment of several neurological and neuropsychiatric disorders and for restoring lost function. However, precise neuromodulation in an individual requires the accurate modelling and prediction of the effects of stimulation on the activity of their large-scale brain networks. Here, we report the development of dynamic input-output models that predict multiregional dynamics of brain networks in response to temporally varying patterns of ongoing microstimulation. In experiments with two awake rhesus macaques, we show that the activities of brain networks are modulated by changes in both stimulation amplitude and frequency, that they exhibit damping and oscillatory response dynamics, and that variabilities in prediction accuracy and in estimated response strength across brain regions can be explained by an at-rest functional connectivity measure computed without stimulation. Input-output models of brain dynamics may enable precise neuromodulation for the treatment of disease and facilitate the investigation of the functional organization of large-scale brain networks.
Collapse
|
35
|
Li Y, Wang PT, Vaidya MP, Flint RD, Liu CY, Slutzky MW, Do AH. Electromyogram (EMG) Removal by Adding Sources of EMG (ERASE)-A Novel ICA-Based Algorithm for Removing Myoelectric Artifacts From EEG. Front Neurosci 2021; 14:597941. [PMID: 33584176 PMCID: PMC7873899 DOI: 10.3389/fnins.2020.597941] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2020] [Accepted: 12/11/2020] [Indexed: 12/01/2022] Open
Abstract
Electroencephalographic (EEG) recordings are often contaminated by electromyographic (EMG) artifacts, especially when recording during movement. Existing methods to remove EMG artifacts include independent component analysis (ICA), and other high-order statistical methods. However, these methods can not effectively remove most of EMG artifacts. Here, we proposed a modified ICA model for EMG artifacts removal in the EEG, which is called EMG Removal by Adding Sources of EMG (ERASE). In this new approach, additional channels of real EMG from neck and head muscles (reference artifacts) were added as inputs to ICA in order to “force” the most power from EMG artifacts into a few independent components (ICs). The ICs containing EMG artifacts (the “artifact ICs”) were identified and rejected using an automated procedure. ERASE was validated first using both simulated and experimentally-recorded EEG and EMG. Simulation results showed ERASE removed EMG artifacts from EEG significantly more effectively than conventional ICA. Also, it had a low false positive rate and high sensitivity. Subsequently, EEG was collected from 8 healthy participants while they moved their hands to test the realistic efficacy of this approach. Results showed that ERASE successfully removed EMG artifacts (on average, about 75% of EMG artifacts were removed when using real EMGs as reference artifacts) while preserving the expected EEG features related to movement. We also tested the ERASE procedure using simulated EMGs as reference artifacts (about 63% of EMG artifacts removed). Compared to conventional ICA, ERASE removed on average 26% more EMG artifacts from EEG. These findings suggest that ERASE can achieve significant separation of EEG signal and EMG artifacts without a loss of the underlying EEG features. These results indicate that using additional real or simulated EMG sources can increase the effectiveness of ICA in removing EMG artifacts from EEG. Combined with automated artifact IC rejection, ERASE also minimizes potential user bias. Future work will focus on improving ERASE so that it can also be used in real-time applications.
Collapse
Affiliation(s)
- Yongcheng Li
- Department of Neurology, University of California, Irvine, Irvine, CA, United States
| | - Po T Wang
- Department of Biomedical Engineering, University of California, Irvine, Irvine, CA, United States
| | - Mukta P Vaidya
- Department of Neurology, Northwestern University, Chicago, IL, United States.,Department of Physiology, Northwestern University, Chicago, IL, United States.,Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - Robert D Flint
- Department of Neurology, Northwestern University, Chicago, IL, United States.,Department of Physiology, Northwestern University, Chicago, IL, United States.,Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - Charles Y Liu
- Department of Neurosurgery, University of Southern California, Los Angeles, CA, United States.,Rancho Los Amigos National Rehabilitation Center, Downey, CA, United States.,Neurorestoration Center, University of Southern California, Los Angeles, CA, United States
| | - Marc W Slutzky
- Department of Neurology, Northwestern University, Chicago, IL, United States.,Department of Physiology, Northwestern University, Chicago, IL, United States.,Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - An H Do
- Department of Neurology, University of California, Irvine, Irvine, CA, United States
| |
Collapse
|
36
|
Dantas H, Hansen TC, Warren DJ, Mathews VJ. Shared Prosthetic Control Based on Multiple Movement Intent Decoders. IEEE Trans Biomed Eng 2020; 68:1547-1556. [PMID: 33326374 DOI: 10.1109/tbme.2020.3045351] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
SIGNIFICANCE A number of movement intent decoders exist in the literature that typically differ in the algorithms used and the nature of the outputs generated. Each approach comes with its own advantages and disadvantages. Combining the estimates of multiple algorithms may have better performance than any of the individual methods. OBJECTIVE This paper presents and evaluates a shared controller framework for prosthetic limbs based on multiple decoders of volitional movement intent. METHODS An algorithm to combine multiple estimates to control the prosthesis is developed in this paper. The capabilities of the approach are validated using a system that combines a Kalman filter-based decoder with a multilayer perceptron classifier-based decoder. The shared controller's performance is validated in online experiments where a virtual limb is controlled in real-time by amputee and intact-arm subjects. During the testing phase subjects controlled a virtual hand in real time to move digits to instructed positions using either a Kalman filter decoder, a multilayer perceptron decoder, or a linear combination of the two. RESULTS The shared controller results in statistically significant improvements over the component decoders. Specifically, certain degrees of shared control result in increases in the time-in-target metric and decreases in unintended movements. CONCLUSION The shared controller of this paper combines the good qualities of component decoders tested in this paper. Herein, combining a Kalman filter decoder with a classifier-based decoder inherits the flexibility of the Kalman filter decoder and the limited unwanted movements from the classifier-based decoder, resulting in a system that may be able to perform the tasks of everyday life more naturally and reliably.
Collapse
|
37
|
Ahmadi N, Constandinou T, Bouganis CS. Impact of referencing scheme on decoding performance of LFP-based brain-machine interface. J Neural Eng 2020; 18. [PMID: 33242850 DOI: 10.1088/1741-2552/abce3c] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2020] [Accepted: 11/26/2020] [Indexed: 12/19/2022]
Abstract
OBJECTIVE There has recently been an increasing interest in local field potential (LFP) for brain-machine interface (BMI) applications due to its desirable properties (signal stability and low bandwidth). LFP is typically recorded with respect to a single unipolar reference which is susceptible to common noise. Several referencing schemes have been proposed to eliminate the common noise, such as bipolar reference, current source density (CSD), and common average reference (CAR). However, to date, there have not been any studies to investigate the impact of these referencing schemes on decoding performance of LFP-based BMIs. APPROACH To address this issue, we comprehensively examined the impact of different referencing schemes and LFP features on the performance of hand kinematic decoding using a deep learning method. We used LFPs chronically recorded from the motor cortex area of a monkey while performing reaching tasks. MAIN RESULTS Experimental results revealed that local motor potential (LMP) emerged as the most informative feature regardless of the referencing schemes. Using LMP as the feature, CAR was found to yield consistently better decoding performance than other referencing schemes over long-term recording sessions. Significance Overall, our results suggest the potential use of LMP coupled with CAR for enhancing the decoding performance of LFP-based BMIs.
Collapse
Affiliation(s)
- Nur Ahmadi
- Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London, London, SW7 2AZ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Timothy Constandinou
- Electrical & Electronic Engineering, Imperial College London, London, London, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Christos-Savvas Bouganis
- Electrical and Electronic Engineering, Imperial College London, London, London, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
38
|
Li Y, Wang PT, Vaidya MP, Flint RD, Liu CY, Slutzky MW, Do AH. Refinement of High-Gamma EEG Features From TBI Patients With Hemicraniectomy Using an ICA Informed by Simulated Myoelectric Artifacts. Front Neurosci 2020; 14:599010. [PMID: 33328870 PMCID: PMC7732541 DOI: 10.3389/fnins.2020.599010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Accepted: 10/30/2020] [Indexed: 11/20/2022] Open
Abstract
Recent studies have shown the ability to record high-γ signals (80-160 Hz) in electroencephalogram (EEG) from traumatic brain injury (TBI) patients who have had hemicraniectomies. However, extraction of the movement-related high-γ remains challenging due to a confounding bandwidth overlap with surface electromyogram (EMG) artifacts related to facial and head movements. In our previous work, we described an augmented independent component analysis (ICA) approach for removal of EMG artifacts from EEG, and referred to as EMG Reduction by Adding Sources of EMG (ERASE). Here, we tested this algorithm on EEG recorded from six TBI patients with hemicraniectomies while they performed a thumb flexion task. ERASE removed a mean of 52 ± 12% (mean ± S.E.M) (maximum 73%) of EMG artifacts. In contrast, conventional ICA removed a mean of 27 ± 19% (mean ± S.E.M) of EMG artifacts from EEG. In particular, high-γ synchronization was significantly improved in the contralateral hand motor cortex area within the hemicraniectomy site after ERASE was applied. A more sophisticated measure of high-γ complexity is the fractal dimension (FD). Here, we computed the FD of EEG high-γ on each channel. Relative FD of high-γ was defined as that the FD in move state was subtracted by FD in idle state. We found relative FD of high-γ over hemicraniectomy after applying ERASE were strongly correlated to the amplitude of finger flexion force. Results showed that significant correlation coefficients across the electrodes related to thumb flexion averaged ~0.76, while the coefficients across the homologous electrodes in non-hemicraniectomy areas were nearly 0. After conventional ICA, a correlation between relative FD of high-γ and force remained high in both hemicraniectomy areas (up to 0.86) and non-hemicraniectomy areas (up to 0.81). Across all subjects, an average of 83% of electrodes significantly correlated with force was located in the hemicraniectomy areas after applying ERASE. After conventional ICA, only 19% of electrodes with significant correlations were located in the hemicraniectomy. These results indicated that the new approach isolated electrophysiological features during finger motor activation while selectively removing confounding EMG artifacts. This approach removed EMG artifacts that can contaminate high-gamma activity recorded over the hemicraniectomy.
Collapse
Affiliation(s)
- Yongcheng Li
- Department of Neurology, University of California, Irvine, Irvine, CA, United States
| | - Po T. Wang
- Department of Biomedical Engineering, University of California, Irvine, Irvine, CA, United States
| | - Mukta P. Vaidya
- Department of Neurology, Northwestern University, Chicago, IL, United States
- Department of Physiology, Northwestern University, Chicago, IL, United States
- Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - Robert D. Flint
- Department of Neurology, Northwestern University, Chicago, IL, United States
- Department of Physiology, Northwestern University, Chicago, IL, United States
- Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - Charles Y. Liu
- Department of Neurosurgery, University of Southern California, Los Angeles, CA, United States
- Rancho Los Amigos National Rehabilitation Center, Downey, CA, United States
- Neurorestoration Center, University of Southern California, Los Angeles, CA, United States
| | - Marc W. Slutzky
- Department of Neurology, Northwestern University, Chicago, IL, United States
- Department of Physiology, Northwestern University, Chicago, IL, United States
- Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - An H. Do
- Department of Neurology, University of California, Irvine, Irvine, CA, United States
| |
Collapse
|
39
|
Kobler RJ, Sburlea AI, Mondini V, Hirata M, Müller-Putz GR. Distance- and speed-informed kinematics decoding improves M/EEG based upper-limb movement decoder accuracy. J Neural Eng 2020; 17:056027. [PMID: 33146148 DOI: 10.1088/1741-2552/abb3b3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g. velocity) and nondirectional (e.g. speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy. APPROACH In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters which did not combine both types of kinematics. MAIN RESULTS At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories. SIGNIFICANCE We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.
Collapse
Affiliation(s)
- Reinmar J Kobler
- Institute of Neural Engineering, Graz University of Technology, Graz 8010, Styria, Austria
| | | | | | | | | |
Collapse
|
40
|
Mondini V, Kobler RJ, Sburlea AI, Müller-Putz GR. Continuous low-frequency EEG decoding of arm movement for closed-loop, natural control of a robotic arm. J Neural Eng 2020; 17:046031. [DOI: 10.1088/1741-2552/aba6f7] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
41
|
Khan MA, Das R, Iversen HK, Puthusserypady S. Review on motor imagery based BCI systems for upper limb post-stroke neurorehabilitation: From designing to application. Comput Biol Med 2020; 123:103843. [PMID: 32768038 DOI: 10.1016/j.compbiomed.2020.103843] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2020] [Revised: 05/18/2020] [Accepted: 06/02/2020] [Indexed: 12/21/2022]
Abstract
Strokes are a growing cause of mortality and many stroke survivors suffer from motor impairment as well as other types of disabilities in their daily life activities. To treat these sequelae, motor imagery (MI) based brain-computer interface (BCI) systems have shown potential to serve as an effective neurorehabilitation tool for post-stroke rehabilitation therapy. In this review, different MI-BCI based strategies, including "Functional Electric Stimulation, Robotics Assistance and Hybrid Virtual Reality based Models," have been comprehensively reported for upper-limb neurorehabilitation. Each of these approaches have been presented to illustrate the in-depth advantages and challenges of the respective BCI systems. Additionally, the current state-of-the-art and main concerns regarding BCI based post-stroke neurorehabilitation devices have also been discussed. Finally, recommendations for future developments have been proposed while discussing the BCI neurorehabilitation systems.
Collapse
Affiliation(s)
- Muhammad Ahmed Khan
- Department of Health Technology, Technical University of Denmark, 2800, Kgs. Lyngby, Denmark.
| | - Rig Das
- Department of Health Technology, Technical University of Denmark, 2800, Kgs. Lyngby, Denmark
| | - Helle K Iversen
- Department of Neurology, University of Copenhagen, Rigshospitalet, 2600, Glostrup, Denmark
| | | |
Collapse
|
42
|
Delgado Saa J, Christen A, Martin S, Pasley BN, Knight RT, Giraud AL. Using Coherence-based spectro-spatial filters for stimulus features prediction from electro-corticographic recordings. Sci Rep 2020; 10:7637. [PMID: 32376909 PMCID: PMC7203138 DOI: 10.1038/s41598-020-63303-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2019] [Accepted: 03/19/2020] [Indexed: 11/08/2022] Open
Abstract
The traditional approach in neuroscience relies on encoding models where brain responses are related to different stimuli in order to establish dependencies. In decoding tasks, on the contrary, brain responses are used to predict the stimuli, and traditionally, the signals are assumed stationary within trials, which is rarely the case for natural stimuli. We hypothesize that a decoding model assuming each experimental trial as a realization of a random process more likely reflects the statistical properties of the undergoing process compared to the assumption of stationarity. Here, we propose a Coherence-based spectro-spatial filter that allows for reconstructing stimulus features from brain signal's features. The proposed method extracts common patterns between features of the brain signals and the stimuli that produced them. These patterns, originating from different recording electrodes are combined, forming a spatial filter that produces a unified prediction of the presented stimulus. This approach takes into account frequency, phase, and spatial distribution of brain features, hence avoiding the need to predefine specific frequency bands of interest or phase relationships between stimulus and brain responses manually. Furthermore, the model does not require the tuning of hyper-parameters, reducing significantly the computational load attached to it. Using three different cognitive tasks (motor movements, speech perception, and speech production), we show that the proposed method consistently improves stimulus feature predictions in terms of correlation (group averages of 0.74 for motor movements, 0.84 for speech perception, and 0.74 for speech production) in comparison with other methods based on regularized multivariate regression, probabilistic graphical models and artificial neural networks. Furthermore, the model parameters revealed those anatomical regions and spectral components that were discriminant in the different cognitive tasks. This novel method does not only provide a useful tool to address fundamental neuroscience questions, but could also be applied to neuroprosthetics.
Collapse
Affiliation(s)
- Jaime Delgado Saa
- Auditory Language Group, University of Geneva, Geneva, Switzerland.
- BSPAI Lab, Universidad del Norte, Barranquilla, Colombia.
| | - Andy Christen
- Auditory Language Group, University of Geneva, Geneva, Switzerland
| | - Stephanie Martin
- Auditory Language Group, University of Geneva, Geneva, Switzerland
| | - Brian N Pasley
- Knight Lab, University of California at Berkeley, Berkeley, USA
| | - Robert T Knight
- Knight Lab, University of California at Berkeley, Berkeley, USA
| | - Anne-Lise Giraud
- Auditory Language Group, University of Geneva, Geneva, Switzerland
| |
Collapse
|
43
|
Farrokhi B, Erfanian A. A state-based probabilistic method for decoding hand position during movement from ECoG signals in non-human primate. J Neural Eng 2020; 17:026042. [PMID: 32224511 DOI: 10.1088/1741-2552/ab848b] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
OBJECTIVE In this study, we proposed a state-based probabilistic method for decoding hand positions during unilateral and bilateral movements using the ECoG signals recorded from the brain of Rhesus monkey. APPROACH A customized electrode array was implanted subdurally in the right hemisphere of the brain covering from the primary motor cortex to the frontal cortex. Three different experimental paradigms were considered: ipsilateral, contralateral, and bilateral movements. During unilateral movement, the monkey was trained to get food with one hand, while during bilateral movement, the monkey used its left and right hands alternately to get food. To estimate the hand positions, a state-based probabilistic method was introduced which was based on the conditional probability of the hand movement state (i.e. idle, right hand movement, and left hand movement) and the conditional expectation of the hand position for each state. Moreover, a hybrid feature extraction method based on linear discriminant analysis and partial least squares (PLS) was introduced. MAIN RESULTS The proposed method could successfully decode the hand positions during ipsilateral, contralateral, and bilateral movements and significantly improved the decoding performance compared to the conventional Kalman and PLS regression methods [Formula: see text]. The proposed hybrid feature extraction method was found to outperform both the PLS and PCA methods [Formula: see text]. Investigating the kinematic information of each frequency band shows that more informative frequency bands were [Formula: see text] (15-30 Hz) and [Formula: see text](50-100 Hz) for ipsilateral and [Formula: see text] and [Formula: see text] (100-200 Hz) for contralateral movements. It is observed that ipsilateral movement was decoded better than contralateral movement for [Formula: see text] (5-15 Hz) and [Formula: see text] bands, while contralateral movements was decoded better for [Formula: see text] (30-200 Hz) and hfECoG (200-400 Hz) bands. SIGNIFICANCE Accurate decoding the bilateral movement using the ECoG recorded from one brain hemisphere is an important issue toward real-life applications of the brain-machine interface technologies.
Collapse
Affiliation(s)
- Behraz Farrokhi
- Department of Biomedical Engineering, School of Electrical Engineering, Iran University of Science and Technology (IUST), Iran Neural Technology Research Centre, Tehran, Iran
| | | |
Collapse
|
44
|
Araki T, Uemura T, Yoshimoto S, Takemoto A, Noda Y, Izumi S, Sekitani T. Wireless Monitoring Using a Stretchable and Transparent Sensor Sheet Containing Metal Nanowires. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2020; 32:e1902684. [PMID: 31782576 DOI: 10.1002/adma.201902684] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2019] [Revised: 09/02/2019] [Indexed: 05/24/2023]
Abstract
Mechanically and visually imperceptible sensor sheets integrated with lightweight wireless loggers are employed in ultimate flexible hybrid electronics (FHE) to reduce vital stress/nervousness and monitor natural biosignal responses. The key technologies and applications for conceptual sensor system fabrication are reported, as exemplified by the use of a stretchable sensor sheet completely conforming to an individual's body surface to realize a low-noise wireless monitoring system (<1 µV) that can be attached to the human forehead for recording electroencephalograms. The above system can discriminate between Alzheimer's disease and the healthy state, thus offering a rapid in-home brain diagnosis possibility. Moreover, the introduction of metal nanowires to improve the transparency of the biocompatible sensor sheet allows one to wirelessly acquire electrocorticograms of nonhuman primates and simultaneously offers optogenetic stimulation such as toward-the-brain-machine interface under free movement. Also discussed are effective methods of improving electrical reliability, biocompatibility, miniaturization, etc., for metal nanowire based tracks and exploring the use of an organic amplifier as an important component to realize a flexible active probe with a high signal-to-noise ratio. Overall, ultimate FHE technologies are demonstrated to achieve efficient closed-loop systems for healthcare management, medical diagnostics, and preclinical studies in neuroscience and neuroengineering.
Collapse
Affiliation(s)
- Teppei Araki
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
- Advanced Photonics and Biosensing Open Innovation Laboratory, National Institute of Advanced Industrial Science and Technology (AIST), Suita, Osaka, 565-0871, Japan
- Graduate School of Engineering, Osaka University, Suita, Osaka, 565-0871, Japan
| | - Takafumi Uemura
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
- Advanced Photonics and Biosensing Open Innovation Laboratory, National Institute of Advanced Industrial Science and Technology (AIST), Suita, Osaka, 565-0871, Japan
| | - Shusuke Yoshimoto
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
| | - Ashuya Takemoto
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
- Advanced Photonics and Biosensing Open Innovation Laboratory, National Institute of Advanced Industrial Science and Technology (AIST), Suita, Osaka, 565-0871, Japan
- Graduate School of Engineering, Osaka University, Suita, Osaka, 565-0871, Japan
| | - Yuki Noda
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
- Artificial Intelligence Research Center, The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
| | - Shintaro Izumi
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
| | - Tsuyoshi Sekitani
- The Institute of Scientific and Industrial Research (ISIR), Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047, Japan
- Advanced Photonics and Biosensing Open Innovation Laboratory, National Institute of Advanced Industrial Science and Technology (AIST), Suita, Osaka, 565-0871, Japan
- Graduate School of Engineering, Osaka University, Suita, Osaka, 565-0871, Japan
| |
Collapse
|
45
|
Abstract
Intracranial electroencephalography (iEEG) is measured from electrodes placed in or on the brain. These measurements have an excellent signal-to-noise ratio and iEEG signals have often been used to decode brain activity or drive brain-computer interfaces (BCIs). iEEG recordings are typically done for seizure monitoring in epilepsy patients who have these electrodes placed for a clinical purpose: to localize both brain regions that are essential for function and others where seizures start. Brain regions not involved in epilepsy are thought to function normally and provide a unique opportunity to learn about human neurophysiology. Intracranial electrodes measure the aggregate activity of large neuronal populations and recorded signals contain many features. Different features are extracted by analyzing these signals in the time and frequency domain. The time domain may reveal an evoked potential at a particular time after the onset of an event. Decomposition into the frequency domain may show narrowband peaks in the spectrum at specific frequencies or broadband signal changes that span a wide range of frequencies. Broadband power increases are generally observed when a brain region is active while most other features are highly specific to brain regions, inputs, and tasks. Here we describe the spatiotemporal dynamics of several iEEG signals that have often been used to decode brain activity and drive BCIs.
Collapse
|
46
|
Kobler RJ, Almeida I, Sburlea AI, Müller-Putz GR. Using machine learning to reveal the population vector from EEG signals. J Neural Eng 2020; 17:026002. [PMID: 32048612 DOI: 10.1088/1741-2552/ab7490] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Since the discovery of the population vector that directly relates neural spiking activity with arm movement direction, it has become feasible to control robotic arms and neuroprostheses using invasively recorded brain signals. For non-invasive approaches, a direct relation between human brain signals and arm movement direction is yet to be established. APPROACH Here, we investigated electroencephalographic (EEG) signals in temporal and spectral domains in a continuous, circular arm movement task. Using machine learning methods that respect the linear mixture of brain activity within EEG signals, we show that directional information is represented in the temporal domain in amplitude modulations of the same frequency as the arm movement, and in the spectral domain in power modulations of the 20-24 Hz frequency band. MAIN RESULTS In the temporal domain, the directional information was mainly expressed in primary sensorimotor cortex (SM1) and posterior parietal cortex (PPC) contralateral to the moving arm, while in the spectral domain SM1 and PPC of both hemispheres predicted arm movement direction. The different cortical representations suggest distinct neural representations in both domains. SIGNIFICANCE This direct relation between neural activity and arm movement direction in both domains demonstrates the potential of machine learning to reveal neuroscientific insights about the dynamics of human arm movements.
Collapse
Affiliation(s)
- Reinmar J Kobler
- Institute of Neural Engineering, Graz University of Technology, Graz, Styria 8010, Austria. These authors contributed equally
| | | | | | | |
Collapse
|
47
|
Yan Y, Qian T, Xu X, Han H, Ling Z, Zhou W, Liu H, Hong B. Human cortical networking by probabilistic and frequency-specific coupling. Neuroimage 2020; 207:116363. [PMID: 31740339 DOI: 10.1016/j.neuroimage.2019.116363] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Revised: 11/03/2019] [Accepted: 11/13/2019] [Indexed: 11/26/2022] Open
Abstract
Large-scale cortical networking patterns have been established based on the correlation of slow fluctuations of resting fMRI signals. However, the electrophysiological mechanism of cortical networking remained to be elucidated. With large-scale human ECoG recording, we developed a novel approach for functional network parcellation on the basis of probabilistic co-activation of cortical sites in spatio-temporal microstates. The parcellated networks were verified by electrical cortical stimulation (ECS) and somatosensory evoked potentials recording, which showed significantly higher accuracy than the traditional long-term correlation method. This provides direct electrophysiological evidence supporting the dynamic nature of cortical networking. Further analysis revealed that the brain-wide connectivity is likely established on the coupling of ECoG power envelop over a common carrier frequency ranging from alpha to low-beta (8-32Hz). Surprisingly, the cortical networking pattern over this specific frequency was found to be consistent across various tasks, which resembles the resting networks. The high similarity between the above functional network parcellation and the fMRI resting network atlas in individuals also suggested the slow power-envelope coupling of band-limited neural oscillations as the electrophysiological basis of spontaneous BOLD signals. Collectively, our findings on direct human recording revealed a probabilistic and frequency specific coupling mechanism for large-scale cortical networking shared by task and resting brain.
Collapse
Affiliation(s)
- Yuxiang Yan
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China
| | - Tianyi Qian
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China
| | - Xin Xu
- Department of Neurosurgery, Chinese PLA General Hospital, Beijing, 100853, China
| | - Hao Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China
| | - Zhipei Ling
- Department of Neurosurgery, Chinese PLA General Hospital, Beijing, 100853, China
| | - Wenjin Zhou
- Epilepsy Center, Yuquan Hospital, Tsinghua University, Beijing, 100040, China
| | - Hesheng Liu
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Charlestown, MA, 02129, USA.
| | - Bo Hong
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, 100084, China.
| |
Collapse
|
48
|
Unterweger J, Seeber M, Zanos S, Ojemann JG, Scherer R. ECoG Beta Suppression and Modulation During Finger Extension and Flexion. Front Neurosci 2020; 14:35. [PMID: 32116497 PMCID: PMC7031656 DOI: 10.3389/fnins.2020.00035] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2019] [Accepted: 01/13/2020] [Indexed: 11/13/2022] Open
Abstract
Neural oscillations originate predominantly from interacting cortical neurons and consequently reflect aspects of cortical information processing. However, their functional role is not yet fully understood and their interpretation is debatable. Amplitude modulations (AMs) in alpha (8–12 Hz), beta (13–30 Hz), and high gamma (70–150 Hz) band in invasive electrocorticogram (ECoG) and non-invasive electroencephalogram (EEG) signals change with behavior. Alpha and beta band AMs are typically suppressed (desynchronized) during motor behavior, while high gamma AMs highly correlate with the behavior. These two phenomena are successfully used for functional brain mapping and brain-computer interface (BCI) applications. Recent research found movement-phase related AMs (MPA) also in high beta/low gamma (24–40 Hz) EEG rhythms. These MPAs were found by separating the suppressed AMs into sustained and dynamic components. Sustained AM components are those with frequencies that are lower than the motor behavior. Dynamic components those with frequencies higher than the behavior. In this paper, we study ECoG beta/low gamma band (12–30 Hz/30–42 Hz) AM during repetitive finger movements addressing the question whether or not MPAs can be found in ECoG beta band. Indeed, MPA in the 12–18 Hz and 18–24 Hz band were found. This additional information may lead to further improvements in ECoG-based prediction and reconstruction of motor behavior by combining high gamma AM and beta band MPA.
Collapse
Affiliation(s)
- Julian Unterweger
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Martin Seeber
- Functional Brain Mapping Laboratory, Department of Fundamental Neurosciences, University of Geneva, Geneva, Switzerland
| | - Stavros Zanos
- Translational Neurophysiology Laboratory, Institute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, Manhasset, NY, United States
| | - Jeffrey G Ojemann
- Department of Neurological Surgery, University of Washington, Seattle, WA, United States
| | - Reinhold Scherer
- Brain-Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| |
Collapse
|
49
|
Brain mechanisms in motor control during reaching movements: Transition of functional connectivity according to movement states. Sci Rep 2020; 10:567. [PMID: 31953515 PMCID: PMC6969071 DOI: 10.1038/s41598-020-57489-7] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2019] [Accepted: 12/05/2019] [Indexed: 12/15/2022] Open
Abstract
Understanding how the brain controls movements is a critical issue in neuroscience. The role of brain changes rapidly according to movement states. To elucidate the motor control mechanism of brain, it is essential to investigate the changes in brain network in motor-related regions according to movement states. Therefore, the objective of this study was to investigate the brain network transitions according to movement states. We measured whole brain magnetoencephalography (MEG) signals and extracted source signals in 24 motor-related areas. Functional connectivity and centralities were calculated according to time flow. Our results showed that brain networks differed between states of motor planning and movement. Connectivities between most motor-related areas were increased in the motor-planning state. In contrast, only connectivities with cerebellum and basal ganglia were increased while those of other motor-related areas were decreased during movement. Our results indicate that most processes involved in motor control are completed before movement. Further, brain developed network related to feedback rather than motor decision during movements. Our findings also suggest that neural signals during motor planning might be more predictive than neural signals during movement. They facilitate accurate prediction of movement for brain-machine interfaces and provide insight into brain mechanisms in motor control.
Collapse
|
50
|
Heydari Beni N, Foodeh R, Shalchyan V, Daliri MR. Force decoding using local field potentials in primary motor cortex: PLS or Kalman filter regression? AUSTRALASIAN PHYSICAL & ENGINEERING SCIENCES IN MEDICINE 2020; 43:10.1007/s13246-019-00833-7. [PMID: 31898242 DOI: 10.1007/s13246-019-00833-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/20/2019] [Accepted: 12/09/2019] [Indexed: 11/30/2022]
Abstract
The development of brain-computer interface (BCI) systems is an important approach in brain studies. Control of communication devices and prostheses in real-world scenarios requires complex movement parameters. Decoding a variety of neural signals captured by micro-wire arrays is a potential applicant for extracting movement-related information. The present work was conducted to compare the functionality of partial least square (PLS) regression and Kalman filter to predict the force parameter from local field potential (LFP) signals of the primary motor cortex (M1). The signals were recorded using a 16-channel micro-wire array from the forelimb-related area of the M1 of three rats performing a behavioral task in which the force signal of the rat's forelimb paw was generated. Our results show that PLS regression and Kalman filters with the mean performance of 0.75 and 0.72 in terms of the correlation coefficient (CC) and 0.37 and 0.48 in terms of normalized mean square error (NMSE), respectively, are effective methods for decoding the force parameter from LFPs. Kalman filter underperforms PLS both in performance and speed. Although adding nonlinearity to the Kalman filter results in equally accurate CC performance as PLS, it has even more computational cost. Therefore, it is inferred that nonlinear methods do not necessarily have better functionality than linear ones and PLS, as a simple fast linear method could be an effectively applicable regression technique for BCIs.
Collapse
Affiliation(s)
- Nargess Heydari Beni
- Neuroscience and Neuroengineering Research Lab, Department of Biomedical Engineering, School of Electrical Engineering, Iran University of Science and Technology (IUST), Narmak, 16846-13114, Tehran, Iran
- Engineering Bionics Lab, Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada
| | - Reza Foodeh
- Neuroscience and Neuroengineering Research Lab, Department of Biomedical Engineering, School of Electrical Engineering, Iran University of Science and Technology (IUST), Narmak, 16846-13114, Tehran, Iran
| | - Vahid Shalchyan
- Neuroscience and Neuroengineering Research Lab, Department of Biomedical Engineering, School of Electrical Engineering, Iran University of Science and Technology (IUST), Narmak, 16846-13114, Tehran, Iran
| | - Mohammad Reza Daliri
- Neuroscience and Neuroengineering Research Lab, Department of Biomedical Engineering, School of Electrical Engineering, Iran University of Science and Technology (IUST), Narmak, 16846-13114, Tehran, Iran.
| |
Collapse
|