Decoding natural grasp types from human ECoG

The causal role of beta band desynchronization: Individualized high-definition transcranial alternating current stimulation improves bimanual motor control

OBJECTIVE

To unveil if 3 mA peak-to-peak high-definition β transcranial alternating current stimulation (tACS) applied over C4 -the area overlaying the right sensorimotor cortex-enhances bimanual motor control and affects movement-related β desynchronization (MRβD), thereby providing causal evidence for the polymorphic role of MRβD in motor control.

METHODS

In this sham-controlled, crossover study, 36 participants underwent 20 min of fixed 20 Hz tACS; tACS individualized to peak β activity during motor planning at baseline; and sham tACS randomized over three consecutive days. Each participant underwent all three conditions for a total of 108 sessions, ensuring within-subject comparisons. Before, during, and after tACS, participants performed a bimanual tracking task (BTT) and 64-channel electroencephalography (EEG) data was measured. Spatiotemporal and temporal clustering statistics with underlying linear mixed effect models were used to test our hypotheses.

RESULTS

Individualized tACS significantly improved bimanual motor control, both online and offline, and increased online MRβD during motor planning compared to fixed tACS. No offline effects of fixed and individualized tACS on MRβD were found compared to sham, although tACS effects did trend towards the hypothesized MRβD increase. Throughout the course of the study, MRβD and bimanual motor performance increased. Exclusively during motor planning, MRβD was positively associated to bimanual motor performance improvements, emphasizing the functionally polymorphic role of MRβD. tACS was well tolerated and no side-effects occurred.

CONCLUSION

Individualized β-tACS improves bimanual motor control and enhances motor planning MRβD online. These findings provide causal evidence for the importance of MRβD when planning complex motor behavior.

Effector specificity in human posterior parietal neurons and local field potentials during movement in virtual reality and online brain control

Objective. Neural prosthetics represent a significant opportunity for control of external effectors like artificial limbs and computer devices as well as a means for interacting with virtual reality. Prior studies have shown posterior parietal cortex (PPC) to be a viable source of signals for the purposes of decoding motor intentions given its representation of both visual inputs and motor outputs. Additionally, signals in parietal cortex have been shown to be associated with tool use the body schema. We investigated if more realistic movement effectors in virtual reality might elicit stronger signals at the single neuron level in parietal cortex.Approach. A quadriplegic human subject was implanted with multi-electrode recording arrays in the PPC. Neural spiking and local field potentials were recorded during attempted movement in a computer-rendered, stereoscopic, 3D virtual environment. Tuning to different movement effectors was examined using a first-person movement generation task in addition to closed loop control performance.Main results. We found single neurons and simultaneously recorded field potentials in a quadriplegic patient exhibited enhanced responses during attempted (rather than passively observed) movement of a realistic and 'attached' 3D arm relative to either a visually similar but 'detached' 2D arm or a non-anthropomorphic abstract effector. These preferences were found despite the patient having lost motor function years prior. These differences did not effect performance during closed loop brain control of the movement effectors.Significance. In human parietal cortex, single neuron activity and local field potentials responded preferentially to visually guided attempted movement of a realistic arm rather than abstract effector. However, this tuning did not affect closed loop brain control in a virtual reality environment when preceded by a text-based decoder training paradigm.

EEG-based recognition of hand movement and its parameter

Objecitve. Brain-computer interface is a cutting-edge technology that enables interaction with external devices by decoding human intentions, and is highly valuable in the fields of medical rehabilitation and human-robot collaboration. The technique of decoding motor intent for motor execution (ME) based on electroencephalographic (EEG) signals is in the feasibility study stage by now. There are still insufficient studies on the accuracy of ME EEG signal recognition in between-subjects classification to reach the level of realistic applications. This paper aims to investigate EEG signal-based hand movement recognition by analyzing low-frequency time-domain information.Approach. Experiments with four types of hand movements, two force parameter (picking up and pushing) tasks, and a four-target directional displacement task were designed and executed, and the EEG data from thirteen healthy volunteers was collected. Sliding window approach is used to expand the dataset in order to address the issue of EEG signal overfitting. Furtherly, Convolutional Neural Network (CNN)-Bidirectional Long Short-Term Memory Network (BiLSTM) model, an end-to-end serial combination of a BiLSTM and (CNN) is constructed to classify and recognize the hand movement based on the raw EEG data.Main results. According to the experimental results, the model is able to categorize four types of hand movements, picking up movements, pushing movements, and four target direction displacement movements with an accuracy of 99.14% ± 0.49%, 99.29% ± 0.11%, 99.23% ± 0.60%, and 98.11% ± 0.23%, respectively.Significance. Furthermore, comparative tests conducted with alternative deep learning models (LSTM, CNN, EEGNet, CNN-LSTM) demonstrates that the CNN-BiLSTM model is with practicable accuracy in terms of EEG-based hand movement recognition and its parameter decoding.

Fast, accurate, and interpretable decoding of electrocorticographic signals using dynamic mode decomposition

Dynamic mode (DM) decomposition decomposes spatiotemporal signals into basic oscillatory components (DMs). DMs can improve the accuracy of neural decoding when used with the nonlinear Grassmann kernel, compared to conventional power features. However, such kernel-based machine learning algorithms have three limitations: large computational time preventing real-time application, incompatibility with non-kernel algorithms, and low interpretability. Here, we propose a mapping function corresponding to the Grassmann kernel that explicitly transforms DMs into spatial DM (sDM) features, which can be used in any machine learning algorithm. Using electrocorticographic signals recorded during various movement and visual perception tasks, the sDM features were shown to improve the decoding accuracy and computational time compared to conventional methods. Furthermore, the components of the sDM features informative for decoding showed similar characteristics to the high-γ power of the signals, but with higher trial-to-trial reproducibility. The proposed sDM features enable fast, accurate, and interpretable neural decoding.

Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome

Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.

A New Compound-Limbs Paradigm: Integrating Upper-Limb Swing Improves Lower-Limb Stepping Intention Decoding From EEG

Brain-computer interface (BCI) systems based on spontaneous electroencephalography (EEG) hold the promise to implement human voluntary control of lower-extremity powered exoskeletons. However, current EEG-BCI paradigms do not consider the cooperation of upper and lower limbs during walking, which is inconsistent with natural human stepping patterns. To deal with this problem, this study proposed a stepping-matched human EEG-BCI paradigm that involved actions of both unilateral lower and contralateral upper limbs (also referred to as compound-limbs movement). Experiments were conducted in motor execution (ME) and motor imagery (MI) conditions to validate the feasibility. Common spatial pattern (CSP) proposed subject-specific CSP (SSCSP), and filter-bank CSP (FBCSP) methods were applied for feature extraction, respectively. The best average classification results based on SSCSP indicated that the accuracies of compound-limbs paradigms in ME and MI conditions achieved 89.02% ± 12.84% and 73.70% ± 12.47%, respectively. Although they were 2.03% and 5.68% lower than those of the single-upper-limb mode that does not match human stepping patterns, they were 24.30% and 11.02% higher than those of the single-lower-limb mode. These findings indicated that the proposed compound-limbs EEG-BCI paradigm is feasible for decoding human stepping intention and thus provides a potential way for natural human control of walking assistance devices.

A click-based electrocorticographic brain-computer interface enables long-term high-performance switch-scan spelling

Background

Brain-computer interfaces (BCIs) can restore communication in movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command "click" decoders provide a basic yet highly functional capability.

Methods

We sought to test the performance and long-term stability of click-decoding using a chronically implanted high density electrocorticographic (ECoG) BCI with coverage of the sensorimotor cortex in a human clinical trial participant (ClinicalTrials.gov, NCT03567213) with amyotrophic lateral sclerosis (ALS). We trained the participant's click decoder using a small amount of training data (< 44 minutes across four days) collected up to 21 days prior to BCI use, and then tested it over a period of 90 days without any retraining or updating.

Results

Using this click decoder to navigate a switch-scanning spelling interface, the study participant was able to maintain a median spelling rate of 10.2 characters per min. Though a transient reduction in signal power modulation interrupted testing with this fixed model, a new click decoder achieved comparable performance despite being trained with even less data (< 15 min, within one day).

Conclusion

These results demonstrate that a click decoder can be trained with a small ECoG dataset while retaining robust performance for extended periods, providing functional text-based communication to BCI users.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

Characterization of High-Gamma Activity in Electrocorticographic Signals

Introduction

Electrocorticographic (ECoG) high-gamma activity (HGA) is a widely recognized and robust neural correlate of cognition and behavior. However, fundamental signal properties of HGA, such as the high-gamma frequency band or temporal dynamics of HGA, have never been systematically characterized. As a result, HGA estimators are often poorly adjusted, such that they miss valuable physiological information.

Methods

To address these issues, we conducted a thorough qualitative and quantitative characterization of HGA in ECoG signals. Our study is based on ECoG signals recorded from 18 epilepsy patients while performing motor control, listening, and visual perception tasks. In this study, we first categorize HGA into HGA types based on the cognitive/behavioral task. For each HGA type, we then systematically quantify three fundamental signal properties of HGA: the high-gamma frequency band, the HGA bandwidth, and the temporal dynamics of HGA.

Results

The high-gamma frequency band strongly varies across subjects and across cognitive/behavioral tasks. In addition, HGA time courses have lowpass character, with transients limited to 10 Hz. The task-related rise time and duration of these HGA time courses depend on the individual subject and cognitive/behavioral task. Task-related HGA amplitudes are comparable across the investigated tasks.

Discussion

This study is of high practical relevance because it provides a systematic basis for optimizing experiment design, ECoG acquisition and processing, and HGA estimation. Our results reveal previously unknown characteristics of HGA, the physiological principles of which need to be investigated in further studies.

Grasp-specific high-frequency broadband mirror neuron activity during reach-and-grasp movements in humans

Broadly congruent mirror neurons, responding to any grasp movement, and strictly congruent mirror neurons, responding only to specific grasp movements, have been reported in single-cell studies with primates. Delineating grasp properties in humans is essential to understand the human mirror neuron system with implications for behavior and social cognition. We analyzed electrocorticography data from a natural reach-and-grasp movement observation and delayed imitation task with 3 different natural grasp types of everyday objects. We focused on the classification of grasp types from high-frequency broadband mirror activation patterns found in classic mirror system areas, including sensorimotor, supplementary motor, inferior frontal, and parietal cortices. Classification of grasp types was successful during movement observation and execution intervals but not during movement retention. Our grasp type classification from combined and single mirror electrodes provides evidence for grasp-congruent activity in the human mirror neuron system potentially arising from strictly congruent mirror neurons.

Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance

Introduction

In brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation.

Methods

We evaluated the multilinear model and two deep learning (DL) models on a long-term BCI &amp; Tetraplegia (ClinicalTrials.gov identifier: NCT02550522) clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. In the experiment, a participant executed 3D virtual hand translation using motor imagery patterns. We designed multiple computational experiments in which training datasets were increased or translated to investigate the relationship between models' performance and different factors influencing recordings.

Results

Our results showed that DL decoders showed similar requirements regarding the dataset size compared to the multilinear model while demonstrating higher decoding performance. Moreover, high decoding performance was obtained with relatively small datasets recorded later in the experiment, suggesting motor imagery patterns improvement and patient adaptation during the long-term experiment. Finally, we proposed UMAP embeddings and local intrinsic dimensionality as a way to visualize the data and potentially evaluate data quality.

Discussion

DL-based decoding is a prospective approach in BCI which may be efficiently applied with real-life dataset size. Patient-decoder co-adaptation is an important factor to consider in long-term clinical BCI.

Towards clinical application of implantable brain-computer interfaces for people with late-stage ALS: medical and ethical considerations

Individuals with amyotrophic lateral sclerosis (ALS) frequently develop speech and communication problems in the course of their disease. Currently available augmentative and alternative communication technologies do not present a solution for many people with advanced ALS, because these devices depend on residual and reliable motor activity. Brain-computer interfaces (BCIs) use neural signals for computer control and may allow people with late-stage ALS to communicate even when conventional technology falls short. Recent years have witnessed fast progression in the development and validation of implanted BCIs, which place neural signal recording electrodes in or on the cortex. Eventual widespread clinical application of implanted BCIs as an assistive communication technology for people with ALS will have significant consequences for their daily life, as well as for the clinical management of the disease, among others because of the potential interaction between the BCI and other procedures people with ALS undergo, such as tracheostomy. This article aims to facilitate responsible real-world implementation of implanted BCIs. We review the state of the art of research on implanted BCIs for communication, as well as the medical and ethical implications of the clinical application of this technology. We conclude that the contribution of all BCI stakeholders, including clinicians of the various ALS-related disciplines, will be needed to develop procedures for, and shape the process of, the responsible clinical application of implanted BCIs.

Aging and Complexity Effects on Hemisphere-Dependent Movement-Related Beta Desynchronization during Bimanual Motor Planning and Execution

With aging comes degradation of bimanual movement performance. A hallmark feature of bimanual movements is movement-related beta desynchronization (MRBD), an attenuation in the amplitude of beta oscillations associated with sensorimotor activation. Here, we investigated MRBD in 39 healthy adults (20 younger and 19 older adults) in frontal, central, and parietal regions across both hemispheres, during the planning and execution of a bimanual tracking task. Task accuracy decreased with age and during more difficult conditions when both hands had to move at different relative speeds. MRBD was mostly situated in the central region, and increased in older versus younger adults during movement execution but not planning. Irrespective of age, motor planning and execution were associated with increased MRBD in the left and right hemispheres, respectively. Notably, right central MRBD during motor planning was associated with bimanual task performance, particularly in older adults. Specifically, persons who demonstrated high MRBD during motor planning performed better on the bimanual tracking task. Our results highlight the importance of lateralized MRBD during motor planning, thereby shining new light on previous research and providing a promising avenue for future interventions.

Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals

Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.

Exploring EEG spectral and temporal dynamics underlying a hand grasp movement

For brain-computer interfaces, resolving the differences between pre-movement and movement requires decoding neural ensemble activity in the motor cortex's functional regions and behavioural patterns. Here, we explored the underlying neural activity and mechanisms concerning a grasped motor task by recording electroencephalography (EEG) signals during the execution of hand movements in healthy subjects. The grasped movement included different tasks; reaching the target, grasping the target, lifting the object upwards, and moving the object in the left or right directions. 163 trials of EEG data were acquired from 30 healthy participants who performed the grasped movement tasks. Rhythmic EEG activity was analysed during the premovement (alert task) condition and compared against grasped movement tasks while the arm was moved towards the left or right directions. The short positive to negative deflection that initiated around -0.5ms as a wave before the onset of movement cue can be used as a potential biomarker to differentiate movement initiation and movement. A rebound increment of 14% of beta oscillations and 26% gamma oscillations in the central regions was observed and could be used to distinguish pre-movement and grasped movement tasks. Comparing movement initiation to grasp showed a decrease of 10% in beta oscillations and 13% in gamma oscillations, and there was a rebound increment 4% beta and 3% gamma from grasp to grasped movement. We also investigated the combination MRCPs and spectral estimates of α, β, and γ oscillations as features for machine learning classifiers that could categorize movement conditions. Support vector machines with 3rd order polynomial kernel yielded 70% accuracy. Pruning the ranked features to 5 leaf nodes reduced the error rate by 16%. For decoding grasped movement and in the context of BCI applications, this study identifies potential biomarkers, including the spatio-temporal characteristics of MRCPs, spectral information, and choice of classifiers for optimally distinguishing initiation and grasped movement.

A Power-Efficient Brain-Machine Interface System With a Sub-mw Feature Extraction and Decoding ASIC Demonstrated in Nonhuman Primates

Intracortical brain-machine interfaces have shown promise for restoring function to people with paralysis, but their translation to portable and implantable devices is hindered by their high power consumption. Recent devices have drastically reduced power consumption compared to standard experimental brain-machine interfaces, but still require wired or wireless connections to computing hardware for feature extraction and inference. Here, we introduce a Neural Recording And Decoding (NeuRAD) application specific integrated circuit (ASIC) in 180 nm CMOS that can extract neural spiking features and predict two-dimensional behaviors in real-time. To reduce amplifier and feature extraction power consumption, the NeuRAD has a hardware accelerator for extracting spiking band power (SBP) from intracortical spiking signals and includes an M0 processor with a fixed-point Matrix Acceleration Unit (MAU) for efficient and flexible decoding. We validated device functionality by recording SBP from a nonhuman primate implanted with a Utah microelectrode array and predicting the one- and two-dimensional finger movements the monkey was attempting to execute in closed-loop using a steady-state Kalman filter (SSKF). Using the NeuRAD's real-time predictions, the monkey achieved 100% success rate and 0.82 s mean target acquisition time to control one-dimensional finger movements using just 581 μW. To predict two-dimensional finger movements, the NeuRAD consumed 588 μW to enable the monkey to achieve a 96% success rate and 2.4 s mean acquisition time. By employing SBP, ASIC brain-machine interfaces can close the gap to enable fully implantable therapies for people with paralysis.

Noninvasively recorded high-gamma signals improve synchrony of force feedback in a novel neurorehabilitation brain-machine interface for brain injury

OBJECTIVE

Brain injury is the leading cause of long-term disability worldwide, often resulting in impaired hand function. Brain-machine interfaces (BMIs) offer a potential way to improve hand function. BMIs often target replacing lost function, but may also be employed in neurorehabilitation (nrBMI) by facilitating neural plasticity and functional recovery. Here, we report a novel nrBMI capable of acquiring high-γ (70-115 Hz) information through a unique post-TBI hemicraniectomy window model, and delivering sensory feedback that is synchronized with, and proportional to, intended grasp force.

APPROACH

We developed the nrBMI to use electroencephalogram recorded over a hemicraniectomy (hEEG) in individuals with traumatic brain injury (TBI). The nrBMI empowered users to exert continuous, proportional control of applied force, and provided continuous force feedback. We report the results of an initial testing group of three human participants with TBI, along with a control group of three skull- and motor-intact volunteers.

MAIN RESULTS

All participants controlled the nrBMI successfully, with high initial success rates (2 of 6 participants) or performance that improved over time (4 of 6 participants). We observed high-γ modulation with force intent in hEEG but not skull-intact EEG. Most significantly, we found that high-γ control significantly improved the timing synchronization between neural modulation onset and nrBMI output/haptic feedback (compared to low-frequency nrBMI control).

SIGNIFICANCE

These proof-of-concept results show that high-γ nrBMIs can be used by individuals with impaired ability to control force (without immediately resorting to invasive signals like ECoG). Of note, the nrBMI includes a parameter to change the fraction of control shared between decoded intent and volitional force, to adjust for recovery progress. The improved synchrony between neural modulations and force control for high-γ signals is potentially important for maximizing the ability of nrBMIs to induce plasticity in neural circuits. Inducing plasticity is critical to functional recovery after brain injury.

AJILE12: Long-term naturalistic human intracranial neural recordings and pose

Understanding the neural basis of human movement in naturalistic scenarios is critical for expanding neuroscience research beyond constrained laboratory paradigms. Here, we describe our Annotated Joints in Long-term Electrocorticography for 12 human participants (AJILE12) dataset, the largest human neurobehavioral dataset that is publicly available; the dataset was recorded opportunistically during passive clinical epilepsy monitoring. AJILE12 includes synchronized intracranial neural recordings and upper body pose trajectories across 55 semi-continuous days of naturalistic movements, along with relevant metadata, including thousands of wrist movement events and annotated behavioral states. Neural recordings are available at 500 Hz from at least 64 electrodes per participant, for a total of 1280 hours. Pose trajectories at 9 upper-body keypoints were estimated from 118 million video frames. To facilitate data exploration and reuse, we have shared AJILE12 on The DANDI Archive in the Neurodata Without Borders (NWB) data standard and developed a browser-based dashboard.

Decoding ECoG signal into 3D hand translation using deep learning

Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.

Human brain mapping with multithousand-channel PtNRGrids resolves spatiotemporal dynamics

Electrophysiological devices are critical for mapping eloquent and diseased brain regions and for therapeutic neuromodulation in clinical settings and are extensively used for research in brain-machine interfaces. However, the existing clinical and experimental devices are often limited in either spatial resolution or cortical coverage. Here, we developed scalable manufacturing processes with a dense electrical connection scheme to achieve reconfigurable thin-film, multithousand-channel neurophysiological recording grids using platinum nanorods (PtNRGrids). With PtNRGrids, we have achieved a multithousand-channel array of small (30 μm) contacts with low impedance, providing high spatial and temporal resolution over a large cortical area. We demonstrated that PtNRGrids can resolve submillimeter functional organization of the barrel cortex in anesthetized rats that captured the tissue structure. In the clinical setting, PtNRGrids resolved fine, complex temporal dynamics from the cortical surface in an awake human patient performing grasping tasks. In addition, the PtNRGrids identified the spatial spread and dynamics of epileptic discharges in a patient undergoing epilepsy surgery at 1-mm spatial resolution, including activity induced by direct electrical stimulation. Collectively, these findings demonstrated the power of the PtNRGrids to transform clinical mapping and research with brain-machine interfaces.

Compartmentalized dynamics within a common multi-area mesoscale manifold represent a repertoire of human hand movements

The human hand is unique in the animal kingdom for unparalleled dexterity, ranging from complex prehension to fine finger individuation. How does the brain represent such a diverse repertoire of movements? We evaluated mesoscale neural dynamics across the human "grasp network," using electrocorticography and dimensionality reduction methods, for a repertoire of hand movements. Strikingly, we found that the grasp network represented both finger and grasping movements alike. Specifically, the manifold characterizing the multi-areal neural covariance structure was preserved during all movements across this distributed network. In contrast, latent neural dynamics within this manifold were surprisingly specific to movement type. Aligning latent activity to kinematics further uncovered distinct submanifolds despite similarities in synergistic coupling of joints between movements. We thus find that despite preserved neural covariance at the distributed network level, mesoscale dynamics are compartmentalized into movement-specific submanifolds; this mesoscale organization may allow flexible switching between a repertoire of hand movements.

Brain Processes Involved in Motor Planning Are a Dominant Factor for Inducing Event-Related Desynchronization

Event-related desynchronization (ERD) is a relative attenuation in the spectral power of an electroencephalogram (EEG) observed over the sensorimotor area during motor execution and motor imagery. It is a well-known EEG feature and is commonly employed in brain-computer interfaces. However, its underlying neural mechanisms are not fully understood, as ERD is a single variable correlated with external events involving numerous pathways, such as motor intention, planning, and execution. In this study, we aimed to identify a dominant factor for inducing ERD. Participants were instructed to grasp their right hand with three different (10, 25, or 40%MVF: maximum voluntary force) levels under two distinct experimental conditions: a closed-loop condition involving real-time visual force feedback (VF) or an open-loop condition in a feedforward (FF) manner. In each condition, participants were instructed to repeat the grasping task a certain number of times with a timeline of Rest (10.0 s), Preparation (1.0 s), and Motor Execution (4.0 s) periods, respectively. EEG signals were recorded simultaneously with the motor task to evaluate the time-course of the event-related spectrum perturbation for each condition and dissect the modulation of EEG power. We performed statistical analysis of mu and beta-ERD under the instructed grasping force levels and the feedback conditions. In the FF condition (i.e., no force feedback), mu and beta-ERD were significantly attenuated in the contralateral motor cortex during the middle of the motor execution period, while ERD in the VF condition was maintained even during keep grasping. Only mu-ERD at the somatosensory cortex tended to be slightly stronger in high load conditions. The results suggest that the extent of ERD reflects neural activity involved in the motor planning process for changing virtual equilibrium point rather than the motor control process for recruiting motor neurons to regulate grasping force.

Investigating Data Cleaning Methods to Improve Performance of Brain-Computer Interfaces Based on Stereo-Electroencephalography

Stereo-electroencephalography (SEEG) utilizes localized and penetrating depth electrodes to directly measure electrophysiological brain activity. The implanted electrodes generally provide a sparse sampling of multiple brain regions, including both cortical and subcortical structures, making the SEEG neural recordings a potential source for the brain–computer interface (BCI) purpose in recent years. For SEEG signals, data cleaning is an essential preprocessing step in removing excessive noises for further analysis. However, little is known about what kinds of effect that different data cleaning methods may exert on BCI decoding performance and, moreover, what are the reasons causing the differentiated effects. To address these questions, we adopted five different data cleaning methods, including common average reference, gray–white matter reference, electrode shaft reference, bipolar reference, and Laplacian reference, to process the SEEG data and evaluated the effect of these methods on improving BCI decoding performance. Additionally, we also comparatively investigated the changes of SEEG signals induced by these different methods from multiple-domain (e.g., spatial, spectral, and temporal domain). The results showed that data cleaning methods could improve the accuracy of gesture decoding, where the Laplacian reference produced the best performance. Further analysis revealed that the superiority of the data cleaning method with excellent performance might be attributed to the increased distinguishability in the low-frequency band. The findings of this work highlighted the importance of applying proper data clean methods for SEEG signals and proposed the application of Laplacian reference for SEEG-based BCI.

Decoding Hand Movement Types and Kinematic Information From Electroencephalogram

Brain-computer interfaces (BCIs) have achieved successful control of assistive devices, e.g. neuroprosthesis or robotic arm. Previous research based on hand movements Electroencephalogram (EEG) has shown limited success in precise and natural control. In this study, we explored the possibilities of decoding movement types and kinematic information for three reach-and-execute actions using movement-related cortical potentials (MRCPs). EEG signals were acquired from 12 healthy subjects during the execution of pinch, palmar and precision disk rotation actions that involved two levels of speeds and forces. In the case of discrimination between hand movement types under each of four different kinematics conditions, we obtained the average peak accuracies of 83.44% and 73.83% for the binary and 3-class classification, respectively. In the case of discrimination between different movement kinematics for each of three actions, the average peak accuracies of 82.9% and 58.2% could be achieved for the two and 4-class scenario. In both cases, peak decoding performance was significantly higher than the subject-specific chance level. We found that hand movement types all could be classified when these actions were executed at four different kinematic parameters. Meanwhile, for each of three hand movements, we decoded movement parameters successfully. Furthermore, the feasibility of decoding hand movements during hand retraction process was also demonstrated. These findings are of great importance for controlling neuroprosthesis or other rehabilitation devices in a fine and natural way, which would drastically increase the acceptance of motor impaired users.

Detection of human white matter activation and evaluation of its function in movement decoding using stereo-electroencephalography (SEEG)

Objective. White matter tissue takes up approximately 50% of the human brain volume and it is widely known as a messenger conducting information between areas of the central nervous system. However, the characteristics of white matter neural activity and whether white matter neural recordings can contribute to movement decoding are often ignored and still remain largely unknown. In this work, we make quantitative analyses to investigate these two important questions using invasive neural recordings.Approach. We recorded stereo-electroencephalography (SEEG) data from 32 human subjects during a visually-cued motor task, where SEEG recordings can tap into gray and white matter electrical activity simultaneously. Using the proximal tissue density method, we identified the location (i.e. gray or white matter) of each SEEG contact. Focusing on alpha oscillatory and high gamma activities, we compared the activation patterns between gray matter and white matter. Then, we evaluated the performance of such white matter activation in movement decoding.Main results. The results show that white matter also presents activation under the task, in a similar way with the gray matter but at a significantly lower amplitude. Additionally, this work also demonstrates that combing white matter neural activities together with that of gray matter significantly promotes the movement decoding accuracy than using gray matter signals only.Significance. Taking advantage of SEEG recordings from a large number of subjects, we reveal the response characteristics of white matter neural signals under the task and demonstrate its enhancing function in movement decoding. This study highlights the importance of taking white matter activities into consideration in further scientific research and translational applications.

Restoring upper extremity function with brain-machine interfaces

One of the most exciting advances to emerge in neural interface technologies has been the development of real-time brain-machine interface (BMI) neuroprosthetic devices to restore upper extremity function. BMI neuroprostheses, made possible by synergistic advances in neural recording technologies, high-speed computation and signal processing, and neuroscience, have permitted the restoration of volitional movement to patients suffering the loss of upper-extremity function. In this chapter, we review the scientific and technological advances underlying these remarkable devices. After presenting an introduction to the current state of the field, we provide an accessible technical discussion of the two fundamental requirements of a successful neuroprosthesis: signal extraction from the brain and signal decoding that results in robust prosthetic control. We close with a presentation of emerging technologies that are likely to substantially advance the field.

Brain-Computer Interfaces for Children With Complex Communication Needs and Limited Mobility: A Systematic Review

Brain-computer interfaces (BCIs) represent a new frontier in the effort to maximize the ability of individuals with profound motor impairments to interact and communicate. While much literature points to BCIs' promise as an alternative access pathway, there have historically been few applications involving children and young adults with severe physical disabilities. As research is emerging in this sphere, this article aims to evaluate the current state of translating BCIs to the pediatric population. A systematic review was conducted using the Scopus, PubMed, and Ovid Medline databases. Studies of children and adolescents that reported BCI performance published in English in peer-reviewed journals between 2008 and May 2020 were included. Twelve publications were identified, providing strong evidence for continued research in pediatric BCIs. Research evidence was generally at multiple case study or exploratory study level, with modest sample sizes. Seven studies focused on BCIs for communication and five on mobility. Articles were categorized and grouped based on type of measurement (i.e., non-invasive and invasive), and the type of brain signal (i.e., sensory evoked potentials or movement-related potentials). Strengths and limitations of studies were identified and used to provide requirements for clinical translation of pediatric BCIs. This systematic review presents the state-of-the-art of pediatric BCIs focused on developing advanced technology to support children and youth with communication disabilities or limited manual ability. Despite a few research studies addressing the application of BCIs for communication and mobility in children, results are encouraging and future works should focus on customizable pediatric access technologies based on brain activity.

Behavioral and Neural Variability of Naturalistic Arm Movements

Motor behaviors are central to many functions and dysfunctions of the brain, and understanding their neural basis has consequently been a major focus in neuroscience. However, most studies of motor behaviors have been restricted to artificial, repetitive paradigms, far removed from natural movements performed "in the wild." Here, we leveraged recent advances in machine learning and computer vision to analyze intracranial recordings from 12 human subjects during thousands of spontaneous, unstructured arm reach movements, observed over several days for each subject. These naturalistic movements elicited cortical spectral power patterns consistent with findings from controlled paradigms, but with considerable neural variability across subjects and events. We modeled interevent variability using 10 behavioral and environmental features; the most important features explaining this variability were reach angle and day of recording. Our work is among the first studies connecting behavioral and neural variability across cortex in humans during unstructured movements and contributes to our understanding of long-term naturalistic behavior.

The Riemannian Spatial Pattern method: mapping and clustering movement imagery using Riemannian geometry

OBJECTIVE

Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying Common Spatial Pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian Spatial Pattern (RSP) method, which is based on the backward channel selection procedure.

APPROACH

The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers.

MAIN RESULTS

Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results.

SIGNIFICANCE

Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Distributed source modeling of intracranial stereoelectro-encephalographic measurements

Intracranial stereoelectroencephalography (sEEG) provides unsurpassed sensitivity and specificity for human neurophysiology. However, functional mapping of brain functions has been limited because the implantations have sparse coverage and differ greatly across individuals. Here, we developed a distributed, anatomically realistic sEEG source-modeling approach for within- and between-subject analyses. In addition to intracranial event-related potentials (iERP), we estimated the sources of high broadband gamma activity (HBBG), a putative correlate of local neural firing. Our novel approach accounted for a significant portion of the variance of the sEEG measurements in leave-one-out cross-validation. After logarithmic transformations, the sensitivity and signal-to-noise ratio were linearly inversely related to the minimal distance between the brain location and electrode contacts (slope≈-3.6). The signa-to-noise ratio and sensitivity in the thalamus and brain stem were comparable to those locations at the vicinity of electrode contact implantation. The HGGB source estimates were remarkably consistent with analyses of intracranial-contact data. In conclusion, distributed sEEG source modeling provides a powerful neuroimaging tool, which facilitates anatomically-normalized functional mapping of human brain using both iERP and HBBG data.

Functional MRI based simulations of ECoG grid configurations for optimal measurement of spatially distributed hand-gesture information

Objective In electrocorticography (ECoG), the physical characteristics of the electrode grid determine which aspect of the neurophysiology is measured. For particular cases, the ECoG grid may be tailored to capture specific features, such as in the development and use of brain-computer-interfaces (BCI). Neural representations of hand movement are increasingly used to control ECoG based BCIs. However, it remains unclear which grid configurations are the most optimal to capture the dynamics of hand gesture information. Here, we investigate how the design and surgical placement of grids would affect the usability of ECoG measurements. Approach High resolution 7T functional MRI was used as a proxy for neural activity in ten healthy participants to simulate various grid configurations, and evaluated the performance of each configuration for decoding hand gestures. The grid configurations varied in number of electrodes, electrode distance and electrode size. Main results Optimal decoding of hand gestures occurred in grid configurations with a higher number of densely-packed, large-size, electrodes up to a grid of ~5x5 electrodes. When restricting the grid placement to a highly informative region of primary sensorimotor cortex, optimal parameters converged to about 3x3 electrodes, an inter-electrode distance of 8mm, and an electrode size of 3mm radius (performing at ~70% 3-class classification accuracy). Significance Our approach might be used to identify the most informative region, find the optimal grid configuration and assist in positioning of the grid to achieve high BCI performance for the decoding of hand-gestures prior to surgical implantation.

Refinement of High-Gamma EEG Features From TBI Patients With Hemicraniectomy Using an ICA Informed by Simulated Myoelectric Artifacts

Recent studies have shown the ability to record high-γ signals (80-160 Hz) in electroencephalogram (EEG) from traumatic brain injury (TBI) patients who have had hemicraniectomies. However, extraction of the movement-related high-γ remains challenging due to a confounding bandwidth overlap with surface electromyogram (EMG) artifacts related to facial and head movements. In our previous work, we described an augmented independent component analysis (ICA) approach for removal of EMG artifacts from EEG, and referred to as EMG Reduction by Adding Sources of EMG (ERASE). Here, we tested this algorithm on EEG recorded from six TBI patients with hemicraniectomies while they performed a thumb flexion task. ERASE removed a mean of 52 ± 12% (mean ± S.E.M) (maximum 73%) of EMG artifacts. In contrast, conventional ICA removed a mean of 27 ± 19% (mean ± S.E.M) of EMG artifacts from EEG. In particular, high-γ synchronization was significantly improved in the contralateral hand motor cortex area within the hemicraniectomy site after ERASE was applied. A more sophisticated measure of high-γ complexity is the fractal dimension (FD). Here, we computed the FD of EEG high-γ on each channel. Relative FD of high-γ was defined as that the FD in move state was subtracted by FD in idle state. We found relative FD of high-γ over hemicraniectomy after applying ERASE were strongly correlated to the amplitude of finger flexion force. Results showed that significant correlation coefficients across the electrodes related to thumb flexion averaged ~0.76, while the coefficients across the homologous electrodes in non-hemicraniectomy areas were nearly 0. After conventional ICA, a correlation between relative FD of high-γ and force remained high in both hemicraniectomy areas (up to 0.86) and non-hemicraniectomy areas (up to 0.81). Across all subjects, an average of 83% of electrodes significantly correlated with force was located in the hemicraniectomy areas after applying ERASE. After conventional ICA, only 19% of electrodes with significant correlations were located in the hemicraniectomy. These results indicated that the new approach isolated electrophysiological features during finger motor activation while selectively removing confounding EMG artifacts. This approach removed EMG artifacts that can contaminate high-gamma activity recorded over the hemicraniectomy.

Movement speed effects on beta-band oscillations in sensorimotor cortex during voluntary activity

Beta-band oscillations are a dominant feature in the sensorimotor system, which includes movement-related beta desynchronization (MRBD) during the preparation and execution phases of movement and postmovement beta synchronization (PMBS) on movement cessation. Many studies have linked this rhythm to motor functions. However, its associations to the movement speed are still unclear. We make a hypothesis that PMBS will be modulated with increasing of movement speeds. We assessed the MRBD and PMBS during isotonic slower self-paced and ballistic movements with 15 healthy subjects. Furthermore, we conduct an additional control experiment with the isometric contraction with two levels of forces to match those in the isotonic slower self-paced and ballistic movements separately. We found that the amplitude of PMBS but not MRBD in motor cortex is modulated by the speed during voluntary movement. PMBS was positively correlated with movement speed and acceleration through the partial correlation analysis. However, there were no changes in the PMBS and MRBD during the isometric contraction with two levels of forces. These results demonstrate a different function of PMBS and MRBD to the movement speed during voluntary activity and suggest that the movement speed would affect the amplitude of PMBS.NEW & NOTEWORTHY Beta-band oscillations are a dominant feature in the sensorimotor system that associate to the motor function. We found that the movement-related postmovement beta synchronization (PMBS) over the contralateral sensorimotor cortex was positively correlated with the speed of a voluntary movement, but the movement-related beta desynchronization (MRBD) was not. Our results show a differential response of the PMBS and MRBD to the movement speed during voluntary movement.

Chaotic behaviour of EEG responses with an identical grasp posture

Individuals with severe neuromuscular ailments can benefit from restoring their grasp activities with a brain-controlled upper-limb neuroprosthesis. EEG signals can be utilized as the driving source, and to implement natural human-like grasping abilities. Although good accuracy has already been achieved in classifying the various grasp patterns for specific sets of objects, unseen objects are still a hurdle in real-life implementation. Generalization of grasp patterns should be explored without any prior knowledge of the objects. In this regard, the similarity of motor imagery for different objects requiring similar grasp pattern can be utilized. It is also necessary to identify the brain regions that exhibit prominent distinguishability during different grasp patterns. In this study, we propose a chaos-based method to decode the motor imagery of two quite similar Power grasp patterns-cylindrical and spherical-for holding various objects. Three distinct suitable objects were chosen for each of the two patterns, and a 29-channel EEG was taken of 18 healthy participants to explore motor imagery for grasping the objects. Nonlinear correlation dimension was employed on the EEG data, at sub-band levels α, upper β, and γ, to analyse the distinguishability, as well as the similarity of grasp patterns for the objects. ANOVA was subsequently performed on the obtained CD parameters to identify the contribution of each electrode channel. Furthermore, using an SVM classifier, more than 80% accuracy was obtained in classifying the grasping patterns at the upper β sub-band. The outcome may lead to identification of optimum feature sets of motor imagery from specific brain regions for random objects grasps.

The future of upper extremity rehabilitation robotics: research and practice

The loss of upper limb motor function can have a devastating effect on people's lives. To restore upper limb control and functionality, researchers and clinicians have developed interfaces to interact directly with the human body's motor system. In this invited review, we aim to provide details on the peripheral nerve interfaces and brain-machine interfaces that have been developed in the past 30 years for upper extremity control, and we highlight the challenges that still remain to transition the technology into the clinical market. The findings show that peripheral nerve interfaces and brain-machine interfaces have many similar characteristics that enable them to be concurrently developed. Decoding neural information from both interfaces may lead to novel physiological models that may one day fully restore upper limb motor function for a growing patient population.

Spatial frequency tuning of motor responses reveals differential contribution of dorsal and ventral systems to action comprehension

Understanding object-directed actions performed by others is central to everyday life. This ability is thought to rely on the interaction between the dorsal action observation network (AON) and a ventral object recognition pathway. On this view, the AON would encode action kinematics, and the ventral pathway, the most likely intention afforded by the objects. However, experimental evidence supporting this model is still scarce. Here, we aimed to disentangle the contribution of dorsal vs. ventral pathways to action comprehension by exploiting their differential tuning to low-spatial frequencies (LSFs) and high-spatial frequencies (HSFs). We filtered naturalistic action images to contain only LSF or HSF and measured behavioral performance and corticospinal excitability (CSE) using transcranial magnetic stimulation (TMS). Actions were embedded in congruent or incongruent scenarios as defined by the compatibility between grips and intentions afforded by the contextual objects. Behaviorally, participants were better at discriminating congruent actions in intact than LSF images. This effect was reversed for incongruent actions, with better performance for LSF than intact and HSF. These modulations were mirrored at the neurophysiological level, with greater CSE facilitation for congruent than incongruent actions for HSF and the opposite pattern for LSF images. Finally, only for LSF did we observe CSE modulations according to grip kinematics. While results point to differential dorsal (LSF) and ventral (HSF) contributions to action comprehension for grip and context encoding, respectively, the negative congruency effect for LSF images suggests that object processing may influence action perception not only through ventral-to-dorsal connections, but also through a dorsal-to-dorsal route involved in predictive processing.

Intent Prediction Based on Biomechanical Coordination of EMG and Vision-Filtered Gaze for End-Point Control of an Arm Prosthesis

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during pick-and-place trials with 7 able-bodied subjects. Subjects positioned an object above a random target on a virtual interface, each completing around 600 trials. On average across all trials and subjects gaze preceded EMG and followed a repeatable pattern that allowed for prediction. A computer vision algorithm was used to extract the initial and target fixations and estimate the target position in 2D space. Two SVRs were trained with EMG data to predict the x- and y- position of the hand; results showed that the y-estimate was significantly better than the x-estimate. The EMG and gaze predictions were fused using a Kalman Filter-based approach, and the positional error from using EMG-only was significantly higher than the fusion of EMG and gaze. The final target position Root Mean Squared Error (RMSE) decreased from 9.28 cm with an EMG-only prediction to 6.94 cm when using a gaze-EMG fusion. This error also increased significantly when removing some or all arm muscle signals. However, using fused EMG and gaze, there were no significant difference between predictors that included all muscles, or only a subset of muscles.

A state-based probabilistic method for decoding hand position during movement from ECoG signals in non-human primate

OBJECTIVE

In this study, we proposed a state-based probabilistic method for decoding hand positions during unilateral and bilateral movements using the ECoG signals recorded from the brain of Rhesus monkey.

APPROACH

A customized electrode array was implanted subdurally in the right hemisphere of the brain covering from the primary motor cortex to the frontal cortex. Three different experimental paradigms were considered: ipsilateral, contralateral, and bilateral movements. During unilateral movement, the monkey was trained to get food with one hand, while during bilateral movement, the monkey used its left and right hands alternately to get food. To estimate the hand positions, a state-based probabilistic method was introduced which was based on the conditional probability of the hand movement state (i.e. idle, right hand movement, and left hand movement) and the conditional expectation of the hand position for each state. Moreover, a hybrid feature extraction method based on linear discriminant analysis and partial least squares (PLS) was introduced.

MAIN RESULTS

The proposed method could successfully decode the hand positions during ipsilateral, contralateral, and bilateral movements and significantly improved the decoding performance compared to the conventional Kalman and PLS regression methods [Formula: see text]. The proposed hybrid feature extraction method was found to outperform both the PLS and PCA methods [Formula: see text]. Investigating the kinematic information of each frequency band shows that more informative frequency bands were [Formula: see text] (15-30 Hz) and [Formula: see text](50-100 Hz) for ipsilateral and [Formula: see text] and [Formula: see text] (100-200 Hz) for contralateral movements. It is observed that ipsilateral movement was decoded better than contralateral movement for [Formula: see text] (5-15 Hz) and [Formula: see text] bands, while contralateral movements was decoded better for [Formula: see text] (30-200 Hz) and hfECoG (200-400 Hz) bands.

SIGNIFICANCE

Accurate decoding the bilateral movement using the ECoG recorded from one brain hemisphere is an important issue toward real-life applications of the brain-machine interface technologies.

Language and motor processing in reading and typing: Insights from beta-frequency band power modulations

Power modulations of the EEG activity within the beta-frequency band were investigated across silent-reading and copy-typing tasks featuring emotionally negative and neutral words in order to clarify the interplay between language and motor processing. In reading, a single desynchronization surfaced 200-600 ms after target presentation, with a stronger power-decrease in lower beta frequencies for neutral compared to negative words. The typing task revealed two distinct desynchronizations. A first one surfaced within spatio-temporal coordinates closely resembling those of the desynchronization observed in the reading task, thus pointing towards a common origin at the level of linguistic processing of the input word stimuli. Additionally, a second motor-related desynchronization surfaced during the typed response, from 700 to 2000 ms after stimulus onset. Here, words' emotional connotation affected the higher beta band. The comparison between tasks thus suggests that different beta desynchronizations reflect distinct EEG landmarks for language and motor processing. Further, the effect of emotional connotation on the motor-related desynchronization of the typing task suggests that language processing can propagate its influence onto the stage of motor response execution, pointing against a serial flow of information from language onto motor processing.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Power Modulations of ECoG Alpha/Beta and Gamma Bands Correlate With Time-Derivative of Force During Hand Grasp

It is well-known that motor cortical oscillatory components are modulated in their amplitude during voluntary and imagined movements. These patterns have been used to develop brain-machine interfaces (BMI) which focused mostly on movement kinematics. In contrast, there have been only a few studies on the relation between brain oscillatory activity and the control of force, in particular, grasping force, which is of primary importance for common daily activities. In this study, we recorded intraoperative high-density electrocorticography (ECoG) from the sensorimotor cortex of four patients while they executed a voluntary isometric hand grasp following verbal instruction. The grasp was held for 2 to 3 s before being instructed to relax. We studied the power modulations of neural oscillations during the whole time-course of the grasp (onset, hold, and offset phases). Phasic event-related desynchronization (ERD) in the low-frequency band (LFB) from 8 to 32 Hz and event-related synchronization (ERS) in the high-frequency band (HFB) from 60 to 200 Hz were observed at grasp onset and offset. However, during the grasp holding period, the magnitude of LFB-ERD and HFB-ERS decreased near or at the baseline level. Overall, LFB-ERD and HFB-ERS show phasic characteristics related to the changes of grasp force (onset/offset) in all four patients. More precisely, the fluctuations of HFB-ERS primarily, and of LFB-ERD to a lesser extent, correlated with the time-course of the first time-derivative of force (yank), rather than with force itself. To the best of our knowledge, this is the first study that establishes such a correlation. These results have fundamental implications for the decoding of grasp in brain oscillatory activity-based neuroprosthetics.

Applications of brain-computer interfaces to the control of robotic and prosthetic arms

Brain-computer interfaces (BCIs) have the potential to improve the quality of life of individuals with severe motor disabilities. BCIs capture the user's brain activity and translate it into commands for the control of an effector, such as a computer cursor, robotic limb, or functional electrical stimulation device. Full dexterous manipulation of robotic and prosthetic arms via a BCI system has been a challenge because of the inherent need to decode high dimensional and preferably real-time control commands from the user's neural activity. Nevertheless, such functionality is fundamental if BCI-controlled robotic or prosthetic limbs are to be used for daily activities. In this chapter, we review how this challenge has been addressed by BCI researchers and how new solutions may improve the BCI user experience with robotic effectors.

Decoding Movement From Electrocorticographic Activity: A Review

Electrocorticography (ECoG) holds promise to provide efficient neuroprosthetic solutions for people suffering from neurological disabilities. This recording technique combines adequate temporal and spatial resolution with the lower risks of medical complications compared to the other invasive methods. ECoG is routinely used in clinical practice for preoperative cortical mapping in epileptic patients. During the last two decades, research utilizing ECoG has considerably grown, including the paradigms where behaviorally relevant information is extracted from ECoG activity with decoding algorithms of different complexity. Several research groups have advanced toward the development of assistive devices driven by brain-computer interfaces (BCIs) that decode motor commands from multichannel ECoG recordings. Here we review the evolution of this field and its recent tendencies, and discuss the potential areas for future development.

A novel algorithm for removing artifacts from EEG data

In recent years, many studies examined if EEG signals from traumatic brain injury (TBI) patients can be used for new rehabilitation technologies, such as BCI systems. However, extraction of the high-gamma band related to movement remains challenging due to the presence of surface electromyogram (sEMG) caused by unconscious facial and head movement of patients. In this paper, we proposed a modified independent component analysis (ICA) model for EMG artifact removal in the EEG data from TBI patients with a hemicraniectomy. Here, simulated EMG was generated and added to the raw EEG data as the extra channels for independent components calculation. After running ICA, the independent components (ICs) related to artifacts were identified and rejected automatically through several criteria. EEG data underlying hand movement from one healthy subject and one TBI patient with a hemicraniectomy were conducted to verify the efficacy of this algorithm. Results showed that the proposed algorithm removed sEMG artifacts from the EEG data by up to 86.72% while preserving the associated brain features. In particular, the high-gamma band (80 to 160 Hz) was found to arise principally from the hemicraniectomy area after this technique was applied. Meanwhile, we found that the magnitude of gamma power during movement improved after removal of sEMG artifacts.