Decoding natural grasp types from human ECoG

Continuous decoding of human grasp kinematics using epidural and subdural signals

OBJECTIVE

Restoring or replacing function in paralyzed individuals will one day be achieved through the use of brain-machine interfaces. Regaining hand function is a major goal for paralyzed patients. Two competing prerequisites for the widespread adoption of any hand neuroprosthesis are accurate control over the fine details of movement, and minimized invasiveness. Here, we explore the interplay between these two goals by comparing our ability to decode hand movements with subdural and epidural field potentials (EFPs).

APPROACH

We measured the accuracy of decoding continuous hand and finger kinematics during naturalistic grasping motions in five human subjects. We recorded subdural surface potentials (electrocorticography; ECoG) as well as with EFPs, with both standard- and high-resolution electrode arrays.

MAIN RESULTS

In all five subjects, decoding of continuous kinematics significantly exceeded chance, using either EGoG or EFPs. ECoG decoding accuracy compared favorably with prior investigations of grasp kinematics (mean ± SD grasp aperture variance accounted for was 0.54 ± 0.05 across all subjects, 0.75 ± 0.09 for the best subject). In general, EFP decoding performed comparably to ECoG decoding. The 7-20 Hz and 70-115 Hz spectral bands contained the most information about grasp kinematics, with the 70-115 Hz band containing greater information about more subtle movements. Higher-resolution recording arrays provided clearly superior performance compared to standard-resolution arrays.

SIGNIFICANCE

To approach the fine motor control achieved by an intact brain-body system, it will be necessary to execute motor intent on a continuous basis with high accuracy. The current results demonstrate that this level of accuracy might be achievable not just with ECoG, but with EFPs as well. Epidural placement of electrodes is less invasive, and therefore may incur less risk of encephalitis or stroke than subdural placement of electrodes. Accurately decoding motor commands at the epidural level may be an important step towards a clinically viable brain-machine interface.

Neural pattern similarity between contra- and ipsilateral movements in high-frequency band of human electrocorticograms

The cortical motor areas are activated not only during contralateral limb movements but also during ipsilateral limb movements. Although these ipsilateral activities have been observed in several brain imaging studies, their functional role is poorly understood. Due to its high temporal resolution and low susceptibility to artifacts from body movements, the electrocorticogram (ECoG) is an advantageous measurement method for assessing the human brain function of motor behaviors. Here, we demonstrate that contra- and ipsilateral movements share a similarity in the high-frequency band of human ECoG signals. The ECoG signals were measured from the unilateral sensorimotor cortex while patients conducted self-paced movements of different body parts, contra- or ipsilateral to the measurement side. The movement categories (wrist, shoulder, or ankle) of ipsilateral movements were decoded as accurately as those of contralateral movements from spatial patterns of the high-frequency band of the precentral motor area (the primary motor and premotor areas). The decoder, trained in the high-frequency band of ipsilateral movements generalized to contralateral movements, and vice versa, confirmed that the activity patterns related to ipsilateral limb movements were similar to contralateral ones in the precentral motor area. Our results suggest that the high-frequency band activity patterns of ipsilateral and contralateral movements might be functionally coupled to control limbs, even during unilateral movements.

Decoding gripping force based on local field potentials recorded from subthalamic nucleus in humans

The basal ganglia are known to be involved in the planning, execution and control of gripping force and movement vigour. Here we aim to define the nature of the basal ganglia control signal for force and to decode gripping force based on local field potential (LFP) activities recorded from the subthalamic nucleus (STN) in patients with deep brain stimulation (DBS) electrodes. We found that STN LFP activities in the gamma (55-90 Hz) and beta (13-30m Hz) bands were most informative about gripping force, and that a first order dynamic linear model with these STN LFP features as inputs can be used to decode the temporal profile of gripping force. Our results enhance the understanding of how the basal ganglia control gripping force, and also suggest that deep brain LFPs could potentially be used to decode movement parameters related to force and movement vigour for the development of advanced human-machine interfaces.

Continuous Force Decoding from Local Field Potentials of the Primary Motor Cortex in Freely Moving Rats

Local field potential (LFP) signals recorded by intracortical microelectrodes implanted in primary motor cortex can be used as a high informative input for decoding of motor functions. Recent studies show that different kinematic parameters such as position and velocity can be inferred from multiple LFP signals as precisely as spiking activities, however, continuous decoding of the force magnitude from the LFP signals in freely moving animals has remained an open problem. Here, we trained three rats to press a force sensor for getting a drop of water as a reward. A 16-channel micro-wire array was implanted in the primary motor cortex of each trained rat, and obtained LFP signals were used for decoding of the continuous values recorded by the force sensor. Average coefficient of correlation and the coefficient of determination between decoded and actual force signals were r = 0.66 and R² = 0.42, respectively. We found that LFP signal on gamma frequency bands (30–120 Hz) had the most contribution in the trained decoding model. This study suggests the feasibility of using low number of LFP channels for the continuous force decoding in freely moving animals resembling BMI systems in real life applications.

Minimum Noise Estimate filter: a Novel Automated Artifacts Removal method for Field Potentials

In this paper a novel automated and unsupervised method for removing artifacts from multichannel field potential signals is introduced which can be used in brain computer interface (BCI) applications. The method, which is called minimum noise estimate (MNE) filter is based on an iterative thresholding followed by Rayleigh quotient which tries to find an estimate of the noise and to minimize it over the original signal. MNE filter is capable to operate without any prior information about field potential signals. The performance of the proposed method is evaluated by its application on two different type of signals namely electrocorticogram (ECoG) and electroencephalogram (EEG) datasets through a decoding procedure. The results indicate that the proposed method outperforms well-known artifacts removal techniques such as common average referencing (CAR), Laplacian method, independent component analysis (ICA) and wavelet denoising approach.

Towards decoding of functional movements from the same limb using EEG

In recent years, there has been an increasing interest in using electroencephalographic (EEG) activity to close the loop between brain oscillations and movement to induce functional motor rehabilitation. Rehabilitation robots or exoskeletons have been controlled using EEG activity. However, all studies have used a 2-class or one-dimensional decoding scheme. In this study we investigated EEG decoding of 5 functional movements of the same limb towards an online scenario. Six healthy participants performed a three-dimensional center-out reaching task based on direction movements (four directions and rest) wearing a 32-channel EEG cap. A BCI design based on multiclass extensions of Spectrally Weighted Common Spatial Patterns (Spec-CSP) and a linear discriminant analysis (LDA) classifier was developed and tested offline. The decoding accuracy was 5-fold cross-validated. A decoding accuracy of 39.5% on average for all the six subjects was obtained (chance level being 20%). The results of the current study demonstrate multiple functional movements decoding (significantly higher than chance level) from the same limb using EEG data. This study represents first steps towards a same limb multi degree of freedom (DOF) online EEG based BCI for motor restoration.

CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave

Recent years have seen an increase in the popularity of multivariate pattern (MVP) analysis of functional magnetic resonance (fMRI) data, and, to a much lesser extent, magneto- and electro-encephalography (M/EEG) data. We present CoSMoMVPA, a lightweight MVPA (MVP analysis) toolbox implemented in the intersection of the Matlab and GNU Octave languages, that treats both fMRI and M/EEG data as first-class citizens. CoSMoMVPA supports all state-of-the-art MVP analysis techniques, including searchlight analyses, classification, correlations, representational similarity analysis, and the time generalization method. These can be used to address both data-driven and hypothesis-driven questions about neural organization and representations, both within and across: space, time, frequency bands, neuroimaging modalities, individuals, and species. It uses a uniform data representation of fMRI data in the volume or on the surface, and of M/EEG data at the sensor and source level. Through various external toolboxes, it directly supports reading and writing a variety of fMRI and M/EEG neuroimaging formats, and, where applicable, can convert between them. As a result, it can be integrated readily in existing pipelines and used with existing preprocessed datasets. CoSMoMVPA overloads the traditional volumetric searchlight concept to support neighborhoods for M/EEG and surface-based fMRI data, which supports localization of multivariate effects of interest across space, time, and frequency dimensions. CoSMoMVPA also provides a generalized approach to multiple comparison correction across these dimensions using Threshold-Free Cluster Enhancement with state-of-the-art clustering and permutation techniques. CoSMoMVPA is highly modular and uses abstractions to provide a uniform interface for a variety of MVP measures. Typical analyses require a few lines of code, making it accessible to beginner users. At the same time, expert programmers can easily extend its functionality. CoSMoMVPA comes with extensive documentation, including a variety of runnable demonstration scripts and analysis exercises (with example data and solutions). It uses best software engineering practices including version control, distributed development, an automated test suite, and continuous integration testing. It can be used with the proprietary Matlab and the free GNU Octave software, and it complies with open source distribution platforms such as NeuroDebian. CoSMoMVPA is Free/Open Source Software under the permissive MIT license.

Website: http://cosmomvpa.org

Source code: https://github.com/CoSMoMVPA/CoSMoMVPA

Biomimetic rehabilitation engineering: the importance of somatosensory feedback for brain-machine interfaces

OBJECTIVE

Brain-machine interfaces (BMIs) re-establish communication channels between the nervous system and an external device. The use of BMI technology has generated significant developments in rehabilitative medicine, promising new ways to restore lost sensory-motor functions. However and despite high-caliber basic research, only a few prototypes have successfully left the laboratory and are currently home-deployed.

APPROACH

The failure of this laboratory-to-user transfer likely relates to the absence of BMI solutions for providing naturalistic feedback about the consequences of the BMI's actions. To overcome this limitation, nowadays cutting-edge BMI advances are guided by the principle of biomimicry; i.e. the artificial reproduction of normal neural mechanisms.

MAIN RESULTS

Here, we focus on the importance of somatosensory feedback in BMIs devoted to reproducing movements with the goal of serving as a reference framework for future research on innovative rehabilitation procedures. First, we address the correspondence between users' needs and BMI solutions. Then, we describe the main features of invasive and non-invasive BMIs, including their degree of biomimicry and respective advantages and drawbacks. Furthermore, we explore the prevalent approaches for providing quasi-natural sensory feedback in BMI settings. Finally, we cover special situations that can promote biomimicry and we present the future directions in basic research and clinical applications.

SIGNIFICANCE

The continued incorporation of biomimetic features into the design of BMIs will surely serve to further ameliorate the realism of BMIs, as well as tremendously improve their actuation, acceptance, and use.

Unsupervised Decoding of Long-Term, Naturalistic Human Neural Recordings with Automated Video and Audio Annotations

Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Implementing Brain Computer Interfaces (BCIs) outside carefully controlled experiments in laboratory settings requires adaptive and scalable strategies with minimal supervision. Here we describe an unsupervised approach to decoding neural states from naturalistic human brain recordings. We analyzed continuous, long-term electrocorticography (ECoG) data recorded over many days from the brain of subjects in a hospital room, with simultaneous audio and video recordings. We discovered coherent clusters in high-dimensional ECoG recordings using hierarchical clustering and automatically annotated them using speech and movement labels extracted from audio and video. To our knowledge, this represents the first time techniques from computer vision and speech processing have been used for natural ECoG decoding. Interpretable behaviors were decoded from ECoG data, including moving, speaking and resting; the results were assessed by comparison with manual annotation. Discovered clusters were projected back onto the brain revealing features consistent with known functional areas, opening the door to automated functional brain mapping in natural settings.

Modulation of post-movement beta rebound by contraction force and rate of force development

Movement induced modulation of the beta rhythm is one of the most robust neural oscillatory phenomena in the brain. In the preparation and execution phases of movement, a loss in beta amplitude is observed [movement related beta decrease (MRBD)]. This is followed by a rebound above baseline on movement cessation [post movement beta rebound (PMBR)]. These effects have been measured widely, and recent work suggests that they may have significant importance. Specifically, they have potential to form the basis of biomarkers for disease, and have been used in neuroscience applications ranging from brain computer interfaces to markers of neural plasticity. However, despite the robust nature of both MRBD and PMBR, the phenomena themselves are poorly understood. In this study, we characterise MRBD and PMBR during a carefully controlled isometric wrist flexion paradigm, isolating two fundamental movement parameters; force output, and the rate of force development (RFD). Our results show that neither altered force output nor RFD has a significant effect on MRBD. In contrast, PMBR was altered by both parameters. Higher force output results in greater PMBR amplitude, and greater RFD results in a PMBR which is higher in amplitude and shorter in duration. These findings demonstrate that careful control of movement parameters can systematically change PMBR. Further, for temporally protracted movements, the PMBR can be over 7 s in duration. This means accurate control of movement and judicious selection of paradigm parameters are critical in future clinical and basic neuroscientific studies of sensorimotor beta oscillations. Hum Brain Mapp 37:2493–2511, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc

Contributions of Subsurface Cortical Modulations to Discrimination of Executed and Imagined Grasp Forces through Stereoelectroencephalography

Stereoelectroencephalographic (SEEG) depth electrodes have the potential to record neural activity from deep brain structures not easily reached with other intracranial recording technologies. SEEG electrodes were placed through deep cortical structures including central sulcus and insular cortex. In order to observe changes in frequency band modulation, participants performed force matching trials at three distinct force levels using two different grasp configurations: a power grasp and a lateral pinch. Signals from these deeper structures were found to contain information useful for distinguishing force from rest trials as well as different force levels in some participants. High frequency components along with alpha and beta bands recorded from electrodes located near the primary motor cortex wall of central sulcus and electrodes passing through sensory cortex were found to be the most useful for classification of force versus rest although one participant did have significant modulation in the insular cortex. This study electrophysiologically corroborates with previous imaging studies that show force-related modulation occurs inside of central sulcus and insular cortex. The results of this work suggest that depth electrodes could be useful tools for investigating the functions of deeper brain structures as well as showing that central sulcus and insular cortex may contain neural signals that could be used for control of a grasp force BMI.

Force Myography to Control Robotic Upper Extremity Prostheses: A Feasibility Study

Advancement in assistive technology has led to the commercial availability of multi-dexterous robotic prostheses for the upper extremity. The relatively low performance of the currently used techniques to detect the intention of the user to control such advanced robotic prostheses, however, limits their use. This article explores the use of force myography (FMG) as a potential alternative to the well-established surface electromyography. Specifically, the use of FMG to control different grips of a commercially available robotic hand, Bebionic3, is investigated. Four male transradially amputated subjects participated in the study, and a protocol was developed to assess the prediction accuracy of 11 grips. Different combinations of grips were examined, ranging from 6 up to 11 grips. The results indicate that it is possible to classify six primary grips important in activities of daily living using FMG with an accuracy of above 70% in the residual limb. Additional strategies to increase classification accuracy, such as using the available modes on the Bebionic3, allowed results to improve up to 88.83 and 89.00% for opposed thumb and non-opposed thumb modes, respectively.

Real-Time Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals from Paralysed Patients

Neuroprosthetic arms might potentially restore motor functions for severely paralysed patients. Invasive measurements of cortical currents using electrocorticography have been widely used for neuroprosthetic control. Moreover, magnetoencephalography (MEG) exhibits characteristic brain signals similar to those of invasively measured signals. However, it remains unclear whether non-invasively measured signals convey enough motor information to control a neuroprosthetic hand, especially for severely paralysed patients whose sensorimotor cortex might be reorganized. We tested an MEG-based neuroprosthetic system to evaluate the accuracy of using cortical currents in the sensorimotor cortex of severely paralysed patients to control a prosthetic hand. The patients attempted to grasp with or open their paralysed hand while the slow components of MEG signals (slow movement fields; SMFs) were recorded. Even without actual movements, the SMFs of all patients indicated characteristic spatiotemporal patterns similar to actual movements, and the SMFs were successfully used to control a neuroprosthetic hand in a closed-loop condition. These results demonstrate that the slow components of MEG signals carry sufficient information to classify movement types. Successful control by paralysed patients suggests the feasibility of using an MEG-based neuroprosthetic hand to predict a patient's ability to control an invasive neuroprosthesis via the same signal sources as the non-invasive method.

Bicycling and Walking are Associated with Different Cortical Oscillatory Dynamics

Although bicycling and walking involve similar complex coordinated movements, surprisingly Parkinson’s patients with freezing of gait typically remain able to bicycle despite severe difficulties in walking. This observation suggests functional differences in the motor networks subserving bicycling and walking. However, a direct comparison of brain activity related to bicycling and walking has never been performed, neither in healthy participants nor in patients. Such a comparison could potentially help elucidating the cortical involvement in motor control and the mechanisms through which bicycling ability may be preserved in patients with freezing of gait. The aim of this study was to contrast the cortical oscillatory dynamics involved in bicycling and walking in healthy participants. To this end, EEG and EMG data of 14 healthy participants were analyzed, who cycled on a stationary bicycle at a slow cadence of 40 revolutions per minute (rpm) and walked at 40 strides per minute (spm), respectively. Relative to walking, bicycling was associated with a stronger power decrease in the high beta band (23–35 Hz) during movement initiation and execution, followed by a stronger beta power increase after movement termination. Walking, on the other hand, was characterized by a stronger and persisting alpha power (8–12 Hz) decrease. Both bicycling and walking exhibited movement cycle-dependent power modulation in the 24–40 Hz range that was correlated with EMG activity. This modulation was significantly stronger in walking. The present findings reveal differential cortical oscillatory dynamics in motor control for two types of complex coordinated motor behavior, i.e., bicycling and walking. Bicycling was associated with a stronger sustained cortical activation as indicated by the stronger high beta power decrease during movement execution and less cortical motor control within the movement cycle. We speculate this to be due to the more continuous nature of bicycling demanding less phase-dependent sensory processing and motor planning, as opposed to walking.

A synergy-based hand control is encoded in human motor cortical areas

How the human brain controls hand movements to carry out different tasks is still debated. The concept of synergy has been proposed to indicate functional modules that may simplify the control of hand postures by simultaneously recruiting sets of muscles and joints. However, whether and to what extent synergic hand postures are encoded as such at a cortical level remains unknown. Here, we combined kinematic, electromyography, and brain activity measures obtained by functional magnetic resonance imaging while subjects performed a variety of movements towards virtual objects. Hand postural information, encoded through kinematic synergies, were represented in cortical areas devoted to hand motor control and successfully discriminated individual grasping movements, significantly outperforming alternative somatotopic or muscle-based models. Importantly, hand postural synergies were predicted by neural activation patterns within primary motor cortex. These findings support a novel cortical organization for hand movement control and open potential applications for brain-computer interfaces and neuroprostheses.

DOI:http://dx.doi.org/10.7554/eLife.13420.001

The human hand can perform an enormous range of movements with great dexterity. Some common everyday actions, such as grasping a coffee cup, involve the coordinated movement of all four fingers and thumb. Others, such as typing, rely on the ability of individual fingers to move relatively independently of one another.

This flexibility is possible in part because of the complex anatomy of the hand, with its 27 bones and their connecting joints and muscles. But with this complexity comes a huge number of possibilities. Any movement-related task – such as picking up a cup – can be achieved via many different combinations of muscle contractions and joint positions. So how does the brain decide which muscles and joints to use?

One theory is that the brain simplifies this problem by encoding particularly useful patterns of joint movements as distinct units or “synergies”. A given task can then be performed by selecting from a small number of synergies, avoiding the need to choose between huge numbers of options every time movement is required.

Leo et al. now provide the first direct evidence for the encoding of synergies by the human brain. Volunteers lying inside a brain scanner reached towards virtual objects – from tennis rackets to toothpicks – while activity was recorded from the area of the brain that controls hand movements. As predicted, the scans showed specific and reproducible patterns of activity. Analysing these patterns revealed that each corresponded to a particular combination of joint positions. These activity patterns, or synergies, could even be ‘decoded’ to work out which type of movement a volunteer had just performed.

Future experiments should examine how the brain combines synergies with sensory feedback to allow movements to be adjusted as they occur. Such findings could help to develop brain-computer interfaces and systems for controlling the movement of artificial limbs.

DOI:http://dx.doi.org/10.7554/eLife.13420.002

Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject

OBJECTIVE

We used native sensorimotor representations of fingers in a brain-machine interface (BMI) to achieve immediate online control of individual prosthetic fingers.

APPROACH

Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was moving, and, if so, (2) which digit was moving. To account for sensory feedback, we also mapped the spatiotemporal activation elicited by vibrotactile stimulation. Finally, we used this prediction framework to provide immediate online control over individual fingers of the Johns Hopkins University Applied Physics Laboratory modular prosthetic limb.

MAIN RESULTS

The balanced classification accuracy for detection of movements during the online control session was 92% (chance: 50%). At the onset of movement, finger classification was 76% (chance: 20%), and 88% (chance: 25%) if the pinky and ring finger movements were coupled. Balanced accuracy of fully flexing the cued finger was 64%, and 77% had we combined pinky and ring commands. Offline decoding yielded a peak finger decoding accuracy of 96.5% (chance: 20%) when using an optimized selection of electrodes. Offline analysis demonstrated significant finger-specific activations throughout sensorimotor cortex. Activations either prior to movement onset or during sensory feedback led to discriminable finger control.

SIGNIFICANCE

Our results demonstrate the ability of ECoG-based BMIs to leverage the native functional anatomy of sensorimotor cortical populations to immediately control individual finger movements in real time.

Comparison of decoding resolution of standard and high-density electrocorticogram electrodes

OBJECTIVE

Electrocorticography (ECoG)-based brain-computer interface (BCI) is a promising platform for controlling arm prostheses. To restore functional independence, a BCI must be able to control arm prostheses along at least six degrees-of-freedoms (DOFs). Prior studies suggest that standard ECoG grids may be insufficient to decode multi-DOF arm movements. This study compared the ability of standard and high-density (HD) ECoG grids to decode the presence/absence of six elementary arm movements and the type of movement performed.

APPROACH

Three subjects implanted with standard grids (4 mm diameter, 10 mm spacing) and three with HD grids (2 mm diameter, 4 mm spacing) had ECoG signals recorded while performing the following movements: (1) pincer grasp/release, (2) wrist flexion/extension, (3) pronation/supination, (4) elbow flexion/extension, (5) shoulder internal/external rotation, and (6) shoulder forward flexion/extension. Data from the primary motor cortex were used to train a state decoder to detect the presence/absence of movement, and a six-class decoder to distinguish between these movements.

MAIN RESULTS

The average performances of the state decoders trained on HD ECoG data were superior (p = 3.05 × 10(-5)) to those of their standard grid counterparts across all combinations of the μ, β, low-γ, and high-γ frequency bands. The average best decoding error for HD grids was 2.6%, compared to 8.5% of standard grids (chance 50%). The movement decoders trained on HD ECoG data were superior (p = 3.05 × 10(-5)) to those based on standard ECoG across all band combinations. The average best decoding errors of 11.9% and 33.1% were obtained for HD and standard grids, respectively (chance error 83.3%). These improvements can be attributed to higher electrode density and signal quality of HD grids.

SIGNIFICANCE

Commonly used ECoG grids are inadequate for multi-DOF BCI arm prostheses. The performance gains by HD grids may eventually lead to independence-restoring BCI arm prosthesis.

Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands

The term 'synergy' - from the Greek synergia - means 'working together'. The concept of multiple elements working together towards a common goal has been extensively used in neuroscience to develop theoretical frameworks, experimental approaches, and analytical techniques to understand neural control of movement, and for applications for neuro-rehabilitation. In the past decade, roboticists have successfully applied the framework of synergies to create novel design and control concepts for artificial hands, i.e., robotic hands and prostheses. At the same time, robotic research on the sensorimotor integration underlying the control and sensing of artificial hands has inspired new research approaches in neuroscience, and has provided useful instruments for novel experiments. The ambitious goal of integrating expertise and research approaches in robotics and neuroscience to study the properties and applications of the concept of synergies is generating a number of multidisciplinary cooperative projects, among which the recently finished 4-year European project "The Hand Embodied" (THE). This paper reviews the main insights provided by this framework. Specifically, we provide an overview of neuroscientific bases of hand synergies and introduce how robotics has leveraged the insights from neuroscience for innovative design in hardware and controllers for biomedical engineering applications, including myoelectric hand prostheses, devices for haptics research, and wearable sensing of human hand kinematics. The review also emphasizes how this multidisciplinary collaboration has generated new ways to conceptualize a synergy-based approach for robotics, and provides guidelines and principles for analyzing human behavior and synthesizing artificial robotic systems based on a theory of synergies.

EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks

GOAL

Sensorimotor-based brain-computer interfaces (BCIs) have achieved successful control of real and virtual devices in up to three dimensions; however, the traditional sensor-based paradigm limits the intuitive use of these systems. Many control signals for state-of-the-art BCIs involve imagining the movement of body parts that have little to do with the output command, revealing a cognitive disconnection between the user's intent and the action of the end effector. Therefore, there is a need to develop techniques that can identify with high spatial resolution the self-modulated neural activity reflective of the actions of a helpful output device.

METHODS

We extend previous EEG source imaging (ESI) work to decoding natural hand/wrist manipulations by applying a novel technique to classifying four complex motor imaginations of the right hand: flexion, extension, supination, and pronation.

RESULTS

We report an increase of up to 18.6% for individual task classification and 12.7% for overall classification using the proposed ESI approach over the traditional sensor-based method.

CONCLUSION

ESI is able to enhance BCI performance of decoding complex right-hand motor imagery tasks.

SIGNIFICANCE

This study may lead to the development of BCI systems with naturalistic and intuitive motor imaginations, thus facilitating broad use of noninvasive BCIs.

Enabling Low-Power, Multi-Modal Neural Interfaces Through a Common, Low-Bandwidth Feature Space

Brain-Machine Interfaces (BMIs) have shown great potential for generating prosthetic control signals. Translating BMIs into the clinic requires fully implantable, wireless systems; however, current solutions have high power requirements which limit their usability. Lowering this power consumption typically limits the system to a single neural modality, or signal type, and thus to a relatively small clinical market. Here, we address both of these issues by investigating the use of signal power in a single narrow frequency band as a decoding feature for extracting information from electrocorticographic (ECoG), electromyographic (EMG), and intracortical neural data. We have designed and tested the Multi-modal Implantable Neural Interface (MINI), a wireless recording system which extracts and transmits signal power in a single, configurable frequency band. In prerecorded datasets, we used the MINI to explore low frequency signal features and any resulting tradeoff between power savings and decoding performance losses. When processing intracortical data, the MINI achieved a power consumption 89.7% less than a more typical system designed to extract action potential waveforms. When processing ECoG and EMG data, the MINI achieved similar power reductions of 62.7% and 78.8%. At the same time, using the single signal feature extracted by the MINI, we were able to decode all three modalities with less than a 9% drop in accuracy relative to using high-bandwidth, modality-specific signal features. We believe this system architecture can be used to produce a viable, cost-effective, clinical BMI.

Brain-controlled muscle stimulation for the restoration of motor function

Loss of the ability to move, as a consequence of spinal cord injury or neuromuscular disorder, has devastating consequences for the paralyzed individual, and great economic consequences for society. Functional electrical stimulation (FES) offers one means to restore some mobility to these individuals, improving not only their autonomy, but potentially their general health and well-being as well. FES uses electrical stimulation to cause the paralyzed muscles to contract. Existing clinical systems require the stimulation to be preprogrammed, with the patient typically using residual voluntary movement of another body part to trigger and control the patterned stimulation. The rapid development of neural interfacing in the past decade offers the promise of dramatically improved control for these patients, potentially allowing continuous control of FES through signals recorded from motor cortex, as the patient attempts to control the paralyzed body part. While application of these 'brain-machine interfaces' (BMIs) has undergone dramatic development for control of computer cursors and even robotic limbs, their use as an interface for FES has been much more limited. In this review, we consider both FES and BMI technologies and discuss the prospect for combining the two to provide important new options for paralyzed individuals.

Single trial detection of hand poses in human ECoG using CSP based feature extraction

Decoding brain activity of corresponding highlevel tasks may lead to an independent and intuitively controlled Brain-Computer Interface (BCI). Most of today's BCI research focuses on analyzing the electroencephalogram (EEG) which provides only limited spatial and temporal resolution. Derived electrocorticographic (ECoG) signals allow the investigation of spatially highly focused task-related activation within the high-gamma frequency band, making the discrimination of individual finger movements or complex grasping tasks possible. Common spatial patterns (CSP) are commonly used for BCI systems and provide a powerful tool for feature optimization and dimensionality reduction. This work focused on the discrimination of (i) three complex hand movements, as well as (ii) hand movement and idle state. Two subjects S1 and S2 performed single `open', `peace' and `fist' hand poses in multiple trials. Signals in the high-gamma frequency range between 100 and 500 Hz were spatially filtered based on a CSP algorithm for (i) and (ii). Additionally, a manual feature selection approach was tested for (i). A multi-class linear discriminant analysis (LDA) showed for (i) an error rate of 13.89 % / 7.22 % and 18.42 % / 1.17 % for S1 and S2 using manually / CSP selected features, where for (ii) a two class LDA lead to a classification error of 13.39 % and 2.33 % for S1 and S2, respectively.

Classifying multiple types of hand motions using electrocorticography during intraoperative awake craniotomy and seizure monitoring processes-case studies

In this work, some case studies were conducted to classify several kinds of hand motions from electrocorticography (ECoG) signals during intraoperative awake craniotomy & extraoperative seizure monitoring processes. Four subjects (P1, P2 with intractable epilepsy during seizure monitoring and P3, P4 with brain tumor during awake craniotomy) participated in the experiments. Subjects performed three types of hand motions (Grasp, Thumb-finger motion and Index-finger motion) contralateral to the motor cortex covered with ECoG electrodes. Two methods were used for signal processing. Method I: autoregressive (AR) model with burg method was applied to extract features, and additional waveform length (WL) feature has been considered, finally the linear discriminative analysis (LDA) was used as the classifier. Method II: stationary subspace analysis (SSA) was applied for data preprocessing, and the common spatial pattern (CSP) was used for feature extraction before LDA decoding process. Applying method I, the three-class accuracy of P1~P4 were 90.17, 96.00, 91.77, and 92.95% respectively. For method II, the three-class accuracy of P1~P4 were 72.00, 93.17, 95.22, and 90.36% respectively. This study verified the possibility of decoding multiple hand motion types during an awake craniotomy, which is the first step toward dexterous neuroprosthetic control during surgical implantation, in order to verify the optimal placement of electrodes. The accuracy during awake craniotomy was comparable to results during seizure monitoring. This study also indicated that ECoG was a promising approach for precise identification of eloquent cortex during awake craniotomy, and might form a promising BCI system that could benefit both patients and neurosurgeons.

Practice changes beta power at rest and its modulation during movement in healthy subjects but not in patients with Parkinson's disease

BACKGROUND

PD (Parkinson's disease) is characterized by impairments in cortical plasticity, in beta frequency at rest and in beta power modulation during movement (i.e., event-related ERS [synchronization] and ERD [desynchronization]). Recent results with experimental protocols inducing long-term potentiation in healthy subjects suggest that cortical plasticity phenomena might be reflected by changes of beta power recorded with EEG during rest. Here, we determined whether motor practice produces changes in beta power at rest and during movements in both healthy subjects and patients with PD. We hypothesized that such changes would be reduced in PD.

METHODS

We thus recorded EEG in patients with PD and age-matched controls before, during and after a 40-minute reaching task. We determined posttask changes of beta power at rest and assessed the progressive changes of beta ERD and ERS during the task over frontal and sensorimotor regions.

RESULTS

We found that beta ERS and ERD changed significantly with practice in controls but not in PD. In PD compared to controls, beta power at rest was greater over frontal sensors but posttask changes, like those during movements, were far less evident. In both groups, kinematic characteristics improved with practice; however, there was no correlation between such improvements and the changes in beta power.

CONCLUSIONS

We conclude that prolonged practice in a motor task produces use-dependent modifications that are reflected in changes of beta power at rest and during movement. In PD, such changes are significantly reduced; such a reduction might represent, at least partially, impairment of cortical plasticity.

Detecting intention to grasp during reaching movements from EEG

Brain-computer interfaces (BCI) have been shown to be a promising tool in rehabilitation and assistive scenarios. Within these contexts, brain signals can be decoded and used as commands for a robotic device, allowing to translate user's intentions into motor actions in order to support the user's impaired neuro-muscular system. Recently, it has been suggested that slow cortical potentials (SCPs), negative deflections in the electroencephalographic (EEG) signals peaking around one second before the initiation of movements, might be of interest because they offer an accurate time resolution for the provided feedback. Many state-of-the-art studies exploiting SCPs have focused on decoding intention of movements related to walking and arm reaching, but up to now few studies have focused on decoding the intention to grasp, which is of fundamental importance in upper-limb tasks. In this work, we present a technique that exploits EEG to decode grasping correlates during reaching movements. Results obtained with four subjects show the existence of SCPs prior to the execution of grasping movements and how they can be used to classify, with accuracy rates greater than 70% across all subjects, the intention to grasp. Using a sliding window approach, we have also demonstrated how this intention can be decoded on average around 400 ms before the grasp movements for two out of four subjects, and after the onset of grasp itself for the two other subjects.

Decoding the evolving grasping gesture from electroencephalographic (EEG) activity

Shared control is emerging as a likely strategy for controlling neuroprosthetic devices, in which users specify high level goals but the low-level implementation is carried out by the machine. In this context, predicting the discrete goal is necessary. Although grasping various objects is critical in determining independence in daily life of amputees, decoding of different grasp types from noninvasively recorded brain activity has not been investigated. Here we show results suggesting electroencephalography (EEG) is a feasible modality to extract information on grasp types from the user's brain activity. We found that the information about the intended grasp increases over the grasping movement, and is significantly greater than chance up to 200 ms before movement onset.

Closed-Loop Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals

Objective

A neuroprosthesis using a brain–machine interface (BMI) is a promising therapeutic option for severely paralyzed patients, but the ability to control it may vary among individual patients and needs to be evaluated before any invasive procedure is undertaken. We have developed a neuroprosthetic hand that can be controlled by magnetoencephalographic (MEG) signals to noninvasively evaluate subjects’ ability to control a neuroprosthesis.

Method

Six nonparalyzed subjects performed grasping or opening movements of their right hand while the slow components of the MEG signals (SMFs) were recorded in an open-loop condition. The SMFs were used to train two decoders to infer the timing and types of movement by support vector machine and Gaussian process regression. The SMFs were also used to calculate estimated slow cortical potentials (eSCPs) to identify the origin of motor information. Finally, using the trained decoders, the subjects controlled a neuroprosthetic hand in a closed-loop condition.

Results

The SMFs in the open-loop condition revealed movement-related cortical field characteristics and successfully inferred the movement type with an accuracy of 75.0 ± 12.9% (mean ± SD). In particular, the eSCPs in the sensorimotor cortex contralateral to the moved hand varied significantly enough among the movement types to be decoded with an accuracy of 76.5 ± 10.6%, which was significantly higher than the accuracy associated with eSCPs in the ipsilateral sensorimotor cortex (58.1 ± 13.7%; p = 0.0072, paired two-tailed Student’s t-test). Moreover, another decoder using SMFs successfully inferred when the accuracy was the greatest. Combining these two decoders allowed the neuroprosthetic hand to be controlled in a closed-loop condition.

Conclusions

Use of real-time MEG signals was shown to successfully control the neuroprosthetic hand. The developed system may be useful for evaluating movement-related slow cortical potentials of severely paralyzed patients to predict the efficacy of invasive BMI.

Local field potentials in primate motor cortex encode grasp kinetic parameters

Reach and grasp kinematics are known to be encoded in the spiking activity of neuronal ensembles and in local field potentials (LFPs) recorded from primate motor cortex during movement planning and execution. However, little is known, especially in LFPs, about the encoding of kinetic parameters, such as forces exerted on the object during the same actions. We implanted two monkeys with microelectrode arrays in the motor cortical areas MI and PMd to investigate encoding of grasp-related parameters in motor cortical LFPs during planning and execution of reach-and-grasp movements. We identified three components of the LFP that modulated during grasps corresponding to low (0.3-7Hz), intermediate (~10-~40Hz) and high (~80-250Hz) frequency bands. We show that all three components can be used to classify not only grip types but also object loads during planning and execution of a grasping movement. In addition, we demonstrate that all three components recorded during planning or execution can be used to continuously decode finger pressure forces and hand position related to the grasping movement. Low and high frequency components provide similar classification and decoding accuracies, which were substantially higher than those obtained from the intermediate frequency component. Our results demonstrate that intended reach and grasp kinetic parameters are encoded in multiple LFP bands during both movement planning and execution. These findings also suggest that the LFP is a reliable signal for the control of parameters related to object load and applied pressure forces in brain-machine interfaces.

Global cortical activity predicts shape of hand during grasping

Recent studies show that the amplitude of cortical field potentials is modulated in the time domain by grasping kinematics. However, it is unknown if these low frequency modulations persist and contain enough information to decode grasp kinematics in macro-scale activity measured at the scalp via electroencephalography (EEG). Further, it is unclear as to whether joint angle velocities or movement synergies are the optimal kinematics spaces to decode. In this offline decoding study, we infer from human EEG, hand joint angular velocities as well as synergistic trajectories as subjects perform natural reach-to-grasp movements. Decoding accuracy, measured as the correlation coefficient (r) between the predicted and actual movement kinematics, was r = 0.49 ± 0.02 across 15 hand joints. Across the first three kinematic synergies, decoding accuracies were r = 0.59 ± 0.04, 0.47 ± 0.06, and 0.32 ± 0.05. The spatial-temporal pattern of EEG channel recruitment showed early involvement of contralateral frontal-central scalp areas followed by later activation of central electrodes over primary sensorimotor cortical areas. Information content in EEG about the grasp type peaked at 250 ms after movement onset. The high decoding accuracies in this study are significant not only as evidence for time-domain modulation in macro-scale brain activity, but for the field of brain-machine interfaces as well. Our decoding strategy, which harnesses the neural “symphony” as opposed to local members of the neural ensemble (as in intracranial approaches), may provide a means of extracting information about motor intent for grasping without the need for penetrating electrodes and suggests that it may be soon possible to develop non-invasive neural interfaces for the control of prosthetic limbs.

Biomaterials. Electronic dura mater for long-term multimodal neural interfaces

The mechanical mismatch between soft neural tissues and stiff neural implants hinders the long-term performance of implantable neuroprostheses. Here, we designed and fabricated soft neural implants with the shape and elasticity of dura mater, the protective membrane of the brain and spinal cord. The electronic dura mater, which we call e-dura, embeds interconnects, electrodes, and chemotrodes that sustain millions of mechanical stretch cycles, electrical stimulation pulses, and chemical injections. These integrated modalities enable multiple neuroprosthetic applications. The soft implants extracted cortical states in freely behaving animals for brain-machine interface and delivered electrochemical spinal neuromodulation that restored locomotion after paralyzing spinal cord injury.

Temporal alignment of electrocorticographic recordings for upper limb movement

The detection of movement-related components of the brain activity is useful in the design of brain-machine interfaces. A common approach is to classify the brain activity into a number of templates or states. To find these templates, the neural responses are averaged over each movement task. For averaging to be effective, one must assume that the neural components occur at identical times over repeated trials. However, complex arm movements such as reaching and grasping are prone to cross-trial variability due to the way movements are performed. Typically initiation time, duration of movement and movement speed are variable even as a subject tries to reproduce the same task identically across trials. Therefore, movement-related neural activity will tend to occur at different times across the trials. Due to this mismatch, the averaging of neural activity will not bring into salience movement-related components. To address this problem, we present a method of alignment that accounts for the variabilities in the way the movements are conducted. In this study, arm speed was used to align neural activity. Four subjects had electrocorticographic (ECoG) electrodes implanted over their primary motor cortex and were asked to perform reaching and retrieving tasks using the upper limb contralateral to the site of electrode implantation. The arm speeds were aligned using a non-linear transformation of the temporal axes resulting in average spectrograms with superior visualization of movement-related neural activity when compared to averaging without alignment.

Neuroprosthetic limb control with electrocorticography: approaches and challenges

Advanced upper limb prosthetics, such as the Johns Hopkins Applied Physics Lab Modular Prosthetic Limb (MPL), are now available for research and preliminary clinical applications. Research attention has shifted to developing means of controlling these prostheses. Penetrating microelectrode arrays are often used in animal and human models to decode action potentials for cortical control. These arrays may suffer signal loss over the long-term and therefore should not be the only implant type investigated for chronic BMI use. Electrocorticographic (ECoG) signals from electrodes on the cortical surface may provide more stable long-term recordings. Several studies have demonstrated ECoG's potential for decoding cortical activity. As a result, clinical studies are investigating ECoG encoding of limb movement, as well as its use for interfacing with and controlling advanced prosthetic arms. This overview presents the technical state of the art in the use of ECoG in controlling prostheses. Technical limitations of the current approach and future directions are also presented.

Multi-modal decoding: longitudinal coherency changes between spike trains, local field potentials and electrocorticogram signals

Neural information degeneracy in chronic implants due to signal instabilities affects optimal performance of brain-machine interfaces (BMIs). Spike-decoders are more vulnerable compared to those using LFPs and ECoG signals. In order for BMIs to perform reliably across years, decoders should be able to use neural information contained in various signal modalities. Hence, it is important to identify information redundancy among signal types. In this work, spikes, LFPs and ECoGs were recorded simultaneously from motor cortex of a rhesus monkey, while the animal was learning to control a multi-DOF robot with a spike-decoder. As the behavioral performance increased, the linear association among the signal types increased. Coherency of these signals increased in specific frequency bands as learning occurred. These results suggest the possibility of substituting the information lost in one modality by another.

Give me a sign: decoding four complex hand gestures based on high-density ECoG

The increasing understanding of human brain functions makes it possible to directly interact with the brain for therapeutic purposes. Implantable brain computer interfaces promise to replace or restore motor functions in patients with partial or complete paralysis. We postulate that neuronal states associated with gestures, as they are used in the finger spelling alphabet of sign languages, provide an excellent signal for implantable brain computer interfaces to restore communication. To test this, we evaluated decodability of four gestures using high-density electrocorticography in two participants. The electrode grids were located subdurally on the hand knob area of the sensorimotor cortex covering a surface of 2.5–5.2 cm². Using a pattern-matching classification approach four types of hand gestures were classified based on their pattern of neuronal activity. In the two participants the gestures were classified with 97 and 74 % accuracy. The high frequencies (>65 Hz) allowed for the best classification results. This proof-of-principle study indicates that the four gestures are associated with a reliable and discriminable spatial representation on a confined area of the sensorimotor cortex. This robust representation on a small area makes hand gestures an interesting control feature for an implantable BCI to restore communication for severely paralyzed people.

Extracting kinetic information from human motor cortical signals

Brain machine interfaces (BMIs) have the potential to provide intuitive control of neuroprostheses to restore grasp to patients with paralyzed or amputated upper limbs. For these neuroprostheses to function, the ability to accurately control grasp force is critical. Grasp force can be decoded from neuronal spikes in monkeys, and hand kinematics can be decoded using electrocorticogram (ECoG) signals recorded from the surface of the human motor cortex. We hypothesized that kinetic information about grasping could also be extracted from ECoG, and sought to decode continuously-graded grasp force. In this study, we decoded isometric pinch force with high accuracy from ECoG in 10 human subjects. The predicted signals explained from 22% to 88% (60 ± 6%, mean ± SE) of the variance in the actual force generated. We also decoded muscle activity in the finger flexors, with similar accuracy to force decoding. We found that high gamma band and time domain features of the ECoG signal were most informative about kinetics, similar to our previous findings with intracortical LFPs. In addition, we found that peak cortical representations of force applied by the index and little fingers were separated by only about 4mm. Thus, ECoG can be used to decode not only kinematics, but also kinetics of movement. This is an important step toward restoring intuitively-controlled grasp to impaired patients.

WIMAGINE: wireless 64-channel ECoG recording implant for long term clinical applications

A wireless 64-channel ElectroCorticoGram (ECoG) recording implant named WIMAGINE has been designed for various clinical applications. The device is aimed at interfacing a cortical electrode array to an external computer for neural recording and control applications. This active implantable medical device is able to record neural activity on 64 electrodes with selectable gain and sampling frequency, with less than 1 μV(RMS) input referred noise in the [0.5 Hz - 300 Hz] band. It is powered remotely through an inductive link at 13.56 MHz which provides up to 100 mW. The digitized data is transmitted wirelessly to a custom designed base station connected to a PC. The hermetic housing and the antennae have been designed and optimized to ease the surgery. The design of this implant takes into account all the requirements of a clinical trial, in particular safety, reliability, and compliance with the regulations applicable to class III AIMD. The main features of this WIMAGINE implantable device and its architecture are presented, as well as its functional performances and long-term biocompatibility results.

From speech to thought: the neuronal basis of cognitive units in non-experimental, real-life communication investigated using ECoG

Exchange of thoughts by means of expressive speech is fundamental to human communication. However, the neuronal basis of real-life communication in general, and of verbal exchange of ideas in particular, has rarely been studied until now. Here, our aim was to establish an approach for exploring the neuronal processes related to cognitive “idea” units (IUs) in conditions of non-experimental speech production. We investigated whether such units corresponding to single, coherent chunks of speech with syntactically-defined borders, are useful to unravel the neuronal mechanisms underlying real-world human cognition. To this aim, we employed simultaneous electrocorticography (ECoG) and video recordings obtained in pre-neurosurgical diagnostics of epilepsy patients. We transcribed non-experimental, daily hospital conversations, identified IUs in transcriptions of the patients' speech, classified the obtained IUs according to a previously-proposed taxonomy focusing on memory content, and investigated the underlying neuronal activity. In each of our three subjects, we were able to collect a large number of IUs which could be assigned to different functional IU subclasses with a high inter-rater agreement. Robust IU-onset-related changes in spectral magnitude could be observed in high gamma frequencies (70–150 Hz) on the inferior lateral convexity and in the superior temporal cortex regardless of the IU content. A comparison of the topography of these responses with mouth motor and speech areas identified by electrocortical stimulation showed that IUs might be of use for extraoperative mapping of eloquent cortex (average sensitivity: 44.4%, average specificity: 91.1%). High gamma responses specific to memory-related IU subclasses were observed in the inferior parietal and prefrontal regions. IU-based analysis of ECoG recordings during non-experimental communication thus elicits topographically- and functionally-specific effects. We conclude that segmentation of spontaneous real-world speech in linguistically-motivated units is a promising strategy for elucidating the neuronal basis of mental processing during non-experimental communication.

Stereovision and augmented reality for closed-loop control of grasping in hand prostheses

OBJECTIVE

Technologically advanced assistive devices are nowadays available to restore grasping, but effective and effortless control integrating both feed-forward (commands) and feedback (sensory information) is still missing. The goal of this work was to develop a user friendly interface for the semi-automatic and closed-loop control of grasping and to test its feasibility.

APPROACH

We developed a controller based on stereovision to automatically select grasp type and size and augmented reality (AR) to provide artificial proprioceptive feedback. The system was experimentally tested in healthy subjects using a dexterous hand prosthesis to grasp a set of daily objects. The subjects wore AR glasses with an integrated stereo-camera pair, and triggered the system via a simple myoelectric interface.

MAIN RESULTS

The results demonstrated that the subjects got easily acquainted with the semi-autonomous control. The stereovision grasp decoder successfully estimated the grasp type and size in realistic, cluttered environments. When allowed (forced) to correct the automatic system decisions, the subjects successfully utilized the AR feedback and achieved close to ideal system performance.

SIGNIFICANCE

The new method implements a high level, low effort control of complex functions in addition to the low level closed-loop control. The latter is achieved by providing rich visual feedback, which is integrated into the real life environment. The proposed system is an effective interface applicable with small alterations for many advanced prosthetic and orthotic/therapeutic rehabilitation devices.

Decoding fingertip trajectory from electrocorticographic signals in humans

Seeking to apply brain-machine interface technology in neuroprosthetics, a number of methods for predicting trajectory of the elbow and wrist have been proposed and have shown remarkable results. Recently, the prediction of hand trajectory and classification of hand gestures or grasping types have attracted considerable attention. However, trajectory prediction for precise finger motion has remained a challenge. We proposed a method for the prediction of fingertip motions from electrocorticographic signals in human cortex. A patient performed extension/flexion tasks with three fingers. Average Pearson's correlation coefficients and normalized root-mean-square errors between decoded and actual trajectories were 0.83-0.90 and 0.24-0.48, respectively. To confirm generalizability to other users, we applied our method to the BCI Competition IV open data sets. Our method showed that the prediction accuracy of fingertip trajectory could be equivalent to that of other results in the competition.

Closed-loop control of spinal cord stimulation to restore hand function after paralysis

As yet, no cure exists for upper-limb paralysis resulting from the damage to motor pathways after spinal cord injury or stroke. Recently, neural activity from the motor cortex of paralyzed individuals has been used to control the movements of a robot arm but restoring function to patients' actual limbs remains a considerable challenge. Previously we have shown that electrical stimulation of the cervical spinal cord in anesthetized monkeys can elicit functional upper-limb movements like reaching and grasping. Here we show that stimulation can be controlled using cortical activity in awake animals to bypass disruption of the corticospinal system, restoring their ability to perform a simple upper-limb task. Monkeys were trained to grasp and pull a spring-loaded handle. After temporary paralysis of the hand was induced by reversible inactivation of primary motor cortex using muscimol, grasp-related single-unit activity from the ventral premotor cortex was converted into stimulation patterns delivered in real-time to the cervical spinal gray matter. During periods of closed-loop stimulation, task-modulated electromyogram, movement amplitude, and task success rate were improved relative to interleaved control periods without stimulation. In some sessions, single motor unit activity from weakly active muscles was also used successfully to control stimulation. These results are the first use of a neural prosthesis to improve the hand function of primates after motor cortex disruption, and demonstrate the potential for closed-loop cortical control of spinal cord stimulation to reanimate paralyzed limbs.

Epidural electrocorticography of phantom hand movement following long-term upper-limb amputation

INTRODUCTION

Prostheses for upper-limb amputees are currently controlled by either myoelectric or peripheral neural signals. Performance and dexterity of these devices is still limited, particularly when it comes to controlling hand function. Movement-related brain activity might serve as a complementary bio-signal for motor control of hand prosthesis.

METHODS

We introduced a methodology to implant a cortical interface without direct exposure of the brain surface in an upper-limb amputee. This bi-directional interface enabled us to explore the cortical physiology following long-term transhumeral amputation. In addition, we investigated neurofeedback of electrocorticographic brain activity related to the patient's motor imagery to open his missing hand, i.e., phantom hand movement, for real-time control of a virtual hand prosthesis.

RESULTS

Both event-related brain activity and cortical stimulation revealed mutually overlapping cortical representations of the phantom hand. Phantom hand movements could be robustly classified and the patient required only three training sessions to gain reliable control of the virtual hand prosthesis in an online closed-loop paradigm that discriminated between hand opening and rest.

CONCLUSION

Epidural implants may constitute a powerful and safe alternative communication pathway between the brain and external devices for upper-limb amputees, thereby facilitating the integrated use of different signal sources for more intuitive and specific control of multi-functional devices in clinical use.

Decoding grasp force profile from electrocorticography signals in non-human primate sensorimotor cortex

The relatively low invasiveness of electrocorticography (ECoG) has made it a promising candidate for the development of practical, high-performance neural prosthetics. Recent ECoG-based studies have shown success in decoding hand and finger movements and muscle activity in reaching and grasping tasks. However, decoding of force profiles is still lacking. Here, we demonstrate that lateral grasp force profile can be decoded using a sparse linear regression from 15 and 16 channel ECoG signals recorded from sensorimotor cortex in two non-human primates. The best average correlation coefficients of prediction after 10-fold cross validation were 0.82±0.09 and 0.79±0.15 for our monkeys A and B, respectively. These results show that grasp force profile was successfully decoded from ECoG signals in reaching and grasping tasks and may potentially contribute to the development of more natural control methods for grasping in neural prosthetics.

Self-recalibrating classifiers for intracortical brain-computer interfaces

OBJECTIVE

Intracortical brain-computer interface (BCI) decoders are typically retrained daily to maintain stable performance. Self-recalibrating decoders aim to remove the burden this may present in the clinic by training themselves autonomously during normal use but have only been developed for continuous control. Here we address the problem for discrete decoding (classifiers).

APPROACH

We recorded threshold crossings from 96-electrode arrays implanted in the motor cortex of two rhesus macaques performing center-out reaches in 7 directions over 41 and 36 separate days spanning 48 and 58 days in total for offline analysis.

MAIN RESULTS

We show that for the purposes of developing a self-recalibrating classifier, tuning parameters can be considered as fixed within days and that parameters on the same electrode move up and down together between days. Further, drift is constrained across time, which is reflected in the performance of a standard classifier which does not progressively worsen if it is not retrained daily, though overall performance is reduced by more than 10% compared to a daily retrained classifier. Two novel self-recalibrating classifiers produce a ~15% increase in classification accuracy over that achieved by the non-retrained classifier to nearly recover the performance of the daily retrained classifier.

SIGNIFICANCE

We believe that the development of classifiers that require no daily retraining will accelerate the clinical translation of BCI systems. Future work should test these results in a closed-loop setting.

Concurrent stable and unstable cortical correlates of human wrist movements

Cortical activity has been shown to correlate with different parameters of movement. However, the dynamic properties of cortico-motor mappings still remain unexplored in humans. Here, we show that during the repetition of simple stereotyped wrist movements both stable and unstable correlates simultaneously emerge in human sensorimotor cortex. Using visual feedback of wrist movement target inferred online from MEG, we assessed the dynamics of the tuning properties of two neuronal signals: the MEG signal below 1.6 Hz and within the 4 to 6 Hz range. We found that both components are modulated by wrist movement allowing for closed-loop inference of movement targets. Interestingly, while tuning of 4 to 6 Hz signals remained stable over time leading to stable inference of movement target using a static classifier, the tuning of cortical signals below 1.6 Hz significantly changed resulting in steadily decreasing inference accuracy. Our findings demonstrate that non-invasive neuronal population signals in human sensorimotor cortex can reflect a stable correlate of voluntary movements. Hence, we provide first evidence for a stable control signal in non-invasive human brain-machine interface research. However, as not all neuronal signals initially tuned to movement were stable across days, a careful selection of features for real-life applications seems to be mandatory.