Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter

A Hippocampal-Parietal Network for Reference Frame Coordination

Navigating space and forming memories based on spatial experience are crucial for survival, including storing memories in an allocentric (map-like) framework and conversion into egocentric (body-centered) action. The hippocampus and parietal cortex (PC) comprise a network for coordinating these reference frames, though the mechanism remains unclear. We used a task requiring remembering previous spatial locations to make correct future action and observed that hippocampus can encode the allocentric place, while PC encodes upcoming actions and relays this to hippocampus. Transformation from location to action unfolds gradually, with "Came From" signals diminishing and future action representations strengthening. PC sometimes encodes previous spatial locations in a route-based reference frame and conveys this to hippocampus. The signal for the future location appears first in PC, and then in hippocampus, in the form of an egocentric direction of future goal locations, suggesting egocentric encoding recently observed in hippocampus may originate in PC (or another "upstream" structure). Bidirectional signaling is apparent between PC and hippocampus and suggests a coordinated mechanism for integrating allocentric, route-centered, and egocentric spatial reference frames at the network level during navigation.

Less is more: selection from a small set of options improves BCI velocity control

Objective.Decoding algorithms used in invasive brain-computer interfaces (iBCIs) typically convert neural activity into continuously varying velocity commands. We hypothesized that putting constraints on which decoded velocity commands are permissible could improve user performance. To test this hypothesis, we designed the discrete direction selection (DDS) decoder, which uses neural activity to select among a small menu of preset cursor velocities.Approach. We tested DDS in a closed-loop cursor control task against many common continuous velocity decoders in both a human-operated real-time iBCI simulator (the jaBCI) and in a monkey using an iBCI. In the jaBCI, we compared performance across four visits by each of 48 naïve, able-bodied human subjects using either DDS, direct regression with assist (an affine map from neural activity to cursor velocity, DR-A), ReFIT, or the velocity Kalman Filter (vKF). In a follow up study to verify the jaBCI results, we compared a monkey's performance using an iBCI with either DDS or the Wiener filter decoder (a direct regression decoder that includes time history, WF).Main Result. In the jaBCI, DDS substantially outperformed all other decoders with 93% mean targets hit per visit compared to DR-A, ReFIT, and vKF with 56%, 39%, and 26% mean targets hit, respectively. With the iBCI, the monkey achieved a 61% success rate with DDS and a 37% success rate with WF.Significance. Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of discretization in simplifying online BCI control.

The state-of-the-art of invasive brain-computer interfaces in humans: a systematic review and individual patient meta-analysis

Objective.Invasive brain-computer interfaces (iBCIs) have evolved significantly since the first neurotrophic electrode was implanted in a human subject three decades ago. Since then, both hardware and software advances have increased the iBCI performance to enable tasks such as decoding conversations in real-time and manipulating external limb prostheses with haptic feedback. In this systematic review, we aim to evaluate the advances in iBCI hardware, software and functionality and describe challenges and opportunities in the iBCI field.Approach.Medline, EMBASE, PubMed and Cochrane databases were searched from inception until 13 April 2024. Primary studies reporting the use of iBCI in human subjects to restore function were included. Endpoints extracted include iBCI electrode type, iBCI implantation, decoder algorithm, iBCI effector, testing and training methodology and functional outcomes. Narrative synthesis of outcomes was done with a focus on hardware and software development trends over time. Individual patient data (IPD) was also collected and an IPD meta-analysis was done to identify factors significant to iBCI performance.Main results.93 studies involving 214 patients were included in this systematic review. The median task performance accuracy for cursor control tasks was 76.00% (Interquartile range [IQR] = 21.2), for motor tasks was 80.00% (IQR = 23.3), and for communication tasks was 93.27% (IQR = 15.3). Current advances in iBCI software include use of recurrent neural network architectures as decoders, while hardware advances such as intravascular stentrodes provide a less invasive alternative for neural recording. Challenges include the lack of standardized testing paradigms for specific functional outcomes and issues with portability and chronicity limiting iBCI usage to laboratory settings.Significance.Our systematic review demonstrated the exponential rate at which iBCIs have evolved over the past two decades. Yet, more work is needed for widespread clinical adoption and translation to long-term home-use.

Neural Correlation Integrated Adaptive Point Process Filtering on Population Spike Trains

Brain encodes information through neural spiking activities that modulate external environmental stimuli and underlying internal states. Population of neurons coordinate through functional connectivity to plan movement trajectories and accurately activate neuromuscular activities. Motor Brain-machine interface (BMI) is a platform to study the relationship between behaviors and neural ensemble activities. In BMI, point process filters model directly on spike timings to extract underlying states such as motion intents from observed multi-neuron spike trains. However, these methods assume the encoded information from individual neurons is conditionally independent, which leads to less precise estimation. It is necessary to incorporate functional neural connectivity into a point process filter to improve the state estimation. In this paper, we propose a neural correlation integrated adaptive point process filter (CIPPF) that can incorporate the information from functional neural connectivity from population spike trains in a recursive Bayesian framework. Functional neural connectivity information is approximated by an artificial neural network to provide extra updating information for the posterior estimation. Gaussian approximation is applied on the probability distribution to obtain a closed-form solution. Our proposed method is validated on both simulation and real data collected from the rat two-lever discrimination task. Due to the simultaneous modeling of functional neural connectivity and single neuronal tuning properties, the proposed method shows better decoding performance. This suggests the possibility to improve BMI performance by processing the coordinated neural population activities.

Effects of Packet Loss on Neural Decoding Effectiveness in Wireless Transmission

BACKGROUND

In brain-computer interfaces, neural decoding plays a central role in translating neural signals into meaningful physical actions. These signals are transmitted to processors for decoding via wired or wireless channels; however, they are often subject to data loss, commonly referred to as "packet loss". Despite their importance, the effects of different types and degrees of packet loss on neural decoding have not yet been comprehensively studied. Understanding these effects is critical for advancing neural signal processing.

METHODS

This study addresses this gap by constructing four distinct packet loss models that simulate the congestion, distribution, and burst loss scenarios. Using macaque superior arm movement decoding experiments, we analyzed the effects of the aforementioned packet loss types on decoding performance across six parameters (position, velocity, and acceleration in the x and y dimensions). The performance was assessed using the R2 metric and statistical comparisons across different loss scenarios.

RESULTS

Our results indicate that sudden, consecutive packet loss significantly degraded decoding performance. For the same packet loss probability, burst loss led to the largest decrease in the R2 value. Notably, when the packet loss rate reached 10%, the decoding performance for acceleration dropped to 73% of the original R2 value. On the other hand, when the packet loss rate was within 2%, the neural signal decoding results across all packet loss models remained largely unaffected. However, as the packet loss rate increased, the impact became more pronounced. These findings highlight the varying degrees to which different packet loss models affect decoding outcomes.

CONCLUSIONS

This study quantitatively evaluated the relationship between packet loss and neural decoding outcomes, highlighting the differential effects of loss patterns on decoding parameters, and it proposed some methods and devices to solve the problem of packet loss. These findings offer valuable insights for the development of resilient neural signal acquisition and processing systems capable of mitigating the impact of packet loss.

Spatio-temporal transformers for decoding neural movement control

Objective. Deep learning tools applied to high-resolution neurophysiological data have significantly progressed, offering enhanced decoding, real-time processing, and readability for practical applications. However, the design of artificial neural networks to analyze neural activityin vivoremains a challenge, requiring a delicate balance between efficiency in low-data regimes and the interpretability of the results.Approach. To address this challenge, we introduce a novel specialized transformer architecture to analyze single-neuron spiking activity. The model is tested on multi-electrode recordings from the dorsal premotor cortex of non-human primates performing a motor inhibition task.Main results. The proposed architecture provides an early prediction of the correct movement direction, achieving accurate results no later than 230 ms after the Go signal presentation across animals. Additionally, the model can forecast whether the movement will be generated or withheld before a stop signal, unattended, is actually presented. To further understand the internal dynamics of the model, we compute the predicted correlations between time steps and between neurons at successive layers of the architecture, with the evolution of these correlations mirrors findings from previous theoretical analyses.Significance. Overall, our framework provides a comprehensive use case for the practical implementation of deep learning tools in motor control research, highlighting both the predictive capabilities and interpretability of the proposed architecture.

Ensemble reactivations during brief rest drive fast learning of sequences

During motor learning, breaks in practice are known to facilitate behavioural optimizations. Although this process has traditionally been studied over long breaks that last hours to days1-6, recent studies in humans have demonstrated that rapid performance gains during early motor sequence learning are most pronounced after very brief breaks lasting seconds to minutes7-10. However, the precise causal neural mechanisms that facilitate performance gains after brief breaks remain poorly understood. Here we recorded neural ensemble activity in the motor cortex of macaques while they performed a visuomotor sequence learning task interspersed with brief breaks. We found that task-related neural cofiring patterns were reactivated during brief breaks. The rate and content of reactivations predicted the magnitude and pattern of subsequent performance gains. Of note, we found that performance gains and reactivations were positively correlated with cortical ripples (80-120 Hz oscillations) but anti-correlated with β bursts (13-30 Hz oscillations), which ultimately dominated breaks after the fast learning phase plateaued. We then applied 20 Hz epidural alternating current stimulation (ACS) to motor cortex, which reduced reactivation rates in a phase-specific and dose-dependent manner. Notably, 20 Hz ACS also eliminated performance gains. Overall, our results indicate that the reactivations of task ensembles during brief breaks are causal drivers of subsequent performance gains. β bursts compete with this process, possibly to support stable performance.

Proportional myoelectric control of a virtual bionic arm in participants with hemiparesis, muscle spasticity, and impaired range of motion

BACKGROUND

This research aims to improve the control of assistive devices for individuals with hemiparesis after stroke by providing intuitive and proportional motor control. Stroke is the leading cause of disability in the United States, with 80% of stroke-related disability coming in the form of hemiparesis, presented as weakness or paresis on half of the body. Current assistive exoskeletonscontrolled via electromyography do not allow for fine force regulation. Current control strategies provide only binary, all-or-nothing control based on a linear threshold of muscle activity.

METHODS

In this study, we demonstrate the ability of participants with hemiparesis to finely regulate their muscle activity to proportionally control the position of a virtual bionic arm. Ten stroke survivors and ten healthy, aged-matched controls completed a target-touching task with the virtual bionic arm. We compared the signal-to-noise ratio (SNR) of the recorded electromyography (EMG) signals used to train the control algorithms and the task performance using root mean square error, percent time in target, and maximum hold time within the target window. Additionally, we looked at the correlation between EMG SNR, task performance, and clinical spasticity scores.

RESULTS

All stroke survivors were able to achieve proportional EMG control despite limited or no physical movement (i.e., modified Ashworth scale of 3). EMG SNR was significantly lower for the paretic arm than the contralateral nonparetic arm and healthy control arms, but proportional EMG control was similar across conditions for hand grasp. In contrast, proportional EMG control for hand extension was significantly worse for paretic arms than healthy control arms. The participants' age, time since their stroke, clinical spasticity rate, and history of botulinum toxin injections had no impact on proportional EMG control.

CONCLUSIONS

It is possible to provide proportional EMG control of assistive devices from a stroke survivor's paretic arm. Importantly, information regulating fine force output is still present in muscle activity, even in extreme cases of spasticity where there is no visible movement. Future work should incorporate proportional EMG control into upper-limb exoskeletons to enhance the dexterity of stroke survivors.

Multiscale effective connectivity analysis of brain activity using neural ordinary differential equations

Neural mechanisms and underlying directionality of signaling among brain regions depend on neural dynamics spanning multiple spatiotemporal scales of population activity. Despite recent advances in multimodal measurements of brain activity, there is no broadly accepted multiscale dynamical models for the collective activity represented in neural signals. Here we introduce a neurobiological-driven deep learning model, termed multiscale neural dynamics neural ordinary differential equation (msDyNODE), to describe multiscale brain communications governing cognition and behavior. We demonstrate that msDyNODE successfully captures multiscale activity using both simulations and electrophysiological experiments. The msDyNODE-derived causal interactions between recording locations and scales not only aligned well with the abstraction of the hierarchical neuroanatomy of the mammalian central nervous system but also exhibited behavioral dependences. This work offers a new approach for mechanistic multiscale studies of neural processes.

Enhancing neuroprosthesis calibration: the advantage of integrating prior training over exclusive use of new data

Objective.Neuroprostheses typically operate under supervised learning, in which a machine-learning algorithm is trained to correlate neural or myoelectric activity with an individual's motor intent. Due to the stochastic nature of neuromyoelectric signals, algorithm performance decays over time. This decay is accelerated when attempting to regress proportional control of multiple joints in parallel, compared with the more typical classification-based pattern recognition control. To overcome this degradation, neuroprostheses and commercial myoelectric prostheses are often recalibrated and retrained frequently so that only the most recent, up-to-date data influences the algorithm performance. Here, we introduce and validate an alternative training paradigm in which training data from past calibrations is aggregated and reused in future calibrations for regression control.Approach.Using a cohort of four transradial amputees implanted with intramuscular electromyographic recording leads, we demonstrate that aggregating prior datasets improves prosthetic regression-based control in offline analyses and an online human-in-the-loop task. In offline analyses, we compared the performance of a convolutional neural network (CNN) and a modified Kalman filter (MKF) to simultaneously regress the kinematics of an eight-degree-of-freedom prosthesis. Both algorithms were trained under the traditional paradigm using a single dataset, as well as under the new paradigm using aggregated datasets from the past five or ten trainings.Main results.Dataset aggregation reduced the root-mean-squared error (RMSE) of algorithm estimates for both the CNN and MKF, although the CNN saw a greater reduction in error. Further offline analyses revealed that dataset aggregation improved CNN robustness when reusing the same algorithm on subsequent test days, as indicated by a smaller increase in RMSE per day. Finally, data from an online virtual-target-touching task with one amputee showed significantly better real-time prosthetic control when using aggregated training data from just two prior datasets.Significance.Altogether, these results demonstrate that training data from past calibrations should not be discarded but, rather, should be reused in an aggregated training dataset such that the increased amount and diversity of data improve algorithm performance. More broadly, this work supports a paradigm shift for the field of neuroprostheses away from daily data recalibration for linear classification models and towards daily data aggregation for non-linear regression models.

A reinforcement learning based software simulator for motor brain-computer interfaces

Intracortical motor brain-computer interfaces (BCIs) are expensive and time-consuming to design because accurate evaluation traditionally requires real-time experiments. In a BCI system, a user interacts with an imperfect decoder and continuously changes motor commands in response to unexpected decoded movements. This "closed-loop" nature of BCI leads to emergent interactions between the user and decoder that are challenging to model. The gold standard for BCI evaluation is therefore real-time experiments, which significantly limits the speed and community of BCI research. We present a new BCI simulator that enables researchers to accurately and quickly design BCIs for cursor control entirely in software. Our simulator replaces the BCI user with a deep reinforcement learning (RL) agent that interacts with a simulated BCI system and learns to optimally control it. We demonstrate that our simulator is accurate and versatile, reproducing the published results of three distinct types of BCI decoders: (1) a state-of-the-art linear decoder (FIT-KF), (2) a "two-stage" BCI decoder requiring closed-loop decoder adaptation (ReFIT-KF), and (3) a nonlinear recurrent neural network decoder (FORCE).

Decoding the brain: From neural representations to mechanistic models

A central principle in neuroscience is that neurons within the brain act in concert to produce perception, cognition, and adaptive behavior. Neurons are organized into specialized brain areas, dedicated to different functions to varying extents, and their function relies on distributed circuits to continuously encode relevant environmental and body-state features, enabling other areas to decode (interpret) these representations for computing meaningful decisions and executing precise movements. Thus, the distributed brain can be thought of as a series of computations that act to encode and decode information. In this perspective, we detail important concepts of neural encoding and decoding and highlight the mathematical tools used to measure them, including deep learning methods. We provide case studies where decoding concepts enable foundational and translational science in motor, visual, and language processing.

Electromyographically controlled prosthetic wrist improves dexterity and reduces compensatory movements without added cognitive load

Wrist function is a top priority for transradial amputees. However, the combined functional, biomechanical, and cognitive impact of using a powered prosthetic wrist is unclear. Here, we quantify task performance, compensatory movements, and cognitive load while three transradial amputees performed a modified Clothespin Relocation Task using two myoelectric prostheses with and without the wrists. The two myoelectric prostheses include a commercial prosthesis with a built-in powered wrist, and a newly developed inexpensive prosthetic wrist for research purposes, called the "Utah wrist", that can be adapted to work with various sockets and prostheses. For these three participants, task failure rate decreased significantly from 66% ± 12% without the wrist to 39% ± 9% with the Utah wrist. Compensatory forward leaning movements also decreased significantly, from 24.2° ± 2.5 without the wrist to 12.6° ± 1.0 with the Utah wrist, and from 23.6° ± 7.6 to 15.3° ± 7.2 with the commercial prosthesis with an integrated wrist. Compensatory leftward bending movements also significantly decreased, from 20.8° ± 8.6 to 12.3° ± 5.3, for the commercial with an integrated wrist. Importantly, simultaneous myoelectric control of either prosthetic wrist had no significant impact on cognitive load, as assessed by the NASA Task Load Index survey and a secondary detection response task. This work suggests that functional prosthetic wrists can improve dexterity and reduce compensation without significantly increasing cognitive effort. These results, and the introduction of a new inexpensive prosthetic wrist for research purposes, can aid future research and development and guide the prescription of upper-limb prostheses.

Practical Bayesian Inference in Neuroscience: Or How I Learned to Stop Worrying and Embrace the Distribution

Typical statistical practices in the biological sciences have been increasingly called into question due to difficulties in the replication of an increasing number of studies, many of which are confounded by the relative difficulty of null significance hypothesis testing designs and interpretation of p-values. Bayesian inference, representing a fundamentally different approach to hypothesis testing, is receiving renewed interest as a potential alternative or complement to traditional null significance hypothesis testing due to its ease of interpretation and explicit declarations of prior assumptions. Bayesian models are more mathematically complex than equivalent frequentist approaches, which have historically limited applications to simplified analysis cases. However, the advent of probability distribution sampling tools with exponential increases in computational power now allows for quick and robust inference under any distribution of data. Here we present a practical tutorial on the use of Bayesian inference in the context of neuroscientific studies in both rat electrophysiological and computational modeling data. We first start with an intuitive discussion of Bayes' rule and inference followed by the formulation of Bayesian-based regression and ANOVA models using data from a variety of neuroscientific studies. We show how Bayesian inference leads to easily interpretable analysis of data while providing an open-source toolbox to facilitate the use of Bayesian tools.

Decoding the Spike-Band Subthreshold Motor Cortical Activity

Intracortical Brain-Computer Interfaces (iBCI) use single-unit activity (SUA), multiunit activity (MUA) and local field potentials (LFP) to control neuroprosthetic devices. SUA and MUA are usually extracted from the bandpassed recording through amplitude thresholding, while subthreshold data are ignored. Here, we show that subthreshold data can actually be decoded to determine behavioral variables with test set accuracy of up to 100%. Although the utility of SUA, MUA and LFP for decoding behavioral variables has been explored previously, this study investigates the utility of spike-band subthreshold activity exclusively. We provide evidence suggesting that this activity can be used to keep decoding performance at acceptable levels even when SUA quality is reduced over time. To the best of our knowledge, the signals that we derive from the subthreshold activity may be the weakest neural signals that have ever been extracted from extracellular neural recordings, while still being decodable with test set accuracy of up to 100%. These results are relevant for the development of fully data-driven and automated methods for amplitude thresholding spike-band extracellular neural recordings in iBCIs containing thousands of electrodes.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodents has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguishes control types at the go cue and target acquisition, respectively, while M1 best predicts target-direction at both task events. We also find effective connectivity from DLPFC → M1 throughout both control types and Cd → M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Identification of interacting neural populations: methods and statistical considerations

As improved recording technologies have created new opportunities for neurophysiological investigation, emphasis has shifted from individual neurons to multiple populations that form circuits, and it has become important to provide evidence of cross-population coordinated activity. We review various methods for doing so, placing them in six major categories while avoiding technical descriptions and instead focusing on high-level motivations and concerns. Our aim is to indicate what the methods can achieve and the circumstances under which they are likely to succeed. Toward this end, we include a discussion of four cross-cutting issues: the definition of neural populations, trial-to-trial variability and Poisson-like noise, time-varying dynamics, and causality.

Invariant neural dynamics drive commands to control different movements

It has been proposed that the nervous system has the capacity to generate a wide variety of movements because it reuses some invariant code. Previous work has identified that dynamics of neural population activity are similar during different movements, where dynamics refer to how the instantaneous spatial pattern of population activity changes in time. Here, we test whether invariant dynamics of neural populations are actually used to issue the commands that direct movement. Using a brain-machine interface (BMI) that transforms rhesus macaques' motor-cortex activity into commands for a neuroprosthetic cursor, we discovered that the same command is issued with different neural-activity patterns in different movements. However, these different patterns were predictable, as we found that the transitions between activity patterns are governed by the same dynamics across movements. These invariant dynamics are low dimensional, and critically, they align with the BMI, so that they predict the specific component of neural activity that actually issues the next command. We introduce a model of optimal feedback control (OFC) that shows that invariant dynamics can help transform movement feedback into commands, reducing the input that the neural population needs to control movement. Altogether our results demonstrate that invariant dynamics drive commands to control a variety of movements and show how feedback can be integrated with invariant dynamics to issue generalizable commands.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodent BMI has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from the primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguish between control types at the go cue and target acquisition, respectively. We also found effective connectivity from DLPFC→M1 throughout trials across both control types and Cd→M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Improving the accuracy of decoding monkey brain-machine interface data by estimating the state of unobserved cell assemblies

BACKGROUND

The brain-machine interface is a technology that has been used for improving the quality of life of individuals with physical disabilities and also healthy individuals. It is important to improve the methods used for decoding the brain-machine interface data as the accuracy and speed of movements achieved using the existing technology are not comparable to the normal body.

COMPARISON WITH THE EXISTING METHOD

Decoding of brain-machine interface data using the proposed method resulted in improved decoding accuracy compared to the existing method.

CONCLUSIONS

The results demonstrated the usefulness of cell assembly state estimation method for decoding the brain-machine interface data.

NEW METHOD

We incorporated a novel method of estimating cell assembly states using spike trains with the existing decoding method that used only firing rate data. Synaptic connectivity pattern was used as feature values in addition to firing rate. Publicly available monkey brain-machine interface datasets were used in the study.

RESULTS

As long as the decoding was successful, the root mean square error of the proposed method was significantly smaller than the existing method. Artificial neural netowork-based decoding method resulted in more stable decoding, and also improved the decoding accuracy due to incorporation of synaptic connectivity pattern.

Bayesian prediction of psychophysical detection responses from spike activity in the rat sensorimotor cortex

Decoding of sensorimotor information is essential for brain-computer interfaces (BCIs) as well as in normal functioning organisms. In this study, Bayesian models were developed for the prediction of binary decisions of 10 awake freely-moving male/female rats based on neural activity in a vibrotactile yes/no detection task. The vibrotactile stimuli were 40-Hz sinusoidal displacements (amplitude: 200 µm, duration: 0.5 s) applied on the glabrous skin. The task was to depress the right lever for stimulus detection and left lever for stimulus-off condition. Spike activity was recorded from 16-channel microwire arrays implanted in the hindlimb representation of primary somatosensory cortex (S1), overlapping also with the associated representation in the primary motor cortex (M1). Single-/multi-unit average spike rate (R_d) within the stimulus analysis window was used as the predictor of the stimulus state and the behavioral response at each trial based on a Bayesian network model. Due to high neural and psychophysical response variability for each rat and also across subjects, mean R_d was not correlated with hit and false alarm rates. Despite the fluctuations in the neural data, the Bayesian model for each rat generated moderately good accuracy (0.60-0.90) and good class prediction scores (recall, precision, F1) and was also tested with subsets of data (e.g. regular vs. fast spike groups). It was generally observed that the models were better for rats with lower psychophysical performance (lower sensitivity index A'). This suggests that Bayesian inference and similar machine learning techniques may be especially helpful during the training phase of BCIs or for rehabilitation with neuroprostheses.

Deep learning for neural decoding in motor cortex

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons.Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain.Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders.Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

Optimal control methods for nonlinear parameter estimation in biophysical neuron models

Functional forms of biophysically-realistic neuron models are constrained by neurobiological and anatomical considerations, such as cell morphologies and the presence of known ion channels. Despite these constraints, neuron models still contain unknown static parameters which must be inferred from experiment. This inference task is most readily cast into the framework of state-space models, which systematically takes into account partial observability and measurement noise. Inferring only dynamical state variables such as membrane voltages is a well-studied problem, and has been approached with a wide range of techniques beginning with the well-known Kalman filter. Inferring both states and fixed parameters, on the other hand, is less straightforward. Here, we develop a method for joint parameter and state inference that combines traditional state space modeling with chaotic synchronization and optimal control. Our methods are tailored particularly to situations with considerable measurement noise, sparse observability, very nonlinear or chaotic dynamics, and highly uninformed priors. We illustrate our approach both in a canonical chaotic model and in a phenomenological neuron model, showing that many unknown parameters can be uncovered reliably and accurately from short and noisy observed time traces. Our method holds promise for estimation in larger-scale systems, given ongoing improvements in calcium reporters and genetically-encoded voltage indicators.

Systems neuroscience aims to understand how individual neurons and neural networks process external stimuli into behavioral responses. Underlying this characterization are mathematical models intimately shaped by experimental observations. But neural systems are high-dimensional and contain highly nonlinear interactions, so developing accurate models remains a challenge given current experimental capabilities. In practice, this means that the dynamical equations characterizing neural activity have many unknown parameters, and these parameters must be inferred from data. This inference problem is nontrivial owing to model nonlinearity, system and measurement noise, and the sparsity of observations from electrode recordings. Here, we present a novel method for inferring model parameters of neural systems. Our technique combines ideas from control theory and optimization, and amounts to using data to “control” estimates toward the best fit. Our method compares well in accuracy against other state-of-the-art inference methods, both in phenomenological chaotic systems and biophysical neuron models. Our work shows that many unknown model parameters of interest can be inferred from voltage measurements, despite signaling noise, instrument noise, and low observability.

A multi-Kalman filter-based approach for decoding arm kinematics from EMG recordings

BACKGROUND

Remarkable work has been recently introduced to enhance the usage of Electromyography (EMG) signals in operating prosthetic arms. Despite the rapid advancements in this field, providing a reliable, naturalistic myoelectric prosthesis remains a significant challenge. Other challenges include the limited number of allowed movements, lack of simultaneous, continuous control and the high computational power that could be needed for accurate decoding. In this study, we propose an EMG-based multi-Kalman filter approach to decode arm kinematics; specifically, the elbow angle (θ), wrist joint horizontal (X) and vertical (Y) positions in a continuous and simultaneous manner.

RESULTS

Ten subjects were examined from which we recorded arm kinematics and EMG signals of the biceps, triceps, lateral and anterior deltoid muscles corresponding to a randomized set of movements. The performance of the proposed decoder is assessed using the correlation coefficient (CC) and the normalized root-mean-square error (NRMSE) computed between the actual and the decoded kinematic. Results demonstrate that when training and testing the decoder using same-subject data, an average CC of 0.68 ± 0.1, 0.67 ± 0.12 and 0.64 ± 0.11, and average NRMSE of 0.21 ± 0.06, 0.18 ± 0.03 and 0.24 ± 0.07 were achieved for θ, X, and Y, respectively. When training the decoder using the data of one subject and decoding the data of other subjects, an average CC of 0.61 ± 0.19, 0.61 ± 0.16 and 0.48 ± 0.17, and an average NRMSE of 0.23 ± 0.07, 0.2 ± 0.05 and 0.38 ± 0.15 were achieved for θ, X, and Y, respectively.

CONCLUSIONS

These results suggest the efficacy of the proposed approach and indicates the possibility of obtaining a subject-independent decoder.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Spelling interface using intracortical signals in a completely locked-in patient enabled via auditory neurofeedback training

Patients with amyotrophic lateral sclerosis (ALS) can lose all muscle-based routes of communication as motor neuron degeneration progresses, and ultimately, they may be left without any means of communication. While others have evaluated communication in people with remaining muscle control, to the best of our knowledge, it is not known whether neural-based communication remains possible in a completely locked-in state. Here, we implanted two 64 microelectrode arrays in the supplementary and primary motor cortex of a patient in a completely locked-in state with ALS. The patient modulated neural firing rates based on auditory feedback and he used this strategy to select letters one at a time to form words and phrases to communicate his needs and experiences. This case study provides evidence that brain-based volitional communication is possible even in a completely locked-in state.

Robust Torque Predictions From Electromyography Across Multiple Levels of Active Exoskeleton Assistance Despite Non-linear Reorganization of Locomotor Output

Robotic exoskeletons can assist humans with walking by providing supplemental torque in proportion to the user's joint torque. Electromyographic (EMG) control algorithms can estimate a user's joint torque directly using real-time EMG recordings from the muscles that generate the torque. However, EMG signals change as a result of supplemental torque from an exoskeleton, resulting in unreliable estimates of the user's joint torque during active exoskeleton assistance. Here, we present an EMG control framework for robotic exoskeletons that provides consistent joint torque predictions across varying levels of assistance. Experiments with three healthy human participants showed that using diverse training data (from different levels of assistance) enables robust torque predictions, and that a convolutional neural network (CNN), but not a Kalman filter (KF), can capture the non-linear transformations in EMG due to exoskeleton assistance. With diverse training, the CNN could reliably predict joint torque from EMG during zero, low, medium, and high levels of exoskeleton assistance [root mean squared error (RMSE) below 0.096 N-m/kg]. In contrast, without diverse training, RMSE of the CNN ranged from 0.106 to 0.144 N-m/kg. RMSE of the KF ranged from 0.137 to 0.182 N-m/kg without diverse training, and did not improve with diverse training. When participant time is limited, training data should emphasize the highest levels of assistance first and utilize at least 35 full gait cycles for the CNN. The results presented here constitute an important step toward adaptive and robust human augmentation via robotic exoskeletons. This work also highlights the non-linear reorganization of locomotor output when using assistive exoskeletons; significant reductions in EMG activity were observed for the soleus and gastrocnemius, and a significant increase in EMG activity was observed for the erector spinae. Control algorithms that can accommodate spatiotemporal changes in muscle activity have broad implications for exoskeleton-based assistance and rehabilitation following neuromuscular injury.

Observing cell assemblies from spike train recordings based on the biological basis of synaptic connectivity

Cell assemblies are difficult to observe because they consist of many neurons. We aimed to observe cell assemblies based on biological statistics, such as synaptic connectivity. We developed an estimation method to estimate the activity and synaptic connectivity of cell assemblies from spike trains using mathematical models of individual neurons and cell assemblies. Synaptic transmissions were averaged to generate postsynaptic currents with the same timing and waveform but different amplitudes, as the number of presynaptic neurons was large. We estimated the average synaptic transmission and synaptic connectivity from active cell assemblies based on the stochastic prediction of membrane potentials and verified the estimation ability of the average synaptic transmission and synaptic connectivity using the proposed method on simulated neural activity. Different cell assembly activities evoked by electrical stimuli were correctly sorted into various clusters in experiments using rat cortical neurons cultured on microelectrode arrays. We observed multiple cell assemblies from the spontaneous activity of rat cortical networks on microelectrode arrays, based on the synaptic connectivity patterns estimated by the proposed method. The proposed method was superior to the conventional method for detecting the activity of multiple cell assemblies. Using the proposed method, it is possible to observe multiple cell assemblies based on the biological basis of synaptic connectivity. In summary, we report a novel method to observe cell assemblies from spike train recordings based on the biological basis of synaptic connectivity, rather than merely relying on a statistical method.

Intention Estimation Based Adaptive Unscented Kalman Filter for Online Neural Decoding

The commonly used fixed discrete Kalman filters (DKF) in neural decoders do not generalize well to the actual relationship between neuronal firing rates and movement intention. This is due to the underlying assumption that the neural activity is linearly related to the output state. They also face the issues of requiring large amount of training datasets to achieve a robust model and a degradation of decoding performance over time. In this paper, an adaptive adjustment is made to the conventional unscented Kalman filter (UKF) via intention estimation. This is done by incorporating a history of newly collected state parameters to develop a new set of model parameters. At each time point, a comparative weighted sum of old and new model parameters using matrix squared sums is used to update the neural decoding model parameters. The effectiveness of the resulting adaptive unscented Kalman filter (AUKF) is compared against the discrete Kalman filter and unscented Kalman filter-based algorithms. The results show that the proposed new algorithm provides higher decoding accuracy and stability while requiring less training data.

Robust neural decoding by kernel regression with Siamese representation learning

Objective. Brain-machine interfaces (BMIs) provide a direct pathway between the brain and external devices such as computer cursors and prosthetics, which have great potential in motor function restoration. One critical limitation of current BMI systems is the unstable performance, partly due to the variability of neural signals. Studies showed that neural activities exhibit trial-to-trial variability, and the preferred direction of neurons frequently changes under different conditions. Therefore, a fixed decoding function does not work well.Approach. To deal with the problems, we propose a novel kernel regression framework. The nonparametric kernel regression is used to fit diverse decoding functions by finding similar neural patterns to handle neural variations caused by varying tuning functions. Further, the representations of raw neural signals are learned by Siamese networks and constrained by kinematic parameters, which can alleviate neural variations caused by intrinsic noises and task-irrelevant information. The representations are jointly learned with the kernel regression framework in an end-to-end manner so that neural variations can be tackled effectively.Main results. Experiments on two datasets demonstrate that our approach outperforms most existing methods and significantly improves the robustness in challenging situations such as limited samples and missing channels.Significance. The proposed approach demonstrates robust performance with different conditions and provides a new and inspiring perspective toward robust BMI control.

Capturing spike train temporal pattern with wavelet average coefficient for brain machine interface

Motor brain machine interfaces (BMIs) directly link the brain to artificial actuators and have the potential to mitigate severe body paralysis caused by neurological injury or disease. Most BMI systems involve a decoder that analyzes neural spike counts to infer movement intent. However, many classical BMI decoders (1) fail to take advantage of temporal patterns of spike trains, possibly over long time horizons; (2) are insufficient to achieve good BMI performance at high temporal resolution, as the underlying Gaussian assumption of decoders based on spike counts is violated. Here, we propose a new statistical feature that represents temporal patterns or temporal codes of spike events with richer description-wavelet average coefficients (WAC)-to be used as decoder input instead of spike counts. We constructed a wavelet decoder framework by using WAC features with a sliding-window approach, and compared the resulting decoder against classical decoders (Wiener and Kalman family) and new deep learning based decoders ( Long Short-Term Memory) using spike count features. We found that the sliding-window approach boosts decoding temporal resolution, and using WAC features significantly improves decoding performance over using spike count features.

A Nonlinear Maximum Correntropy Information Filter for High-Dimensional Neural Decoding

Neural signal decoding is a critical technology in brain machine interface (BMI) to interpret movement intention from multi-neural activity collected from paralyzed patients. As a commonly-used decoding algorithm, the Kalman filter is often applied to derive the movement states from high-dimensional neural firing observation. However, its performance is limited and less effective for noisy nonlinear neural systems with high-dimensional measurements. In this paper, we propose a nonlinear maximum correntropy information filter, aiming at better state estimation in the filtering process for a noisy high-dimensional measurement system. We reconstruct the measurement model between the high-dimensional measurements and low-dimensional states using the neural network, and derive the state estimation using the correntropy criterion to cope with the non-Gaussian noise and eliminate large initial uncertainty. Moreover, analyses of convergence and robustness are given. The effectiveness of the proposed algorithm is evaluated by applying it on multiple segments of neural spiking data from two rats to interpret the movement states when the subjects perform a two-lever discrimination task. Our results demonstrate better and more robust state estimation performance when compared with other filters.

Replay of Learned Neural Firing Sequences during Rest in Human Motor Cortex

The offline “replay” of neural firing patterns underlying waking experience, previously observed in non-human animals, is thought to be a mechanism for memory consolidation. Here, we test for replay in the human brain by recording spiking activity from the motor cortex of two participants who had intracortical microelectrode arrays placed chronically as part of a brain-computer interface pilot clinical trial. Participants took a nap before and after playing a neurally controlled sequence-copying game that consists of many repetitions of one “repeated” sequence sparsely interleaved with varying “control” sequences. Both participants performed repeated sequences more accurately than control sequences, consistent with learning. We compare the firing rate patterns that caused the cursor movements when performing each sequence to firing rate patterns throughout both rest periods. Correlations with repeated sequences increase more from pre- to post-task rest than do correlations with control sequences, providing direct evidence of learning-related replay in the human brain.

Eichenlaub et al. show that in the motor cortex of brain-computer interface trial participants, the firing rate patterns corresponding to a previously learned motor sequence are replayed during rest. These findings provide direct evidence of memory replay in the human brain.

An artificial intelligence that increases simulated brain-computer interface performance

Objective.Brain-computer interfaces (BCIs) translate neural activity into control signals for assistive devices in order to help people with motor disabilities communicate effectively. In this work, we introduce a new BCI architecture that improves control of a BCI computer cursor to type on a virtual keyboard.Approach.Our BCI architecture incorporates an external artificial intelligence (AI) that beneficially augments the movement trajectories of the BCI. This AI-BCI leverages past user actions, at both long (100 s of seconds ago) and short (100 s of milliseconds ago) timescales, to modify the BCI's trajectories.Main results.We tested our AI-BCI in a closed-loop BCI simulator with nine human subjects performing a typing task. We demonstrate that our AI-BCI achieves: (1) categorically higher information communication rates, (2) quicker ballistic movements between targets, (3) improved precision control to 'dial in' on targets, and (4) more efficient movement trajectories. We further show that our AI-BCI increases performance across a wide control quality spectrum from poor to proficient control.Significance.This AI-BCI architecture, by increasing BCI performance across all key metrics evaluated, may increase the clinical viability of BCI systems.

Voxel-Based State Space Modeling Recovers Task-Related Cognitive States in Naturalistic fMRI Experiments

Complex natural tasks likely recruit many different functional brain networks, but it is difficult to predict how such tasks will be represented across cortical areas and networks. Previous electrophysiology studies suggest that task variables are represented in a low-dimensional subspace within the activity space of neural populations. Here we develop a voxel-based state space modeling method for recovering task-related state spaces from human fMRI data. We apply this method to data acquired in a controlled visual attention task and a video game task. We find that each task induces distinct brain states that can be embedded in a low-dimensional state space that reflects task parameters, and that attention increases state separation in the task-related subspace. Our results demonstrate that the state space framework offers a powerful approach for modeling human brain activity elicited by complex natural tasks.

Hierarchical Dynamical Model for Multiple Cortical Neural Decoding

Motor brain machine interfaces (BMIs) interpret neural activities from motor-related cortical areas in the brain into movement commands to control a prosthesis. As the subject adapts to control the neural prosthesis, the medial prefrontal cortex (mPFC), upstream of the primary motor cortex (M1), is heavily involved in reward-guided motor learning. Thus, considering mPFC and M1 functionality within a hierarchical structure could potentially improve the effectiveness of BMI decoding while subjects are learning. The commonly used Kalman decoding method with only one simple state model may not be able to represent the multiple brain states that evolve over time as well as along the neural pathway. In addition, the performance of Kalman decoders degenerates in heavy-tailed nongaussian noise, which is usually generated due to the nonlinear neural system or influences of movement-related noise in online neural recording. In this letter, we propose a hierarchical model to represent the brain states from multiple cortical areas that evolve along the neural pathway. We then introduce correntropy theory into the hierarchical structure to address the heavy-tailed noise existing in neural recordings. We test the proposed algorithm on in vivo recordings collected from the mPFC and M1 of two rats when the subjects were learning to perform a lever-pressing task. Compared with the classic Kalman filter, our results demonstrate better movement decoding performance due to the hierarchical structure that integrates the past failed trial information over multisite recording and the combination with correntropy criterion to deal with noisy heavy-tailed neural recordings.

Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison

BACKGROUND

Advanced prostheses can restore function and improve quality of life for individuals with amputations. Unfortunately, most commercial control strategies do not fully utilize the rich control information from residual nerves and musculature. Continuous decoders can provide more intuitive prosthesis control using multi-channel neural or electromyographic recordings. Three components influence continuous decoder performance: the data used to train the algorithm, the algorithm, and smoothing filters on the algorithm's output. Individual groups often focus on a single decoder, so very few studies compare different decoders using otherwise similar experimental conditions.

METHODS

We completed a two-phase, head-to-head comparison of 12 continuous decoders using activities of daily living. In phase one, we compared two training types and a smoothing filter with three algorithms (modified Kalman filter, multi-layer perceptron, and convolutional neural network) in a clothespin relocation task. We compared training types that included only individual digit and wrist movements vs. combination movements (e.g., simultaneous grasp and wrist flexion). We also compared raw vs. nonlinearly smoothed algorithm outputs. In phase two, we compared the three algorithms in fragile egg, zipping, pouring, and folding tasks using the combination training and smoothing found beneficial in phase one. In both phases, we collected objective, performance-based (e.g., success rate), and subjective, user-focused (e.g., preference) measures.

RESULTS

Phase one showed that combination training improved prosthesis control accuracy and speed, and that the nonlinear smoothing improved accuracy but generally reduced speed. Phase one importantly showed simultaneous movements were used in the task, and that the modified Kalman filter and multi-layer perceptron predicted more simultaneous movements than the convolutional neural network. In phase two, user-focused metrics favored the convolutional neural network and modified Kalman filter, whereas performance-based metrics were generally similar among all algorithms.

CONCLUSIONS

These results confirm that state-of-the-art algorithms, whether linear or nonlinear in nature, functionally benefit from training on more complex data and from output smoothing. These studies will be used to select a decoder for a long-term take-home trial with implanted neuromyoelectric devices. Overall, clinical considerations may favor the mKF as it is similar in performance, faster to train, and computationally less expensive than neural networks.

A Nonlinear Latching Filter to Remove Jitter From Movement Estimates for Prostheses

Continuous movement intent decoders are critical for precise control of hand and wrist prostheses. Noise in biological signals (e.g., myoelectric or neural signals) can lead to undesirable jitter in the output of these types of decoders. A low-pass filter (LPF) at the output of the decoder effectively reduces jitter, but also substantially slows intended movements. This paper introduces an alternative, the latching filter (LF), a recursive, nonlinear filter that provides smoothing of small-amplitude jitter but allows quick changes to its output in response to large input changes. The performance of a Kalman filter (KF) decoder smoothed with an LF is compared with that of both an KF decoder without an additional smoother and a KF decoder smoothed with a LPF. These three algorithms were tested in real-time on target holding and target reaching tasks using surface electromyographic signals recorded from 5 non-amputee subjects, and intramuscular electromyographic and peripheral neural signals recorded from an amputee subject. When compared with the LPF, the LF provided a statistically significant improvement in amputee and non-amputee subjects' ability to hold the hand steady at requested positions and achieve movement goals faster. The KF decoder with LF provided a statistically significant improvement in all subjects' ability to hold the prosthetic hand steady, with only slightly lower speeds, when compared to the unsmoothed KF.

Auditory cues reveal intended movement information in middle frontal gyrus neuronal ensemble activity of a person with tetraplegia

Intracortical brain-computer interfaces (iBCIs) allow people with paralysis to directly control assistive devices using neural activity associated with the intent to move. Realizing the full potential of iBCIs critically depends on continued progress in understanding how different cortical areas contribute to movement control. Here we present the first comparison between neuronal ensemble recordings from the left middle frontal gyrus (MFG) and precentral gyrus (PCG) of a person with tetraplegia using an iBCI. As expected, PCG was more engaged in selecting and generating intended movements than in earlier perceptual stages of action planning. By contrast, MFG displayed movement-related information during the sensorimotor processing steps preceding the appearance of the action plan in PCG, but only when the actions were instructed using auditory cues. These results describe a previously unreported function for neurons in the human left MFG in auditory processing contributing to motor control.

Shared Prosthetic Control Based on Multiple Movement Intent Decoders

SIGNIFICANCE

A number of movement intent decoders exist in the literature that typically differ in the algorithms used and the nature of the outputs generated. Each approach comes with its own advantages and disadvantages. Combining the estimates of multiple algorithms may have better performance than any of the individual methods.

OBJECTIVE

This paper presents and evaluates a shared controller framework for prosthetic limbs based on multiple decoders of volitional movement intent.

METHODS

An algorithm to combine multiple estimates to control the prosthesis is developed in this paper. The capabilities of the approach are validated using a system that combines a Kalman filter-based decoder with a multilayer perceptron classifier-based decoder. The shared controller's performance is validated in online experiments where a virtual limb is controlled in real-time by amputee and intact-arm subjects. During the testing phase subjects controlled a virtual hand in real time to move digits to instructed positions using either a Kalman filter decoder, a multilayer perceptron decoder, or a linear combination of the two.

RESULTS

The shared controller results in statistically significant improvements over the component decoders. Specifically, certain degrees of shared control result in increases in the time-in-target metric and decreases in unintended movements.

CONCLUSION

The shared controller of this paper combines the good qualities of component decoders tested in this paper. Herein, combining a Kalman filter decoder with a classifier-based decoder inherits the flexibility of the Kalman filter decoder and the limited unwanted movements from the classifier-based decoder, resulting in a system that may be able to perform the tasks of everyday life more naturally and reliably.

Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification

Neural activity exhibits complex dynamics related to various brain functions, internal states and behaviors. Understanding how neural dynamics explain specific measured behaviors requires dissociating behaviorally relevant and irrelevant dynamics, which is not achieved with current neural dynamic models as they are learned without considering behavior. We develop preferential subspace identification (PSID), which is an algorithm that models neural activity while dissociating and prioritizing its behaviorally relevant dynamics. Modeling data in two monkeys performing three-dimensional reach and grasp tasks, PSID revealed that the behaviorally relevant dynamics are significantly lower-dimensional than otherwise implied. Moreover, PSID discovered distinct rotational dynamics that were more predictive of behavior. Furthermore, PSID more accurately learned behaviorally relevant dynamics for each joint and recording channel. Finally, modeling data in two monkeys performing saccades demonstrated the generalization of PSID across behaviors, brain regions and neural signal types. PSID provides a general new tool to reveal behaviorally relevant neural dynamics that can otherwise go unnoticed.

Bilaterally Mirrored Movements Improve the Accuracy and Precision of Training Data for Supervised Learning of Neural or Myoelectric Prosthetic Control

Intuitive control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time performance, but it is difficult to label hand kinematics accurately after the hand has been amputated. We quantified the accuracy and precision of labeling hand kinematics for two different training approaches: 1) assuming a participant is perfectly mimicking predetermined motions of a prosthesis (mimicked training), and 2) assuming a participant is perfectly mirroring their contralateral hand during identical bilateral movements (mirrored training). We compared these approaches in non-amputee individuals, using an infrared camera to track eight different joint angles of the hands in real-time. Aggregate data revealed that mimicked training does not account for biomechanical coupling or temporal changes in hand posture. Mirrored training was significantly more accurate and precise at labeling hand kinematics. However, when training a modified Kalman filter to estimate motor intent, the mimicked and mirrored training approaches were not significantly different. The results suggest that the mirrored training approach creates a more faithful but more complex dataset. Advanced algorithms, more capable of learning the complex mirrored training dataset, may yield better run-time prosthetic control.

Decoding Kinematic Information From Primary Motor Cortex Ensemble Activities Using a Deep Canonical Correlation Analysis

The control of arm movements through intracortical brain–machine interfaces (BMIs) mainly relies on the activities of the primary motor cortex (M1) neurons and mathematical models that decode their activities. Recent research on decoding process attempts to not only improve the performance but also simultaneously understand neural and behavioral relationships. In this study, we propose an efficient decoding algorithm using a deep canonical correlation analysis (DCCA), which maximizes correlations between canonical variables with the non-linear approximation of mappings from neuronal to canonical variables via deep learning. We investigate the effectiveness of using DCCA for finding a relationship between M1 activities and kinematic information when non-human primates performed a reaching task with one arm. Then, we examine whether using neural activity representations from DCCA improves the decoding performance through linear and non-linear decoders: a linear Kalman filter (LKF) and a long short-term memory in recurrent neural networks (LSTM-RNN). We found that neural representations of M1 activities estimated by DCCA resulted in more accurate decoding of velocity than those estimated by linear canonical correlation analysis, principal component analysis, factor analysis, and linear dynamical system. Decoding with DCCA yielded better performance than decoding the original FRs using LSTM-RNN (6.6 and 16.0% improvement on average for each velocity and position, respectively; Wilcoxon rank sum test, p < 0.05). Thus, DCCA can identify the kinematics-related canonical variables of M1 activities, thus improving the decoding performance. Our results may help advance the design of decoding models for intracortical BMIs.

Portable Take-Home System Enables Proportional Control and High-Resolution Data Logging With a Multi-Degree-of-Freedom Bionic Arm

This paper describes a portable, prosthetic control system and the first at-home use of a multi-degree-of-freedom, proportionally controlled bionic arm. The system uses a modified Kalman filter to provide 6 degree-of-freedom, real-time, proportional control. We describe (a) how the system trains motor control algorithms for use with an advanced bionic arm, and (b) the system's ability to record an unprecedented and comprehensive dataset of EMG, hand positions and force sensor values. Intact participants and a transradial amputee used the system to perform activities-of-daily-living, including bi-manual tasks, in the lab and at home. This technology enables at-home dexterous bionic arm use, and provides a high-temporal resolution description of daily use—essential information to determine clinical relevance and improve future research for advanced bionic arms.

The future of upper extremity rehabilitation robotics: research and practice

The loss of upper limb motor function can have a devastating effect on people's lives. To restore upper limb control and functionality, researchers and clinicians have developed interfaces to interact directly with the human body's motor system. In this invited review, we aim to provide details on the peripheral nerve interfaces and brain-machine interfaces that have been developed in the past 30 years for upper extremity control, and we highlight the challenges that still remain to transition the technology into the clinical market. The findings show that peripheral nerve interfaces and brain-machine interfaces have many similar characteristics that enable them to be concurrently developed. Decoding neural information from both interfaces may lead to novel physiological models that may one day fully restore upper limb motor function for a growing patient population.

Tutorial Review of Bio-Inspired Approaches to Robotic Manipulation for Space Debris Salvage

We present a comprehensive tutorial review that explores the application of bio-inspired approaches to robot control systems for grappling and manipulating a wide range of space debris targets. Current robot manipulator control systems exploit limited techniques which can be supplemented by additional bio-inspired methods to provide a robust suite of robot manipulation technologies. In doing so, we review bio-inspired control methods because this will be the key to enabling such capabilities. In particular, force feedback control may be supplemented with predictive forward models and software emulation of viscoelastic preflexive joint behaviour. This models human manipulation capabilities as implemented by the cerebellum and muscles/joints respectively. In effect, we are proposing a three-level control strategy based on biomimetic forward models for predictive estimation, traditional feedback control and biomimetic muscle-like preflexes. We place emphasis on bio-inspired forward modelling suggesting that all roads lead to this solution for robust and adaptive manipulator control. This promises robust and adaptive manipulation for complex tasks in salvaging space debris.

A state-based probabilistic method for decoding hand position during movement from ECoG signals in non-human primate

OBJECTIVE

In this study, we proposed a state-based probabilistic method for decoding hand positions during unilateral and bilateral movements using the ECoG signals recorded from the brain of Rhesus monkey.

APPROACH

A customized electrode array was implanted subdurally in the right hemisphere of the brain covering from the primary motor cortex to the frontal cortex. Three different experimental paradigms were considered: ipsilateral, contralateral, and bilateral movements. During unilateral movement, the monkey was trained to get food with one hand, while during bilateral movement, the monkey used its left and right hands alternately to get food. To estimate the hand positions, a state-based probabilistic method was introduced which was based on the conditional probability of the hand movement state (i.e. idle, right hand movement, and left hand movement) and the conditional expectation of the hand position for each state. Moreover, a hybrid feature extraction method based on linear discriminant analysis and partial least squares (PLS) was introduced.

MAIN RESULTS

The proposed method could successfully decode the hand positions during ipsilateral, contralateral, and bilateral movements and significantly improved the decoding performance compared to the conventional Kalman and PLS regression methods [Formula: see text]. The proposed hybrid feature extraction method was found to outperform both the PLS and PCA methods [Formula: see text]. Investigating the kinematic information of each frequency band shows that more informative frequency bands were [Formula: see text] (15-30 Hz) and [Formula: see text](50-100 Hz) for ipsilateral and [Formula: see text] and [Formula: see text] (100-200 Hz) for contralateral movements. It is observed that ipsilateral movement was decoded better than contralateral movement for [Formula: see text] (5-15 Hz) and [Formula: see text] bands, while contralateral movements was decoded better for [Formula: see text] (30-200 Hz) and hfECoG (200-400 Hz) bands.

SIGNIFICANCE

Accurate decoding the bilateral movement using the ECoG recorded from one brain hemisphere is an important issue toward real-life applications of the brain-machine interface technologies.

Stabilization of a brain-computer interface via the alignment of low-dimensional spaces of neural activity

The instability of neural recordings can render clinical brain-computer interfaces (BCIs) uncontrollable. Here, we show that the alignment of low-dimensional neural manifolds (low-dimensional spaces that describe specific correlation patterns between neurons) can be used to stabilize neural activity, thereby maintaining BCI performance in the presence of recording instabilities. We evaluated the stabilizer with non-human primates during online cursor control via intracortical BCIs in the presence of severe and abrupt recording instabilities. The stabilized BCIs recovered proficient control under different instability conditions and across multiple days. The stabilizer does not require knowledge of user intent and can outperform supervised recalibration. It stabilized BCIs even when neural activity contained little information about the direction of cursor movement. The stabilizer may be applicable to other neural interfaces and may improve the clinical viability of BCIs.