Encoding of kinetic and kinematic movement parameters in the sensorimotor cortex: A Brain-Computer Interface perspective

A Generalist Intracortical Motor Decoder

Mapping the relationship between neural activity and motor behavior is a central aim of sensorimotor neuroscience and neurotechnology. While most progress to this end has relied on restricting complexity, the advent of foundation models instead proposes integrating a breadth of data as an alternate avenue for broadly advancing downstream modeling. We quantify this premise for motor decoding from intracortical microelectrode data, pretraining an autoregressive Transformer on 2000 hours of neural population spiking activity paired with diverse motor covariates from over 30 monkeys and humans. The resulting model is broadly useful, benefiting decoding on 8 downstream decoding tasks and generalizing to a variety of neural distribution shifts. However, we also highlight that scaling autoregressive Transformers seems unlikely to resolve limitations stemming from sensor variability and output stereotypy in neural datasets. Code: https://github.com/joel99/ndt3.

Decoding the brain-machine interaction for upper limb assistive technologies: advances and challenges

Understanding how the brain encodes upper limb movements is crucial for developing control mechanisms in assistive technologies. Advances in assistive technologies, particularly Brain-machine Interfaces (BMIs), highlight the importance of decoding motor intentions and kinematics for effective control. EEG-based BMI systems show promise due to their non-invasive nature and potential for inducing neural plasticity, enhancing motor rehabilitation outcomes. While EEG-based BMIs show potential for decoding motor intention and kinematics, studies indicate inconsistent correlations with actual or planned movements, posing challenges for achieving precise and reliable prosthesis control. Further, the variability in predictive EEG patterns across individuals necessitates personalized tuning to improve BMI efficiency. Integrating multiple physiological signals could enhance BMI precision and reliability, paving the way for more effective motor rehabilitation strategies. Studies have shown that brain activity adapts to gravitational and inertial constraints during movement, highlighting the critical role of neural adaptation to biomechanical changes in creating control systems for assistive devices. This review aims to provide a comprehensive overview of recent progress in deciphering neural activity patterns associated with both physiological and assisted upper limb movements, highlighting avenues for future exploration in neurorehabilitation and brain-machine interface development.

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

Beta-band power classification of go/no-go arm-reaching responses in the human hippocampus

Objective.Can we classify movement execution and inhibition from hippocampal oscillations during arm-reaching tasks? Traditionally associated with memory encoding, spatial navigation, and motor sequence consolidation, the hippocampus has come under scrutiny for its potential role in movement processing. Stereotactic electroencephalography (SEEG) has provided a unique opportunity to study the neurophysiology of the human hippocampus during motor tasks. In this study, we assess the accuracy of discriminant functions, in combination with principal component analysis (PCA), in classifying between 'Go' and 'No-go' trials in a Go/No-go arm-reaching task.Approach.Our approach centers on capturing the modulation of beta-band (13-30 Hz) power from multiple SEEG contacts in the hippocampus and minimizing the dimensional complexity of channels and frequency bins. This study utilizes SEEG data from the human hippocampus of 10 participants diagnosed with epilepsy. Spectral power was computed during a 'center-out' Go/No-go arm-reaching task, where participants reached or withheld their hand based on a colored cue. PCA was used to reduce data dimension and isolate the highest-variance components within the beta band. The Silhouette score was employed to measure the quality of clustering between 'Go' and 'No-go' trials. The accuracy of five different discriminant functions was evaluated using cross-validation.Main results.The Diagonal-Quadratic model performed best of the 5 classification models, exhibiting the lowest error rate in all participants (median: 9.91%, average: 14.67%). PCA showed that the first two principal components collectively accounted for 54.83% of the total variance explained on average across all participants, ranging from 36.92% to 81.25% among participants.Significance.This study shows that PCA paired with a Diagonal-Quadratic model can be an effective method for classifying between Go/No-go trials from beta-band power in the hippocampus during arm-reaching responses. This emphasizes the significance of hippocampal beta-power modulation in motor control, unveiling its potential implications for brain-computer interface applications.

Decoding the Spike-Band Subthreshold Motor Cortical Activity

Intracortical Brain-Computer Interfaces (iBCI) use single-unit activity (SUA), multiunit activity (MUA) and local field potentials (LFP) to control neuroprosthetic devices. SUA and MUA are usually extracted from the bandpassed recording through amplitude thresholding, while subthreshold data are ignored. Here, we show that subthreshold data can actually be decoded to determine behavioral variables with test set accuracy of up to 100%. Although the utility of SUA, MUA and LFP for decoding behavioral variables has been explored previously, this study investigates the utility of spike-band subthreshold activity exclusively. We provide evidence suggesting that this activity can be used to keep decoding performance at acceptable levels even when SUA quality is reduced over time. To the best of our knowledge, the signals that we derive from the subthreshold activity may be the weakest neural signals that have ever been extracted from extracellular neural recordings, while still being decodable with test set accuracy of up to 100%. These results are relevant for the development of fully data-driven and automated methods for amplitude thresholding spike-band extracellular neural recordings in iBCIs containing thousands of electrodes.

Cortical processing during robot and functional electrical stimulation

Introduction

Like alpha rhythm, the somatosensory mu rhythm is suppressed in the presence of somatosensory inputs by implying cortical excitation. Sensorimotor rhythm (SMR) can be classified into two oscillatory frequency components: mu rhythm (8-13 Hz) and beta rhythm (14-25 Hz). The suppressed/enhanced SMR is a neural correlate of cortical activation related to efferent and afferent movement information. Therefore, it would be necessary to understand cortical information processing in diverse movement situations for clinical applications.

Methods

In this work, the EEG of 10 healthy volunteers was recorded while fingers were moved passively under different kinetic and kinematic conditions for proprioceptive stimulation. For the kinetics aspect, afferent brain activity (no simultaneous volition) was compared under two conditions of finger extension: (1) generated by an orthosis and (2) generated by the orthosis simultaneously combined and assisted with functional electrical stimulation (FES) applied at the forearm muscles related to finger extension. For the kinematic aspect, the finger extension was divided into two phases: (1) dynamic extension and (2) static extension (holding the extended position).

Results

In the kinematic aspect, both mu and beta rhythms were more suppressed during a dynamic than a static condition. However, only the mu rhythm showed a significant difference between kinetic conditions (with and without FES) affected by attention to proprioception after transitioning from dynamic to static state, but the beta rhythm was not.

Discussion

Our results indicate that mu rhythm was influenced considerably by muscle kinetics during finger movement produced by external devices, which has relevant implications for the design of neuromodulation and neurorehabilitation interventions.

Role of path information in visual perception of joint stiffness

Humans have an astonishing ability to extract hidden information from the movement of others. In previous work, subjects observed the motion of a simulated stick-figure, two-link planar arm and estimated its stiffness. Fundamentally, stiffness is the relation between force and displacement. Given that subjects were unable to physically interact with the simulated arm, they were forced to make their estimates solely based on observed kinematic information. Remarkably, subjects were able to correctly correlate their stiffness estimates with changes in the simulated stiffness, despite the lack of force information. We hypothesized that subjects were only able to do this because the controller used to produce the simulated arm's movement, composed of oscillatory motions driving mechanical impedances, resembled the controller humans use to produce their own movement. However, it is still unknown what motion features subjects used to estimate stiffness. Human motion exhibits systematic velocity-curvature patterns, and it has previously been shown that these patterns play an important role in perceiving and interpreting motion. Thus, we hypothesized that manipulating the velocity profile should affect subjects' ability to estimate stiffness. To test this, we changed the velocity profile of the simulated two-link planar arm while keeping the simulated joint paths the same. Even with manipulated velocity signals, subjects were still able to estimate changes in simulated joint stiffness. However, when subjects were shown the same simulated path with different velocity profiles, they perceived motions that followed a veridical velocity profile to be less stiff than that of a non-veridical profile. These results suggest that path information (displacement) predominates over temporal information (velocity) when humans use visual observation to estimate stiffness.

Association of bilateral lower limb coordination while standing with body sway control and aging

PURPOSE

Coordinated movements of both lower limbs may be a clinically important indicator of motor control during quiet standing. From a neurological point of view, it is known that extensive coupling of muscles must be coordinated an upright posture. However, movement coordination between the lower limbs is the final motor output, is unknown. In this study, we focussed on the ground reaction force (GRF) vector and clarified the time and frequency characteristics of the force vectors of both lower limbs.

MATERIALS AND METHODS

A total of 16 healthy young adults and 18 healthy older adults participated and placed each bare foot on one of two force plates to measure the GRF vectors (i.e., anteroposterior, mediolateral, and vertical) of each lower limb and determine the centre of mass (COM) acceleration in the anteroposterior direction (COMacc). Characteristics of the coordination of both lower limbs during movements were analysed using coherence analysis and cross-correlation function analysis (CCF).

RESULTS

The coherence levels of the force vectors of both lower limbs were higher in all three directions and significantly increased in the older adults. CCF analysis showed that the force vectors of both lower limbs were negatively correlated at the zero-time lag. Moreover, a weak correlation was observed between COMacc and coherence values.

CONCLUSIONS

The assessment of bilateral lower limb connectivity using force vectors can be used as an evaluation method to reflect changes in the ability to control bipedal standing during ageing.

Bradykinesia in motoneuron diseases

OBJECTIVE

Only few studies investigated voluntary movement abnormalities in patients with motoneuron diseases (MNDs) or their neurophysiological correlates. We aimed to kinematically assess finger tapping abnormalities in patients with amyotrophic lateral sclerosis (ALS) and primary lateral sclerosis (PLS), as compared to healthy controls (HCs), and their relationship with motoneuron involvement.

METHODS

Fourteen ALS and 5 PLS patients were enrolled. Finger tapping was assessed by a motion analysis system. Patients underwent a central motor conduction time assessment, a motor nerve conduction study, and needle electromyography. Data were compared to those of 79 HCs using non-parametric tests. Possible relationships between clinical, kinematic, and neurophysiological data were assessed in patients.

RESULTS

As a major finding, ALS and PLS patients performed finger tapping slower than HCs. In both conditions, movement slowness correlated with muscle strength. In ALS, movement slowness also correlated with the amplitude of the compound muscle action potential recorded from the muscles involved in the task and with denervation activity. No correlations were found between slowness, measures of upper motoneuron involvement, and other clinical and neurophysiological data.

CONCLUSIONS

This study provides novel information on voluntary movement abnormalities in MNDs.

SIGNIFICANCE

The results highlight the pathophysiological role of motoneurons in generating movement slowness.

Deep learning multimodal fNIRS and EEG signals for bimanual grip force decoding

Objective.Non-invasive brain-machine interfaces (BMIs) offer an alternative, safe and accessible way to interact with the environment. To enable meaningful and stable physical interactions, BMIs need to decode forces. Although previously addressed in the unimanual case, controlling forces from both hands would enable BMI-users to perform a greater range of interactions. We here investigate the decoding of hand-specific forces.Approach.We maximise cortical information by using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) and developing a deep-learning architecture with attention and residual layers (cnnatt) to improve their fusion. Our task required participants to generate hand-specific force profiles on which we trained and tested our deep-learning and linear decoders.Main results.The use of EEG and fNIRS improved the decoding of bimanual force and the deep-learning models outperformed the linear model. In both cases, the greatest gain in performance was due to the detection of force generation. In particular, the detection of forces was hand-specific and better for the right dominant hand andcnnattwas better at fusing EEG and fNIRS. Consequently, the study ofcnnattrevealed that forces from each hand were differently encoded at the cortical level.Cnnattalso revealed traces of the cortical activity being modulated by the level of force which was not previously found using linear models.Significance.Our results can be applied to avoid hand-cross talk during hand force decoding to improve the robustness of BMI robotic devices. In particular, we improve the fusion of EEG and fNIRS signals and offer hand-specific interpretability of the encoded forces which are valuable during motor rehabilitation assessment.

Behavioral and Neural Variability of Naturalistic Arm Movements

Motor behaviors are central to many functions and dysfunctions of the brain, and understanding their neural basis has consequently been a major focus in neuroscience. However, most studies of motor behaviors have been restricted to artificial, repetitive paradigms, far removed from natural movements performed "in the wild." Here, we leveraged recent advances in machine learning and computer vision to analyze intracranial recordings from 12 human subjects during thousands of spontaneous, unstructured arm reach movements, observed over several days for each subject. These naturalistic movements elicited cortical spectral power patterns consistent with findings from controlled paradigms, but with considerable neural variability across subjects and events. We modeled interevent variability using 10 behavioral and environmental features; the most important features explaining this variability were reach angle and day of recording. Our work is among the first studies connecting behavioral and neural variability across cortex in humans during unstructured movements and contributes to our understanding of long-term naturalistic behavior.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

TDCS effects on pointing task learning in young and old adults

Skill increase in motor performance can be defined as explicitly measuring task success but also via more implicit measures of movement kinematics. Even though these measures are often related, there is evidence that they represent distinct concepts of learning. In the present study, the effect of multiple tDCS-sessions on both explicit and implicit measures of learning are investigated in a pointing task in 30 young adults (YA) between 27.07 ± 3.8 years and 30 old adults (OA) between 67.97 years ± 5.3 years. We hypothesized, that OA would show slower explicit skill learning indicated by higher movement times/lower accuracy and slower implicit learning indicated by higher spatial variability but profit more from anodal tDCS compared with YA. We found age-related differences in movement time but not in accuracy or spatial variability. TDCS did not skill learning facilitate learning neither in explicit nor implicit parameters. However, contrary to our hypotheses, we found tDCS-associated higher accuracy only in YA but not in spatial variability. Taken together, our data shows limited overlapping of tDCS effects in explicit and implicit skill parameters. Furthermore, it supports the assumption that tDCS is capable of producing a performance-enhancing brain state at least for explicit skill acquisition.

The Representation of Finger Movement and Force in Human Motor and Premotor Cortices

The ability to grasp and manipulate objects requires controlling both finger movement kinematics and isometric force in rapid succession. Previous work suggests that these behavioral modes are controlled separately, but it is unknown whether the cerebral cortex represents them differently. Here, we asked the question of how movement and force were represented cortically, when executed sequentially with the same finger. We recorded high-density electrocorticography (ECoG) from the motor and premotor cortices of seven human subjects performing a movement-force motor task. We decoded finger movement [0.7 ± 0.3 fractional variance accounted for (FVAF)] and force (0.7 ± 0.2 FVAF) with high accuracy, yet found different spatial representations. In addition, we used a state-of-the-art deep learning method to uncover smooth, repeatable trajectories through ECoG state space during the movement-force task. We also summarized ECoG across trials and participants by developing a new metric, the neural vector angle (NVA). Thus, state-space techniques can help to investigate broad cortical networks. Finally, we were able to classify the behavioral mode from neural signals with high accuracy (90 ± 6%). Thus, finger movement and force appear to have distinct representations in motor/premotor cortices. These results inform our understanding of the neural control of movement, as well as the design of grasp brain-machine interfaces (BMIs).

Human brain function and brain-computer interfaces

Human brain function research has evolved dramatically in the last decades. In this chapter the role of modern methods of recording brain activity in understanding human brain function is explained. Current knowledge of brain function relevant to brain-computer interface (BCI) research is detailed, with an emphasis on the motor system which provides an exceptional level of detail to decoding of intended or attempted movements in paralyzed beneficiaries of BCI technology and translation to computer-mediated actions. BCI technologies that stand to benefit the most of the detailed organization of the human cortex are, and for the foreseeable future are likely to be, reliant on intracranial electrodes. These evolving technologies are expected to enable severely paralyzed people to regain the faculty of movement and speech in the coming decades.

Encoding of kinetic and kinematic movement parameters in the sensorimotor cortex: A Brain-Computer Interface perspective

For severely paralyzed people, Brain-Computer Interfaces (BCIs) can potentially replace lost motor output and provide a brain-based control signal for augmentative and alternative communication devices or neuroprosthetics. Many BCIs focus on neuronal signals acquired from the hand area of the sensorimotor cortex, employing changes in the patterns of neuronal firing or spectral power associated with one or more types of hand movement. Hand and finger movement can be described by two groups of movement features, namely kinematics (spatial and motion aspects) and kinetics (muscles and forces). Despite extensive primate and human research, it is not fully understood how these features are represented in the SMC and how they lead to the appropriate movement. Yet, the available information may provide insight into which features are most suitable for BCI control. To that purpose, the current paper provides an in-depth review on the movement features encoded in the SMC. Even though there is no consensus on how exactly the SMC generates movement, we conclude that some parameters are well represented in the SMC and can be accurately used for BCI control with discrete as well as continuous feedback. However, the vast evidence also suggests that movement should be interpreted as a combination of multiple parameters rather than isolated ones, pleading for further exploration of sensorimotor control models for accurate BCI control.

High-frequency band temporal dynamics in response to a grasp force task

OBJECTIVE

Brain-computer interfaces (BCIs) are being developed to restore reach and grasping movements of paralyzed individuals. Recent studies have shown that the kinetics of grasping movement, such as grasp force, can be successfully decoded from electrocorticography (ECoG) signals, and that the high-frequency band (HFB) power changes provide discriminative information that contribute to an accurate decoding of grasp force profiles. However, as the models used in these studies contained simultaneous information from multiple spectral features over multiple areas in the brain, it remains unclear what parameters of movement and force are encoded by the HFB signals and how these are represented temporally and spatially in the SMC.

APPROACH

To investigate this, and to gain insight in the temporal dynamics of the HFB during grasping, we continuously modelled the ECoG HFB response recorded from nine individuals with epilepsy temporarily implanted with ECoG grids, who performed three different grasp force tasks.

MAIN RESULTS

We show that a model based on the force onset and offset consistently provides a better fit to the HFB power responses when compared with a model based on the force magnitude, irrespective of electrode location.

SIGNIFICANCE

Our results suggest that HFB power, although potentially useful for continuous decoding, is more closely related to the changes in movement. This finding may potentially contribute to the more natural decoding of grasping movement in neural prosthetics.