Going beyond primary motor cortex to improve brain–computer interfaces

Interfacing with the Brain: How Nanotechnology Can Contribute

Interfacing artificial devices with the human brain is the central goal of neurotechnology. Yet, our imaginations are often limited by currently available paradigms and technologies. Suggestions for brain-machine interfaces have changed over time, along with the available technology. Mechanical levers and cable winches were used to move parts of the brain during the mechanical age. Sophisticated electronic wiring and remote control have arisen during the electronic age, ultimately leading to plug-and-play computer interfaces. Nonetheless, our brains are so complex that these visions, until recently, largely remained unreachable dreams. The general problem, thus far, is that most of our technology is mechanically and/or electrically engineered, whereas the brain is a living, dynamic entity. As a result, these worlds are difficult to interface with one another. Nanotechnology, which encompasses engineered solid-state objects and integrated circuits, excels at small length scales of single to a few hundred nanometers and, thus, matches the sizes of biomolecules, biomolecular assemblies, and parts of cells. Consequently, we envision nanomaterials and nanotools as opportunities to interface with the brain in alternative ways. Here, we review the existing literature on the use of nanotechnology in brain-machine interfaces and look forward in discussing perspectives and limitations based on the authors' expertise across a range of complementary disciplines─from neuroscience, engineering, physics, and chemistry to biology and medicine, computer science and mathematics, and social science and jurisprudence. We focus on nanotechnology but also include information from related fields when useful and complementary.

A neural implementation model of feedback-based motor learning

Animals use feedback to rapidly correct ongoing movements in the presence of a perturbation. Repeated exposure to a predictable perturbation leads to behavioural adaptation that compensates for its effects. Here, we tested the hypothesis that all the processes necessary for motor adaptation may emerge as properties of a controller that adaptively updates its policy. We trained a recurrent neural network to control its own output through an error-based feedback signal, which allowed it to rapidly counteract external perturbations. Implementing a biologically plausible plasticity rule based on this same feedback signal enabled the network to learn to compensate for persistent perturbations through a trial-by-trial process. The network activity changes during learning matched those from populations of neurons from monkey primary motor cortex - known to mediate both movement correction and motor adaptation - during the same task. Furthermore, our model natively reproduced several key aspects of behavioural studies in humans and monkeys. Thus, key features of trial-by-trial motor adaptation can arise from the internal properties of a recurrent neural circuit that adaptively controls its output based on ongoing feedback.

The mind-machine connection: adaptive information processing and new technologies promoting mental health in older adults

The human brain demonstrates an exceptional adaptability, which encompasses the ability to regulate emotions, exhibit cognitive flexibility, and generate behavioral responses, all supported by neuroplasticity. Brain-computer interfaces (BCIs) employ adaptive algorithms and machine learning techniques to adapt to variations in the user's brain activity, allowing for customized interactions with external devices. Older adults may experience cognitive decline, which could affect the ability to learn and adapt to new technologies such as BCIs, but both (human brain and BCI) demonstrate adaptability in their responses. The human brain is skilled at quickly switching between tasks and regulating emotions, while BCIs can modify signal-processing algorithms to accommodate changes in brain activity. Furthermore, the human brain and BCI participate in knowledge acquisition; the first one strengthens cognitive abilities through exposure to new experiences, and the second one improves performance through ongoing adjustment and improvement. Current research seeks to incorporate emotional states into BCI systems to improve the user experience, despite the exceptional emotional regulation abilities of the human brain. The implementation of BCIs for older adults could be more effective, inclusive, and beneficial in improving their quality of life. This review aims to improve the understanding of brain-machine interfaces and their implications for mental health in older adults.

Reward signals in the motor cortex: from biology to neurotechnology

Over the past decade, research has shown that the primary motor cortex (M1), the brain's main output for movement, also responds to rewards. These reward signals may shape motor output in its final stages, influencing movement invigoration and motor learning. In this Perspective, we highlight the functional roles of M1 reward signals and propose how they could guide advances in neurotechnologies for movement restoration, specifically brain-computer interfaces and non-invasive brain stimulation. Understanding M1 reward signals may open new avenues for enhancing motor control and rehabilitation.

Deciphering brain activation during wrist movements: comparative fMRI and fNIRS analysis of active, passive, and imagery states

Understanding the complex activation patterns of brain regions during motor tasks is crucial. Integrated functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS) offers advanced insights into how brain activity fluctuates with motor activities. This study explores neuronal activation patterns in the cerebral cortex during active, passive, and imagined wrist movements using these functional imaging techniques. Data were collected from 10 right-handed volunteers performing a motor task using fMRI and fNIRS. fMRI utilized a 3T scanner and a 20-channel head coil, while fNIRS recorded data with a 48-channel device at 765 nm and 855 nm. Analysis focused on key motor and sensory cortices using NIRS-SPM and SPM12, applying a significance threshold of p < 0.05 and a minimum cluster size of 10 voxels for group analysis. Super-threshold voxels were identified with FWE thresholding in SPM12. For activation map extraction we focused on the primary motor cortex, primary somatosensory cortex, somatosensory association cortex, premotor cortex, and supplementary motor cortex. Both fMRI and fNIRS detected activation in the primary motor cortex (M1). The primary somatosensory cortex was found to influence movement direction coding, with smaller activation sizes for upward movements. Combining fNIRS with fMRI provided clearer differentiation of brain activation patterns for wrist movements in various directions and conditions (p < 0.05). This study highlights variations in left motor cortex activity across different movement states. fNIRS proved effective in detecting brain function and showed strong correlation with fMRI results, suggesting it as a viable alternative for those unable to undergo fMRI.

A dance movement quality evaluation model using transformer encoder and convolutional neural network

The purpose of this study is to put forward a new evaluation model of dance movement quality to deal with the subjectivity and inconsistency in traditional evaluation methods. In view of the complexity and diversity of dance art and the widespread popularity of dance videos on social media, it is particularly urgent to develop an automatic and efficient tool for evaluating the quality of dance movements. Therefore, this study puts forward the Transformer Convolutional Neural Network with Dynamic and Static Streams (TransCNN-DSSS) model, which combines the analysis of dynamic flow and static flow, and makes use of the advantages of Transformer and Convolutional Neural Network (CNN) to deeply analyze and evaluate the dance movements. The core of the model is Quality Score Decoupling (QSD), which decouples and weights different quality dimensions of dance movements through attention mechanism, such as accuracy, fluency and expressiveness. Score Prediction module (SPM) uses Transformer network to further process the fused features, and outputs the final evaluation score through the full connection layer. In the experimental part, the TransCNN-DSSS model is trained and tested on the marked dance movement dataset. The performance of the model is evaluated by accuracy, recall and F1 score. The results show that the model has achieved 90% accuracy, 89% recall and F1 score of 0.90 in the task of evaluating the quality of dance movements. These results prove the effectiveness and reliability of the model. In addition, the adaptability test of the model in different dance styles also shows good generalization ability. The research contribution of this study is to put forward a new evaluation model of dance movement quality, which provides an objective and automatic evaluation tool for dance teaching, competition scoring and fans.

Superior cortical venous anatomy for endovascular device implantation: a systematic review

Endovascular electrode arrays provide a minimally invasive approach to access intracranial structures for neural recording and stimulation. These arrays are currently used as brain-computer interfaces (BCIs) and are deployed within the superior sagittal sinus (SSS), although cortical vein implantation could improve the quality and quantity of recorded signals. However, the anatomy of the superior cortical veins is heterogenous and poorly characterised. MEDLINE and Embase databases were systematically searched from inception to December 15, 2023 for studies describing the anatomy of the superior cortical veins. A total of 28 studies were included: 19 cross-sectional imaging studies, six cadaveric studies, one intraoperative anatomical study and one review. There was substantial variability in cortical vein diameter, length, confluence angle, and location relative to the underlying cortex. The mean number of SSS branches ranged from 11 to 45. The vein of Trolard was most often reported as the largest superior cortical vein, with a mean diameter ranging from 2.1 mm to 3.3 mm. The mean vein of Trolard was identified posterior to the central sulcus. One study found a significant age-related variability in cortical vein diameter and another identified myoendothelial sphincters at the base of the cortical veins. Cortical vein anatomical data are limited and inconsistent. The vein of Trolard is the largest tributary vein of the SSS; however, its relation to the underlying cortex is variable. Variability in cortical vein anatomy may necessitate individualized pre-procedural planning of training and neural decoding in endovascular BCI. Future focus on the relation to the underlying cortex, sulcal vessels, and vessel wall anatomy is required.

Decoding Single and Paired Phonemes Using 7T Functional MRI

Several studies have shown that mouth movements related to the pronunciation of individual phonemes are represented in the sensorimotor cortex. This would theoretically allow for brain computer interfaces that are capable of decoding continuous speech by training classifiers based on the activity in the sensorimotor cortex related to the production of individual phonemes. To address this, we investigated the decodability of trials with individual and paired phonemes (pronounced consecutively with one second interval) using activity in the sensorimotor cortex. Fifteen participants pronounced 3 different phonemes and 3 combinations of two of the same phonemes in a 7T functional MRI experiment. We confirmed that support vector machine (SVM) classification of single and paired phonemes was possible. Importantly, by combining classifiers trained on single phonemes, we were able to classify paired phonemes with an accuracy of 53% (33% chance level), demonstrating that activity of isolated phonemes is present and distinguishable in combined phonemes. A SVM searchlight analysis showed that the phoneme representations are widely distributed in the ventral sensorimotor cortex. These findings provide insights about the neural representations of single and paired phonemes. Furthermore, it supports the notion that speech BCI may be feasible based on machine learning algorithms trained on individual phonemes using intracranial electrode grids.

Neural subspaces of imagined movements in parietal cortex remain stable over several years in humans

Objective.A crucial goal in brain-machine interfacing is the long-term stability of neural decoding performance, ideally without regular retraining. Long-term stability has only been previously demonstrated in non-human primate experiments and only in primary sensorimotor cortices. Here we extend previous methods to determine long-term stability in humans by identifying and aligning low-dimensional structures in neural data.Approach.Over a period of 1106 and 871 d respectively, two participants completed an imagined center-out reaching task. The longitudinal accuracy between all day pairs was assessed by latent subspace alignment using principal components analysis and canonical correlations analysis of multi-unit intracortical recordings in different brain regions (Brodmann Area 5, Anterior Intraparietal Area and the junction of the postcentral and intraparietal sulcus).Main results.We show the long-term stable representation of neural activity in subspaces of intracortical recordings from higher-order association areas in humans.Significance.These results can be practically applied to significantly expand the longevity and generalizability of brain-computer interfaces.Clinical TrialsNCT01849822, NCT01958086, NCT01964261.

Improving efficacy of repetitive transcranial magnetic stimulation for treatment of Parkinson disease gait disorders

Parkinson disease (PD) is a neurodegenerative disorder that causes motor and cognitive deficits, presenting complex challenges for therapeutic interventions. Repetitive transcranial magnetic stimulation (rTMS) is a type of neuromodulation that can produce plastic changes in neural activity. rTMS has been trialed as a therapy to treat motor and non-motor symptoms in persons with Parkinson disease (PwP), particularly treatment-refractory postural instability and gait difficulties such as Freezing of Gait (FoG), but clinical outcomes have been variable. We suggest improving rTMS neuromodulation therapy for balance and gait abnormalities in PwP by targeting brain regions in cognitive-motor control networks. rTMS studies in PwP often targeted motor targets such as the primary motor cortex (M1) or supplementary motor area (SMA), overlooking network interactions involved in posture-gait control disorders. We propose a shift in focus toward alternative stimulation targets in basal ganglia-cortex-cerebellum networks involved in posture-gait control, emphasizing the dorsolateral prefrontal cortex (dlPFC), cerebellum (CB), and posterior parietal cortex (PPC) as potential targets. rTMS might also be more effective if administered during behavioral tasks designed to activate posture-gait control networks during stimulation. Optimizing stimulation parameters such as dosage and frequency as used clinically for the treatment of depression may also be useful. A network-level perspective suggests new directions for exploring optimal rTMS targets and parameters to maximize neural plasticity to treat postural instabilities and gait difficulties in PwP.

An EEG channel selection method for motor imagery based on Fisher score and local optimization

Objective. Multi-channel electroencephalogram (EEG) technology in brain-computer interface (BCI) research offers the advantage of enhanced spatial resolution and system performance. However, this also implies that more time is needed in the data processing stage, which is not conducive to the rapid response of BCI. Hence, it is a necessary and challenging task to reduce the number of EEG channels while maintaining decoding effectiveness.Approach. In this paper, we propose a local optimization method based on the Fisher score for within-subject EEG channel selection. Initially, we extract the common spatial pattern characteristics of EEG signals in different bands, calculate Fisher scores for each channel based on these characteristics, and rank them accordingly. Subsequently, we employ a local optimization method to finalize the channel selection.Main results. On the BCI Competition IV Dataset IIa, our method selects an average of 11 channels across four bands, achieving an average accuracy of 79.37%. This represents a 6.52% improvement compared to using the full set of 22 channels. On our self-collected dataset, our method similarly achieves a significant improvement of 24.20% with less than half of the channels, resulting in an average accuracy of 76.95%.Significance. This research explores the importance of channel combinations in channel selection tasks and reveals that appropriately combining channels can further enhance the quality of channel selection. The results indicate that the model selected a small number of channels with higher accuracy in two-class motor imagery EEG classification tasks. Additionally, it improves the portability of BCI systems through channel selection and combinations, offering the potential for the development of portable BCI systems.

Biomimetic versus arbitrary motor control strategies for bionic hand skill learning

A long-standing engineering ambition has been to design anthropomorphic bionic limbs: devices that look like and are controlled in the same way as the biological body (biomimetic). The untested assumption is that biomimetic motor control enhances device embodiment, learning, generalization and automaticity. To test this, we compared biomimetic and non-biomimetic control strategies for non-disabled participants when learning to control a wearable myoelectric bionic hand operated by an eight-channel electromyography pattern-recognition system. We compared motor learning across days and behavioural tasks for two training groups: biomimetic (mimicking the desired bionic hand gesture with biological hand) and arbitrary control (mapping an unrelated biological hand gesture with the desired bionic gesture). For both trained groups, training improved bionic limb control, reduced cognitive reliance and increased embodiment over the bionic hand. Biomimetic users had more intuitive and faster control early in training. Arbitrary users matched biomimetic performance later in training. Furthermore, arbitrary users showed increased generalization to a new control strategy. Collectively, our findings suggest that biomimetic and arbitrary control strategies provide different benefits. The optimal strategy is probably not strictly biomimetic, but rather a flexible strategy within the biomimetic-to-arbitrary spectrum, depending on the user, available training opportunities and user requirements.

Nonlinear manifolds underlie neural population activity during behaviour

There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.

An Investigation of Manifold-Based Direct Control for a Brain-to-Body Neural Bypass

Objective: Brain-body interfaces (BBIs) have emerged as a very promising solution for restoring voluntary hand control in people with upper-limb paralysis. The BBI module decoding motor commands from brain signals should provide the user with intuitive, accurate, and stable control. Here, we present a preliminary investigation in a monkey of a brain decoding strategy based on the direct coupling between the activity of intrinsic neural ensembles and output variables, aiming at achieving ease of learning and long-term robustness. Results: We identified an intrinsic low-dimensional space (called manifold) capturing the co-variation patterns of the monkey's neural activity associated to reach-to-grasp movements. We then tested the animal's ability to directly control a computer cursor using cortical activation along the manifold axes. By daily recalibrating only scaling factors, we achieved rapid learning and stable high performance in simple, incremental 2D tasks over more than 12 weeks of experiments. Finally, we showed that this brain decoding strategy can be effectively coupled to peripheral nerve stimulation to trigger voluntary hand movements. Conclusions: These results represent a proof of concept of manifold-based direct control for BBI applications.

Altered resting-state brain function in endurance athletes

Previous research has confirmed significant differences in regional brain activity and functional connectivity between endurance athletes and non-athletes. However, no studies have investigated the differences in topological efficiency of the brain functional network between endurance athletes and non-athletes. Here, we compared differences in regional activities, functional connectivity, and topological properties to explore the functional basis associated with endurance training. The results showed significant correlations between Regional Homogeneity in the motor cortex, visual cortex, cerebellum, and the training intensity parameters. Alterations in functional connectivity among the motor cortex, visual cortex, cerebellum, and the inferior frontal gyrus and cingulate gyrus were significantly correlated with training intensity parameters. In addition, the graph theoretical analysis results revealed a significant reduction in global efficiency among athletes. This decline is mainly caused by decreased nodal efficiency and nodal local efficiency of the cerebellar regions. Notably, the sensorimotor regions, such as the precentral gyrus and supplementary motor areas, still exhibit increased nodal efficiency and nodal local efficiency. This study not only confirms the improvement of regional activity in brain regions related to endurance training, but also offers novel insights into the mechanisms through which endurance athletes undergo changes in the topological efficiency of the brain functional network.

NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

A review of combined functional neuroimaging and motion capture for motor rehabilitation

BACKGROUND

Technological advancements in functional neuroimaging and motion capture have led to the development of novel methods that facilitate the diagnosis and rehabilitation of motor deficits. These advancements allow for the synchronous acquisition and analysis of complex signal streams of neurophysiological data (e.g., EEG, fNIRS) and behavioral data (e.g., motion capture). The fusion of those data streams has the potential to provide new insights into cortical mechanisms during movement, guide the development of rehabilitation practices, and become a tool for assessment and therapy in neurorehabilitation.

RESEARCH OBJECTIVE

This paper aims to review the existing literature on the combined use of motion capture and functional neuroimaging in motor rehabilitation. The objective is to understand the diversity and maturity of technological solutions employed and explore the clinical advantages of this multimodal approach.

METHODS

This paper reviews literature related to the combined use of functional neuroimaging and motion capture for motor rehabilitation following the PRISMA guidelines. Besides study and participant characteristics, technological aspects of the used systems, signal processing methods, and the nature of multimodal feature synchronization and fusion were extracted.

RESULTS

Out of 908 publications, 19 were included in the final review. Basic or translation studies were mainly represented and based predominantly on healthy participants or stroke patients. EEG and mechanical motion capture technologies were most used for biomechanical data acquisition, and their subsequent processing is based mainly on traditional methods. The system synchronization techniques at large were underreported. The fusion of multimodal features mainly supported the identification of movement-related cortical activity, and statistical methods were occasionally employed to examine cortico-kinematic relationships.

CONCLUSION

The fusion of motion capture and functional neuroimaging might offer advantages for motor rehabilitation in the future. Besides facilitating the assessment of cognitive processes in real-world settings, it could also improve rehabilitative devices' usability in clinical environments. Further, by better understanding cortico-peripheral coupling, new neuro-rehabilitation methods can be developed, such as personalized proprioceptive training. However, further research is needed to advance our knowledge of cortical-peripheral coupling, evaluate the validity and reliability of multimodal parameters, and enhance user-friendly technologies for clinical adaptation.

Fog Computing for Control of Cyber-Physical Systems in Industry Using BCI

Brain-computer interfaces use signals from the brain, such as EEG, to determine brain states, which in turn can be used to issue commands, for example, to control industrial machinery. While Cloud computing can aid in the creation and operation of industrial multi-user BCI systems, the vast amount of data generated from EEG signals can lead to slow response time and bandwidth problems. Fog computing reduces latency in high-demand computation networks. Hence, this paper introduces a fog computing solution for BCI processing. The solution consists in using fog nodes that incorporate machine learning algorithms to convert EEG signals into commands to control a cyber-physical system. The machine learning module uses a deep learning encoder to generate feature images from EEG signals that are subsequently classified into commands by a random forest. The classification scheme is compared using various classifiers, being the random forest the one that obtained the best performance. Additionally, a comparison was made between the fog computing approach and using only cloud computing through the use of a fog computing simulator. The results indicate that the fog computing method resulted in less latency compared to the solely cloud computing approach.

Advanced Electrode Technologies for Noninvasive Brain-Computer Interfaces

Brain-computer interfaces (BCIs) have garnered significant attention in recent years due to their potential applications in medical, assistive, and communication technologies. Building on this, noninvasive BCIs stand out as they provide a safe and user-friendly method for interacting with the human brain. In this work, we provide a comprehensive overview of the latest developments and advancements in material, design, and application of noninvasive BCIs electrode technology. We also explore the challenges and limitations currently faced by noninvasive BCI electrode technology and sketch out the technological roadmap from three dimensions: Materials and Design; Performances; Mode and Function. We aim to unite research efforts within the field of noninvasive BCI electrode technology, focusing on the consolidation of shared goals and fostering integrated development strategies among a diverse array of multidisciplinary researchers.

Preserved neural dynamics across animals performing similar behaviour

Animals of the same species exhibit similar behaviours that are advantageously adapted to their body and environment. These behaviours are shaped at the species level by selection pressures over evolutionary timescales. Yet, it remains unclear how these common behavioural adaptations emerge from the idiosyncratic neural circuitry of each individual. The overall organization of neural circuits is preserved across individuals1 because of their common evolutionarily specified developmental programme2-4. Such organization at the circuit level may constrain neural activity5-8, leading to low-dimensional latent dynamics across the neural population9-11. Accordingly, here we suggested that the shared circuit-level constraints within a species would lead to suitably preserved latent dynamics across individuals. We analysed recordings of neural populations from monkey and mouse motor cortex to demonstrate that neural dynamics in individuals from the same species are surprisingly preserved when they perform similar behaviour. Neural population dynamics were also preserved when animals consciously planned future movements without overt behaviour12 and enabled the decoding of planned and ongoing movement across different individuals. Furthermore, we found that preserved neural dynamics extend beyond cortical regions to the dorsal striatum, an evolutionarily older structure13,14. Finally, we used neural network models to demonstrate that behavioural similarity is necessary but not sufficient for this preservation. We posit that these emergent dynamics result from evolutionary constraints on brain development and thus reflect fundamental properties of the neural basis of behaviour.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodents has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguishes control types at the go cue and target acquisition, respectively, while M1 best predicts target-direction at both task events. We also find effective connectivity from DLPFC → M1 throughout both control types and Cd → M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Effectiveness of motor and prefrontal cortical areas for brain-controlled functional electrical stimulation neuromodulation

Objective. Brain-computer interface (BCI)-controlled functional electrical stimulation (FES) could excite the central nervous system to enhance upper limb motor recovery. Our current study assessed the effectiveness of motor and prefrontal cortical activity-based BCI-FES to help elucidate the underlying neuromodulation mechanisms of this neurorehabilitation approach.Approach. The primary motor cortex (M1) and prefrontal cortex (PFC) BCI-FES interventions were performed for 25 min on separate days with twelve non-disabled participants. During the interventions, a single electrode from the contralateral M1 or PFC was used to detect event-related desynchronization (ERD) in the calibrated frequency range. If the BCI system detected ERD within 15 s of motor imagery, FES activated wrist extensor muscles. Otherwise, if the BCI system did not detect ERD within 15 s, a subsequent trial was initiated without FES. To evaluate neuromodulation effects, corticospinal excitability was assessed using single-pulse transcranial magnetic stimulation, and cortical excitability was assessed by motor imagery ERD and resting-state functional connectivity before, immediately, 30 min, and 60 min after each intervention.Main results. M1 and PFC BCI-FES interventions had similar success rates of approximately 80%, while the M1 intervention was faster in detecting ERD activity. Consequently, only the M1 intervention effectively elicited corticospinal excitability changes for at least 60 min around the targeted cortical area in the M1, suggesting a degree of spatial localization. However, cortical excitability measures did not indicate changes after either M1 or PFC BCI-FES.Significance. Neural mechanisms underlying the effectiveness of BCI-FES neuromodulation may be attributed to the M1 direct corticospinal projections and/or the closer timing between ERD detection and FES, which likely enhanced Hebbian-like plasticity by synchronizing cortical activation detected by the BCI system with the sensory nerve activation and movement related reafference elicited by FES.

Hierarchical action encoding in prefrontal cortex of freely moving macaques

Our natural behavioral repertoires include coordinated actions of characteristic types. To better understand how neural activity relates to the expression of actions and action switches, we studied macaques performing a freely moving foraging task in an open environment. We developed a novel analysis pipeline that can identify meaningful units of behavior, corresponding to recognizable actions such as sitting, walking, jumping, and climbing. On the basis of transition probabilities between these actions, we found that behavior is organized in a modular and hierarchical fashion. We found that, after regressing out many potential confounders, actions are associated with specific patterns of firing in each of six prefrontal brain regions and that, overall, encoding of action category is progressively stronger in more dorsal and more caudal prefrontal regions. Together, these results establish a link between selection of units of primate behavior on one hand and neuronal activity in prefrontal regions on the other.

Neural subspaces of imagined movements in parietal cortex remain stable over several years in humans

A crucial goal in brain-machine interfacing is long-term stability of neural decoding performance, ideally without regular retraining. Here we demonstrate stable neural decoding over several years in two human participants, achieved by latent subspace alignment of multi-unit intracortical recordings in posterior parietal cortex. These results can be practically applied to significantly expand the longevity and generalizability of future movement decoding devices.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Distinct neural representations during a brain-machine interface and manual reaching task in motor cortex, prefrontal cortex, and striatum

Although brain-machine interfaces (BMIs) are directly controlled by the modulation of a select local population of neurons, distributed networks consisting of cortical and subcortical areas have been implicated in learning and maintaining control. Previous work in rodent BMI has demonstrated the involvement of the striatum in BMI learning. However, the prefrontal cortex has been largely ignored when studying motor BMI control despite its role in action planning, action selection, and learning abstract tasks. Here, we compare local field potentials simultaneously recorded from the primary motor cortex (M1), dorsolateral prefrontal cortex (DLPFC), and the caudate nucleus of the striatum (Cd) while nonhuman primates perform a two-dimensional, self-initiated, center-out task under BMI control and manual control. Our results demonstrate the presence of distinct neural representations for BMI and manual control in M1, DLPFC, and Cd. We find that neural activity from DLPFC and M1 best distinguish between control types at the go cue and target acquisition, respectively. We also found effective connectivity from DLPFC→M1 throughout trials across both control types and Cd→M1 during BMI control. These results suggest distributed network activity between M1, DLPFC, and Cd during BMI control that is similar yet distinct from manual control.

Neurocognitive and motor-control challenges for the realization of bionic augmentation

Robotic fingers and arms that augment the motor abilities of non-disabled individuals are increasingly feasible yet face neurocognitive barriers and hurdles in efferent motor control.

Should bionic limb control mimic the human body? Impact of control strategy on bionic hand skill learning

A longstanding engineering ambition has been to design anthropomorphic bionic limbs: devices that look like and are controlled in the same way as the biological body (biomimetic). The untested assumption is that biomimetic motor control enhances device embodiment, learning, generalization, and automaticity. To test this, we compared biomimetic and non-biomimetic control strategies for able-bodied participants when learning to operate a wearable myoelectric bionic hand. We compared motor learning across days and behavioural tasks for two training groups: Biomimetic (mimicking the desired bionic hand gesture with biological hand) and Arbitrary control (mapping an unrelated biological hand gesture with the desired bionic gesture). For both trained groups, training improved bionic limb control, reduced cognitive reliance, and increased embodiment over the bionic hand. Biomimetic users had more intuitive and faster control early in training. Arbitrary users matched biomimetic performance later in training. Further, arbitrary users showed increased generalization to a novel control strategy. Collectively, our findings suggest that biomimetic and arbitrary control strategies provide different benefits. The optimal strategy is likely not strictly biomimetic, but rather a flexible strategy within the biomimetic to arbitrary spectrum, depending on the user, available training opportunities and user requirements.

Flexible intentions: An Active Inference theory

We present a normative computational theory of how the brain may support visually-guided goal-directed actions in dynamically changing environments. It extends the Active Inference theory of cortical processing according to which the brain maintains beliefs over the environmental state, and motor control signals try to fulfill the corresponding sensory predictions. We propose that the neural circuitry in the Posterior Parietal Cortex (PPC) compute flexible intentions-or motor plans from a belief over targets-to dynamically generate goal-directed actions, and we develop a computational formalization of this process. A proof-of-concept agent embodying visual and proprioceptive sensors and an actuated upper limb was tested on target-reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, different sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by dynamic and flexible intentions can thus support goal-directed behavior in constantly changing environments, and the PPC might putatively host its core intention mechanism. More broadly, the study provides a normative computational basis for research on goal-directed behavior in end-to-end settings and further advances mechanistic theories of active biological systems.

Movement decoding using spatio-spectral features of cortical and subcortical local field potentials

The first commercially sensing enabled deep brain stimulation (DBS) devices for the treatment of movement disorders have recently become available. In the future, such devices could leverage machine learning based brain signal decoding strategies to individualize and adapt therapy in real-time. As multi-channel recordings become available, spatial information may provide an additional advantage for informing machine learning models. To investigate this concept, we compared decoding performances from single channels vs. spatial filtering techniques using intracerebral multitarget electrophysiology in Parkinson's disease patients undergoing DBS implantation. We investigated the feasibility of spatial filtering in invasive neurophysiology and the putative utility of combined cortical ECoG and subthalamic local field potential signals for decoding grip-force, a well-defined and continuous motor readout. We found that adding spatial information to the model can improve decoding (6% gain in decoding), but the spatial patterns and additional benefit was highly individual. Beyond decoding performance results, spatial filters and patterns can be used to obtain meaningful neurophysiological information about the brain networks involved in target behavior. Our results highlight the importance of individualized approaches for brain signal decoding, for which multielectrode recordings and spatial filtering can improve precision medicine approaches for clinical brain computer interfaces.

Somatosensory ECoG-based brain-machine interface with electrical stimulation on medial forebrain bundle

Brain-machine interface (BMI) provides an alternative route for controlling an external device with one's intention. For individuals with motor-related disability, the BMI technologies can be used to replace or restore motor functions. Therefore, BMIs for movement restoration generally decode the neural activity from the motor-related brain regions. In this study, however, we designed a BMI system that uses sensory-related neural signals for BMI combined with electrical stimulation for reward. Four-channel electrocorticographic (ECoG) signals were recorded from the whisker-related somatosensory cortex of rats and converted to extract the BMI signals to control the one-dimensional movement of a dot on the screen. At the same time, we used operant conditioning with electrical stimulation on medial forebrain bundle (MFB), which provides a virtual reward to motivate the rat to move the dot towards the desired center region. The BMI task training was performed for 7 days with ECoG recording and MFB stimulation. Animals successfully learned to move the dot location to the desired position using S1BF neural activity. This study successfully demonstrated that it is feasible to utilize the neural signals from the whisker somatosensory cortex for BMI system. In addition, the MFB electrical stimulation is effective for rats to learn the behavioral task for BMI.

Decoding trajectories of imagined hand movement using electrocorticograms for brain-machine interface

OBJECTIVE

Reaching hand movement is an important motor skill actively examined in brain-computer interface (BCI). Among various components of movement analyzed is the hand's trajectory, which describes the hand's continuous positions in three-dimensional space. While a large body of studies have investigated the decoding of real movements and the reconstruction of real hand movement trajectories from neural signals, fewer studies have attempted to decode the trajectory of imagined hand movement. To develop BCI systems for patients with hand motor dysfunctions, the systems essentially require to achieve movement-free control of external devices, which is only possible through successful decoding of purely imagined hand movement.

APPROACH

To achieve this goal, this study used a machine learning technique (i.e., the variational Bayesian least square) to analyze the electrocorticogram (ECoG) of eighteen epilepsy patients obtained from when they performed movement execution (ME) and kinesthetic movement imagination (KMI) of the reach-and-grasp hand action.

MAIN RESULTS

The variational Bayesian decoding model was able to successfully predict the imagined trajectories of hand movement significantly above chance level. The Pearson's correlation coefficient between imagined and predicted trajectories was 0.3393 and 0.4936 for the KMI (KMI trials only) and MEKMI paradigm (alternating trials of ME and KMI) respectively.

SIGNIFICANCE

This study demonstrated a high accuracy of prediction for trajectories of imagined hand movement, and more importantly, higher decoding accuracy of imagined trajectories in the MEKMI paradigm than in the KMI paradigm solely.

Extracting synchronized neuronal activity from local field potentials based on a marked point process framework

OBJECTIVE

Brain-machine interfaces (BMIs) translate neural activity into motor commands to restore motor functions for people with paralysis. Local field potentials (LFPs) are promising for long-term BMIs, since the quality of the recording lasts longer than single neuronal spikes. Inferring neuronal spike activity from population activities such as LFPs is challenging, because LFPs stem from synaptic currents flowing in the neural tissue produced by various neuronal ensembles and reflect neural synchronization. Existing studies that combine LFPs with spikes leverage the spectrogram of LFPs, which can neither detect the transient characteristics of LFP features (here, neuromodulation in a specific frequency band) with high accuracy, nor correlate them with relevant neuronal activity with a sufficient time resolution.

APPROACH

We propose a feature extraction and validation framework to directly extract LFP neuromodulations related to synchronized spike activity using recordings from the primary motor cortex of six Sprague Dawley (SD) rats during a lever-press task. We first select important LFP frequency bands relevant to behavior, and then implement a marked point process (MPP) methodology to extract transient LFP neuromodulations. We validate the LFP feature extraction by examining the correlation with the pairwise synchronized firing probability of important neurons, which are selected according to their contribution to behavioral decoding. The highly correlated synchronized firings identified by the LFP neuromodulations are fed into a decoder to check whether they can serve as a reliable neural data source for movement decoding.

MAIN RESULTS

We find that the gamma band (30-80Hz) LFP neuromodulations demonstrate significant correlation with synchronized firings. Compared with traditional spectrogram-based method, the higher-temporal resolution MPP method captures the synchronized firing patterns with fewer false alarms, and demonstrates significantly higher correlation than single neuron spikes. The decoding performance using the synchronized neuronal firings identified by the LFP neuromodulations can reach 90% compared to the full recorded neuronal ensembles.

SIGNIFICANCE

Our proposed framework successfully extracts the sparse LFP neuromodulations that can identify temporal synchronized neuronal spikes with high correlation. The identified neuronal spike pattern demonstrates high decoding performance, which reveals the possibility of using LFP as an effective modality for long-term BMI decoding.

Dissociable use-dependent processes for volitional goal-directed reaching

Repetition of specific movement biases subsequent actions towards the practiced movement, a phenomenon known as use-dependent learning (UDL). Recent experiments that impose strict constraints on planning time have revealed two sources of use-dependent biases, one arising from dynamic changes occurring during motor planning and another reflecting a stable shift in motor execution. Here, we used a distributional analysis to examine the contribution of these biases in reaching. To create the conditions for UDL, the target appeared at a designated 'frequent' location on most trials, and at one of six 'rare' locations on other trials. Strikingly, the heading angles were bimodally distributed, with peaks at both frequent and rare target locations. Despite having no constraints on planning time, participants exhibited a robust bias towards the frequent target when movements were self-initiated quickly, the signature of a planning bias; notably, the peak near the rare target was shifted in the frequently practiced direction, the signature of an execution bias. Furthermore, these execution biases were not only replicated in a delayed-response task but were also insensitive to reward. Taken together, these results extend our understanding of how volitional movements are influenced by recent experience.

Are reaching and grasping effector-independent? Similarities and differences in reaching and grasping kinematics between the hand and foot

While reaching and grasping are highly prevalent manual actions, neuroimaging studies provide evidence that their neural representations may be shared between different body parts, i.e., effectors. If these actions are guided by effector-independent mechanisms, similar kinematics should be observed when the action is performed by the hand or by a cortically remote and less experienced effector, such as the foot. We tested this hypothesis with two characteristic components of action: the initial ballistic stage of reaching, and the preshaping of the digits during grasping based on object size. We examined if these kinematic features reflect effector-independent mechanisms by asking participants to reach toward and to grasp objects of different widths with their hand and foot. First, during both reaching and grasping, the velocity profile up to peak velocity matched between the hand and the foot, indicating a shared ballistic acceleration phase. Second, maximum grip aperture and time of maximum grip aperture of grasping increased with object size for both effectors, indicating encoding of object size during transport. Differences between the hand and foot were found in the deceleration phase and time of maximum grip aperture, likely due to biomechanical differences and the participants’ inexperience with foot actions. These findings provide evidence for effector-independent visuomotor mechanisms of reaching and grasping that generalize across body parts.