Stabilization of a brain-computer interface via the alignment of low-dimensional spaces of neural activity

Deep transfer learning-based decoder calibration for intracortical brain-machine interfaces

Intracortical brain-machine interfaces (iBMIs) aim to establish a communication path between the brain and external devices. However, in the daily use of iBMIs, the non-stationarity of recorded neural signals necessitates frequent recalibration of the iBMI decoder to maintain decoding performance, which requires collecting and labeling a large amount of new data. To address this challenge and minimize the time needed for decoder recalibration, we proposed an active learning domain adversarial neural network (AL-DANN). This model leveraged a substantial volume of historical data alongside a small amount of current data (four samples per category) to calibrate the decoder. By incorporating domain adversarial and active learning strategies, the model effectively transferred knowledge from historical data to new data, reducing the demand for new samples. We validated the proposed method using neural signals recorded from three monkeys performing different movements in a classification task or a regression task. The results showed that the AL-DANN outperformed existing state-of-the-art methods. Impressively, it required only four new samples per category for decoder recalibration, leading to a recalibration time reduction of over 80 %. To our knowledge, this is the first study to incorporate deep transfer learning into iBMI decoder calibration, highlighting the significant potential of applying deep learning technologies in iBMIs.

Stabilizing brain-computer interfaces through alignment of latent dynamics

Intracortical brain-computer interfaces (iBCIs) restore motor function to people with paralysis by translating brain activity into control signals for external devices. In current iBCIs, instabilities at the neural interface result in a degradation of decoding performance, which necessitates frequent supervised recalibration using new labeled data. One potential solution is to use the latent manifold structure that underlies neural population activity to facilitate a stable mapping between brain activity and behavior. Recent efforts using unsupervised approaches have improved iBCI stability using this principle; however, existing methods treat each time step as an independent sample and do not account for latent dynamics. Dynamics have been used to enable high-performance prediction of movement intention, and may also help improve stabilization. Here, we present a platform for Nonlinear Manifold Alignment with Dynamics (NoMAD), which stabilizes decoding using recurrent neural network models of dynamics. NoMAD uses unsupervised distribution alignment to update the mapping of nonstationary neural data to a consistent set of neural dynamics, thereby providing stable input to the decoder. In applications to data from monkey motor cortex collected during motor tasks, NoMAD enables accurate behavioral decoding with unparalleled stability over weeks- to months-long timescales without any supervised recalibration.

Restoring Speech Using Brain-Computer Interfaces

People who have lost the ability to speak due to neurological injuries would greatly benefit from assistive technology that provides a fast, intuitive, and naturalistic means of communication. This need can be met with brain-computer interfaces (BCIs): medical devices that bypass injured parts of the nervous system and directly transform neural activity into outputs such as text or sound. BCIs for restoring movement and typing have progressed rapidly in recent clinical trials; speech BCIs are the next frontier. This review covers the clinical need for speech BCIs, surveys foundational studies that point to where and how speech can be decoded in the brain, describes recent progress in both discrete and continuous speech decoding and closed-loop speech BCIs, provides metrics for assessing these systems' performance, and highlights key remaining challenges on the road toward clinically useful speech neuroprostheses.

Reward influences movement vigor through multiple motor cortical mechanisms

The prospect of greater rewards often invigorates movements. What neural mechanisms support this increase of movement vigor for greater rewards? We had three rhesus monkeys perform reaching movements to targets worth different magnitudes of reward. We recorded neural population activity from primary motor and dorsal premotor cortex, brain areas at the output of cortical processing for voluntary movements, and asked how neural activity mediated the translation of reward into increased vigor. We identified features of neural activity during movement preparation, initiation, and execution that were both correlated with vigor and modulated by reward. We also found that the neural metrics that correlate with different aspects of movement vigor exhibit only limited correlation with one another, suggesting that there are multiple mechanisms through which reward modulates vigor. Finally, we note that the majority of reward's modulation of motor cortical activity cannot be accounted for by reward-mediated vigor differences in behavior, indicating that reward modulations within motor cortex may serve roles in addition to affecting vigor. Overall, our results provide insight into the neural mechanisms that link reward-driven motivation to the modulation of the details of movement.

Assistive sensory-motor perturbations influence learned neural representations

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Over time, task-relevant information became concentrated in fewer neurons, unlike with fixed decoders. At the population level, task information also became largely confined to a few neural modes that accounted for an unexpectedly small fraction of the population variance. A neural network model suggests the adaptive decoders directly contribute to forming these more compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

Population Transformer: Learning Population-level Representations of Neural Activity

We present a self-supervised framework that learns population-level codes for arbitrary ensembles of neural recordings at scale. We address key challenges in scaling models with neural time-series data, namely, sparse and variable electrode distribution across subjects and datasets. The Population Transformer (PopT) stacks on top of pretrained temporal embeddings and enhances downstream decoding by enabling learned aggregation of multiple spatially-sparse data channels. The pretrained PopT lowers the amount of data required for downstream decoding experiments, while increasing accuracy, even on held-out subjects and tasks. Compared to end-to-end methods, this approach is computationally lightweight, while achieving similar or better decoding performance. We further show how our framework is generalizable to multiple time-series embeddings and neural data modalities. Beyond decoding, we interpret the pretrained and fine-tuned PopT models to show how they can be used to extract neuroscience insights from large amounts of data. We release our code as well as a pretrained PopT to enable off-the-shelf improvements in multi-channel intracranial data decoding and interpretability. Code is available at https://github.com/czlwang/PopulationTransformer.

TRACKING THE TOPOLOGY OF NEURAL MANIFOLDS ACROSS POPULATIONS

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds.

Less is more: selection from a small set of options improves BCI velocity control

Objective.Decoding algorithms used in invasive brain-computer interfaces (iBCIs) typically convert neural activity into continuously varying velocity commands. We hypothesized that putting constraints on which decoded velocity commands are permissible could improve user performance. To test this hypothesis, we designed the discrete direction selection (DDS) decoder, which uses neural activity to select among a small menu of preset cursor velocities.Approach. We tested DDS in a closed-loop cursor control task against many common continuous velocity decoders in both a human-operated real-time iBCI simulator (the jaBCI) and in a monkey using an iBCI. In the jaBCI, we compared performance across four visits by each of 48 naïve, able-bodied human subjects using either DDS, direct regression with assist (an affine map from neural activity to cursor velocity, DR-A), ReFIT, or the velocity Kalman Filter (vKF). In a follow up study to verify the jaBCI results, we compared a monkey's performance using an iBCI with either DDS or the Wiener filter decoder (a direct regression decoder that includes time history, WF).Main Result. In the jaBCI, DDS substantially outperformed all other decoders with 93% mean targets hit per visit compared to DR-A, ReFIT, and vKF with 56%, 39%, and 26% mean targets hit, respectively. With the iBCI, the monkey achieved a 61% success rate with DDS and a 37% success rate with WF.Significance. Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of discretization in simplifying online BCI control.

Sampling representational plasticity of simple imagined movements across days enables long-term neuroprosthetic control

The nervous system needs to balance the stability of neural representations with plasticity. It is unclear what the representational stability of simple well-rehearsed actions is, particularly in humans, and their adaptability to new contexts. Using an electrocorticography brain-computer interface (BCI) in tetraplegic participants, we found that the low-dimensional manifold and relative representational distances for a repertoire of simple imagined movements were remarkably stable. The manifold's absolute location, however, demonstrated constrained day-to-day drift. Strikingly, neural statistics, especially variance, could be flexibly regulated to increase representational distances during BCI control without somatotopic changes. Discernability strengthened with practice and was BCI-specific, demonstrating contextual specificity. Sampling representational plasticity and drift across days subsequently uncovered a meta-representational structure with generalizable decision boundaries for the repertoire; this allowed long-term neuroprosthetic control of a robotic arm and hand for reaching and grasping. Our study offers insights into mesoscale representational statistics that also enable long-term complex neuroprosthetic control.

Online Unsupervised Adaptation of Latent Representation for Myoelectric Control During User-Decoder Co-Adaptation

Myoelectric control interfaces, which map electromyographic (EMG) signals into control commands for external devices, have applications in active prosthesis control. However, the statistical characteristics of EMG signals change over time (e.g., because of changes in the electrode location), which makes interfaces based on static mapping unstable. Thus the user-decoder co-adaptation is needed during online operations. Nevertheless, current online decoder adaptation approaches present several practical challenges, such as expensive data labeling and slow convergence. Thus we introduce an unsupervised decoder adaptation method that converges rapidly. We use an autoencoder to extract motor intent representation in the latent manifold space rather than the sensor space, and further introduce an online unsupervised adaptation scheme based on Moore-Penrose Inverse, a noniterative approach suited for fast network re-training, to track the evolving manifold. A validation experiment first showed that the convergence time of the proposed adaptation scheme was reduced to about 50% of that for state-of-the-art methods. Online experiments further evaluated cursor and prosthetic hand control by the proposed myocontrol interface, where perturbations were representatively introduced by shifting the electrodes. Results showed that our scheme reached comparable improvements in robustness as supervised counterparts. Moreover, in a cup relocation test with a prosthetic hand, the completion time in the post-adaptation phase with electrode shift was comparable to that in the baseline phase without shift. These results suggest that our method effectively improves the accessibility and reliability of decoder adaptation, which has the potential to reduce the translational gap of myoelectric control interfaces by effective co-adaptation during operation.

AM-MTEEG: multi-task EEG classification based on impulsive associative memory

Electroencephalogram-based brain-computer interfaces (BCIs) hold promise for healthcare applications but are hindered by cross-subject variability and limited data. This article proposes a multi-task (MT) classification model, AM-MTEEG, which integrates deep learning-based convolutional and impulsive networks with bidirectional associative memory (AM) for cross-subject EEG classification. AM-MTEEG deals with the EEG classification of each subject as an independent task and utilizes common features across subjects. The model is built with a convolutional encoder-decoder and a population of impulsive neurons to extract shared features across subjects, as well as a Hebbian-learned bidirectional associative memory matrix to classify EEG within one subject. Experimental results on two BCI competition datasets demonstrate that AM-MTEEG improves average accuracy over state-of-the-art methods and reduces performance variance across subjects. Visualization of neuronal impulses in the bidirectional associative memory network reveal a precise mapping between hidden-layer neuron activities and specific movements. Given four motor imagery categories, the reconstructed waveforms resemble the real event-related potentials, highlighting the biological interpretability of the model beyond classification.

The mind-machine connection: adaptive information processing and new technologies promoting mental health in older adults

The human brain demonstrates an exceptional adaptability, which encompasses the ability to regulate emotions, exhibit cognitive flexibility, and generate behavioral responses, all supported by neuroplasticity. Brain-computer interfaces (BCIs) employ adaptive algorithms and machine learning techniques to adapt to variations in the user's brain activity, allowing for customized interactions with external devices. Older adults may experience cognitive decline, which could affect the ability to learn and adapt to new technologies such as BCIs, but both (human brain and BCI) demonstrate adaptability in their responses. The human brain is skilled at quickly switching between tasks and regulating emotions, while BCIs can modify signal-processing algorithms to accommodate changes in brain activity. Furthermore, the human brain and BCI participate in knowledge acquisition; the first one strengthens cognitive abilities through exposure to new experiences, and the second one improves performance through ongoing adjustment and improvement. Current research seeks to incorporate emotional states into BCI systems to improve the user experience, despite the exceptional emotional regulation abilities of the human brain. The implementation of BCIs for older adults could be more effective, inclusive, and beneficial in improving their quality of life. This review aims to improve the understanding of brain-machine interfaces and their implications for mental health in older adults.

How different immersive environments affect intracortical brain computer interfaces

Objective. As brain-computer interface (BCI) research advances, many new applications are being developed. Tasks can be performed in different virtual environments, and whether a BCI user can switch environments seamlessly will influence the ultimate utility of a clinical device.Approach. Here we investigate the importance of the immersiveness of the virtual environment used to train BCI decoders on the resulting decoder and its generalizability between environments. Two participants who had intracortical electrodes implanted in their precentral gyrus used a BCI to control a virtual arm, both viewed immersively through virtual reality goggles and at a distance on a flat television monitor.Main results. Each participant performed better with a decoder trained and tested in the environment they had used the most prior to the study, one for each environment type. The neural tuning to the desired movement was minimally influenced by the immersiveness of the environment. Finally, in further testing with one of the participants, we found that decoders trained in one environment generalized well to the other environment, but the order in which the environments were experienced within a session mattered.Significance. Overall, experience with an environment was more influential on performance than the immersiveness of the environment, but BCI performance generalized well after accounting for experience.Clinical Trial: NCT01894802.

Exploring Synergies in Brain-Machine Interfaces: Compression vs. Performance

Individuals with severe neurological injuries often rely on assistive technologies, but current methods have limitations in accurately decoding multi-degree-of-freedom (DoF) movements. Intracortical brain-machine interfaces (iBMIs) use neural signals to provide a more natural control method, but currently struggle with higher-DoF movements-something the brain handles effortlessly. It has been theorized that the brain simplifies high-DoF movement through muscle synergies, which link multiple muscles to function as a single unit. These synergies have been studied using dimensionality reduction techniques like principal component analysis (PCA), non-negative matrix factorization (NMF), and demixed PCA (dPCA) and successfully used to reduce noise and improve offline decoder stability in non-invasive applications. However, their effectiveness in improving decoding and generalizability for implanted recordings across varied tasks is unclear. Here, we evaluated if brain and muscle synergies can enhance iBMI performance in non-human primates performing a two-DoF finger task. Specifically, we tested if PCA, dPCA, and NMF could compress and denoise brain and muscle data and improve decoder generalization across tasks. Our results showed that while all methods effectively compressed data with minimal loss in decoding accuracy, none improved performance through denoising. Additionally, none of the methods enhanced generalization across tasks. These findings suggest that while dimensionality reduction can aid data compression, alone it may not reveal the "true" control space needed to improve decoder performance or generalizability. Further research is required to determine whether synergies are the optimal control framework or if alternative approaches are required to enhance decoder robustness in iBMI applications.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective.Creating an intracortical brain computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.Approach.We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.Main results.We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network decoder with 10-12 clusters.Significance.This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

A high-performance brain-computer interface for finger decoding and quadcopter game control in an individual with paralysis

People with paralysis express unmet needs for peer support, leisure activities and sporting activities. Many within the general population rely on social media and massively multiplayer video games to address these needs. We developed a high-performance, finger-based brain-computer-interface system allowing continuous control of three independent finger groups, of which the thumb can be controlled in two dimensions, yielding a total of four degrees of freedom. The system was tested in a human research participant with tetraplegia due to spinal cord injury over sequential trials requiring fingers to reach and hold on targets, with an average acquisition rate of 76 targets per minute and completion time of 1.58 ± 0.06 seconds-comparing favorably to prior animal studies despite a twofold increase in the decoded degrees of freedom. More importantly, finger positions were then used to control a virtual quadcopter-the number-one restorative priority for the participant-using a brain-to-finger-to-computer interface to allow dexterous navigation around fixed- and random-ringed obstacle courses. The participant expressed or demonstrated a sense of enablement, recreation and social connectedness that addresses many of the unmet needs of people with paralysis.

Enhanced control of a brain-computer interface by tetraplegic participants via neural-network-mediated feature extraction

To infer intent, brain-computer interfaces must extract features that accurately estimate neural activity. However, the degradation of signal quality over time hinders the use of feature-engineering techniques to recover functional information. By using neural data recorded from electrode arrays implanted in the cortices of three human participants, here we show that a convolutional neural network can be used to map electrical signals to neural features by jointly optimizing feature extraction and decoding under the constraint that all the electrodes must use the same neural-network parameters. In all three participants, the neural network led to offline and online performance improvements in a cursor-control task across all metrics, outperforming the rate of threshold crossings and wavelet decomposition of the broadband neural data (among other feature-extraction techniques). We also show that the trained neural network can be used without modification for new datasets, brain areas and participants.

Inter-participant transfer learning with attention based domain adversarial training for P300 detection

A Brain-computer interface (BCI) system establishes a novel communication channel between the human brain and a computer. Most event related potential-based BCI applications make use of decoding models, which requires training. This training process is often time-consuming and inconvenient for new users. In recent years, deep learning models, especially participant-independent models, have garnered significant attention in the domain of ERP classification. However, individual differences in EEG signals hamper model generalization, as the ERP component and other aspects of the EEG signal vary across participants, even when they are exposed to the same stimuli. This paper proposes a novel One-source domain transfer learning method based Attention Domain Adversarial Neural Network (OADANN) to mitigate data distribution discrepancies for cross-participant classification tasks. We train and validate our proposed model on both a publicly available OpenBMI dataset and a Self-collected dataset, employing a leave one participant out cross validation scheme. Experimental results demonstrate that the proposed OADANN method achieves the highest and most robust classification performance and exhibits significant improvements when compared to baseline methods (CNN, EEGNet, ShallowNet, DeepCovNet) and domain generalization methods (ERM, Mixup, and Groupdro). These findings underscore the efficacy of our proposed method.

Aligned and oblique dynamics in recurrent neural networks

The relation between neural activity and behaviorally relevant variables is at the heart of neuroscience research. When strong, this relation is termed a neural representation. There is increasing evidence, however, for partial dissociations between activity in an area and relevant external variables. While many explanations have been proposed, a theoretical framework for the relationship between external and internal variables is lacking. Here, we utilize recurrent neural networks (RNNs) to explore the question of when and how neural dynamics and the network's output are related from a geometrical point of view. We find that training RNNs can lead to two dynamical regimes: dynamics can either be aligned with the directions that generate output variables, or oblique to them. We show that the choice of readout weight magnitude before training can serve as a control knob between the regimes, similar to recent findings in feedforward networks. These regimes are functionally distinct. Oblique networks are more heterogeneous and suppress noise in their output directions. They are furthermore more robust to perturbations along the output directions. Crucially, the oblique regime is specific to recurrent (but not feedforward) networks, arising from dynamical stability considerations. Finally, we show that tendencies toward the aligned or the oblique regime can be dissociated in neural recordings. Altogether, our results open a new perspective for interpreting neural activity by relating network dynamics and their output.

A reinforcement learning based software simulator for motor brain-computer interfaces

Intracortical motor brain-computer interfaces (BCIs) are expensive and time-consuming to design because accurate evaluation traditionally requires real-time experiments. In a BCI system, a user interacts with an imperfect decoder and continuously changes motor commands in response to unexpected decoded movements. This "closed-loop" nature of BCI leads to emergent interactions between the user and decoder that are challenging to model. The gold standard for BCI evaluation is therefore real-time experiments, which significantly limits the speed and community of BCI research. We present a new BCI simulator that enables researchers to accurately and quickly design BCIs for cursor control entirely in software. Our simulator replaces the BCI user with a deep reinforcement learning (RL) agent that interacts with a simulated BCI system and learns to optimally control it. We demonstrate that our simulator is accurate and versatile, reproducing the published results of three distinct types of BCI decoders: (1) a state-of-the-art linear decoder (FIT-KF), (2) a "two-stage" BCI decoder requiring closed-loop decoder adaptation (ReFIT-KF), and (3) a nonlinear recurrent neural network decoder (FORCE).

Tracking the topology of neural manifolds across populations

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds.

Reducing power requirements for high-accuracy decoding in iBCIs

Objective.Current intracortical brain-computer interfaces (iBCIs) rely predominantly on threshold crossings ('spikes') for decoding neural activity into a control signal for an external device. Spiking data can yield high accuracy online control during complex behaviors; however, its dependence on high-sampling-rate data collection can pose challenges. An alternative signal for iBCI decoding is the local field potential (LFP), a continuous-valued signal that can be acquired simultaneously with spiking activity. However, LFPs are seldom used alone for online iBCI control as their decoding performance has yet to achieve parity with spikes.Approach.Here, we present a strategy to improve the performance of LFP-based decoders by first training a neural dynamics model to use LFPs to reconstruct the firing rates underlying spiking data, and then decoding from the estimated rates. We test these models on previously-collected macaque data during center-out and random-target reaching tasks as well as data collected from a human iBCI participant during attempted speech.Main results.In all cases, training models from LFPs enables firing rate reconstruction with accuracy comparable to spiking-based dynamics models. In addition, LFP-based dynamics models enable decoding performance exceeding that of LFPs alone and approaching that of spiking-based models. In all applications except speech, LFP-based dynamics models also facilitate decoding accuracy exceeding that of direct decoding from spikes.Significance.Because LFP-based dynamics models operate on lower bandwidth and with lower sampling rate than spiking models, our findings indicate that iBCI devices can be designed to operate with lower power requirements than devices dependent on recorded spiking activity, without sacrificing high-accuracy decoding.

Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark

Intracortical brain-computer interfaces (iBCIs) can restore movement and communication abilities to individuals with paralysis by decoding their intended behavior from neural activity recorded with an implanted device. While this activity yields high-performance decoding over short timescales, neural data are often nonstationary, which can lead to decoder failure if not accounted for. To maintain performance, users must frequently recalibrate decoders, which requires the arduous collection of new neural and behavioral data. Aiming to reduce this burden, several approaches have been developed that either limit recalibration data requirements (few-shot approaches) or eliminate explicit recalibration entirely (zero-shot approaches). However, progress is limited by a lack of standardized datasets and comparison metrics, causing methods to be compared in an ad hoc manner. Here we introduce the FALCON benchmark suite (Few-shot Algorithms for COnsistent Neural decoding) to standardize evaluation of iBCI robustness. FALCON curates five datasets of neural and behavioral data that span movement and communication tasks to focus on behaviors of interest to modern-day iBCIs. Each dataset includes calibration data, optional few-shot recalibration data, and private evaluation data. We implement a flexible evaluation platform which only requires user-submitted code to return behavioral predictions on unseen data. We also seed the benchmark by applying baseline methods spanning several classes of possible approaches. FALCON aims to provide rigorous selection criteria for robust iBCI decoders, easing their translation to real-world devices.

A neural basis of choking under pressure

Incentives tend to drive improvements in performance. But when incentives get too high, we can "choke under pressure" and underperform right when it matters most. What neural processes might lead to choking under pressure? We studied rhesus monkeys performing a challenging reaching task in which they underperformed when an unusually large "jackpot" reward was at stake, and we sought a neural mechanism that might result in that underperformance. We found that increases in reward drive neural activity during movement preparation into, and then past, a zone of optimal performance. We conclude that neural signals of reward and motor preparation interact in the motor cortex (MC) in a manner that can explain why we choke under pressure.

Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings

Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.

Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces

Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method, "MINDFUL", to measure instabilities in neural data for useful long-term iBCI, without needing labels of user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.

Approach

We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.

Main results

We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters.

Significance

This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Neural subspaces of imagined movements in parietal cortex remain stable over several years in humans

Objective.A crucial goal in brain-machine interfacing is the long-term stability of neural decoding performance, ideally without regular retraining. Long-term stability has only been previously demonstrated in non-human primate experiments and only in primary sensorimotor cortices. Here we extend previous methods to determine long-term stability in humans by identifying and aligning low-dimensional structures in neural data.Approach.Over a period of 1106 and 871 d respectively, two participants completed an imagined center-out reaching task. The longitudinal accuracy between all day pairs was assessed by latent subspace alignment using principal components analysis and canonical correlations analysis of multi-unit intracortical recordings in different brain regions (Brodmann Area 5, Anterior Intraparietal Area and the junction of the postcentral and intraparietal sulcus).Main results.We show the long-term stable representation of neural activity in subspaces of intracortical recordings from higher-order association areas in humans.Significance.These results can be practically applied to significantly expand the longevity and generalizability of brain-computer interfaces.Clinical TrialsNCT01849822, NCT01958086, NCT01964261.

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Complex, learned motor behaviors involve the coordination of large-scale neural activity across multiple brain regions, but our understanding of the population-level dynamics within different regions tied to the same behavior remains limited. Here, we investigate the neural population dynamics underlying learned vocal production in awake-singing songbirds. We use Neuropixels probes to record the simultaneous extracellular activity of populations of neurons in two regions of the vocal motor pathway. In line with observations made in non-human primates during limb-based motor tasks, we show that the population-level activity in both the premotor nucleus HVC and the motor nucleus RA is organized on low-dimensional neural manifolds upon which coordinated neural activity is well described by temporally structured trajectories during singing behavior. Both the HVC and RA latent trajectories provide relevant information to predict vocal sequence transitions between song syllables. However, the dynamics of these latent trajectories differ between regions. Our state-space models suggest a unique and continuous-over-time correspondence between the latent space of RA and vocal output, whereas the corresponding relationship for HVC exhibits a higher degree of neural variability. We then demonstrate that comparable high-fidelity reconstruction of continuous vocal outputs can be achieved from HVC and RA neural latents and spiking activity. Unlike those that use spiking activity, however, decoding models using neural latents generalize to novel sub-populations in each region, consistent with the existence of preserved manifolds that confine vocal-motor activity in HVC and RA.

The speech neuroprosthesis

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

Neuronal functional connectivity is impaired in a layer dependent manner near chronically implanted intracortical microelectrodes in C57BL6 wildtype mice

Objective.This study aims to reveal longitudinal changes in functional network connectivity within and across different brain structures near chronically implanted microelectrodes. While it is well established that the foreign-body response (FBR) contributes to the gradual decline of the signals recorded from brain implants over time, how the FBR affects the functional stability of neural circuits near implanted brain-computer interfaces (BCIs) remains unknown. This research aims to illuminate how the chronic FBR can alter local neural circuit function and the implications for BCI decoders.Approach.This study utilized single-shank, 16-channel,100µm site-spacing Michigan-style microelectrodes (3 mm length, 703µm2 site area) that span all cortical layers and the hippocampal CA1 region. Sex balanced C57BL6 wildtype mice (11-13 weeks old) received perpendicularly implanted microelectrode in left primary visual cortex. Electrophysiological recordings were performed during both spontaneous activity and visual sensory stimulation. Alterations in neuronal activity near the microelectrode were tested assessing cross-frequency synchronization of local field potential (LFP) and spike entrainment to LFP oscillatory activity throughout 16 weeks after microelectrode implantation.Main results. The study found that cortical layer 4, the input-receiving layer, maintained activity over the implantation time. However, layers 2/3 rapidly experienced severe impairment, leading to a loss of proper intralaminar connectivity in the downstream output layers 5/6. Furthermore, the impairment of interlaminar connectivity near the microelectrode was unidirectional, showing decreased connectivity from Layers 2/3 to Layers 5/6 but not the reverse direction. In the hippocampus, CA1 neurons gradually became unable to properly entrain to the surrounding LFP oscillations.Significance. This study provides a detailed characterization of network connectivity dysfunction over long-term microelectrode implantation periods. This new knowledge could contribute to the development of targeted therapeutic strategies aimed at improving the health of the tissue surrounding brain implants and potentially inform engineering of adaptive decoders as the FBR progresses. Our study's understanding of the dynamic changes in the functional network over time opens the door to developing interventions for improving the long-term stability and performance of intracortical microelectrodes.

Increasing Robustness of Intracortical Brain-Computer Interfaces for Recording Condition Changes via Data Augmentation

BACKGROUND AND OBJECTIVE

Intracortical brain-computer interfaces (iBCIs) aim to help paralyzed individuals restore their motor functions by decoding neural activity into intended movement. However, changes in neural recording conditions hinder the decoding performance of iBCIs, mainly because the neural-to-kinematic mappings shift. Conventional approaches involve either training the neural decoders using large datasets before deploying the iBCI or conducting frequent calibrations during its operation. However, collecting data for extended periods can cause user fatigue, negatively impacting the quality and consistency of neural signals. Furthermore, frequent calibration imposes a substantial computational load.

METHODS

This study proposes a novel approach to increase iBCIs' robustness against changing recording conditions. The approach uses three neural augmentation operators to generate augmented neural activity that mimics common recording conditions. Then, contrastive learning is used to learn latent factors by maximizing the similarity between the augmented neural activities. The learned factors are expected to remain stable despite varying recording conditions and maintain a consistent correlation with the intended movement.

RESULTS

Experimental results demonstrate that the proposed iBCI outperformed the state-of-the-art iBCIs and was robust to changing recording conditions across days for long-term use on one publicly available nonhuman primate dataset. It achieved satisfactory offline decoding performance, even when a large training dataset was unavailable.

CONCLUSIONS

This study paves the way for reducing the need for frequent calibration of iBCIs and collecting a large amount of annotated training data. Potential future works aim to improve offline decoding performance with an ultra-small training dataset and improve the iBCIs' robustness to severely disabled electrodes.

Identifying Distinct Neural Features between the Initial and Corrective Phases of Precise Reaching Using AutoLFADS

Many initial movements require subsequent corrective movements, but how the motor cortex transitions to make corrections and how similar the encoding is to initial movements is unclear. In our study, we explored how the brain's motor cortex signals both initial and corrective movements during a precision reaching task. We recorded a large population of neurons from two male rhesus macaques across multiple sessions to examine the neural firing rates during not only initial movements but also subsequent corrective movements. AutoLFADS, an autoencoder-based deep-learning model, was applied to provide a clearer picture of neurons' activity on individual corrective movements across sessions. Decoding of reach velocity generalized poorly from initial to corrective submovements. Unlike initial movements, it was challenging to predict the velocity of corrective movements using traditional linear methods in a single, global neural space. We identified several locations in the neural space where corrective submovements originated after the initial reaches, signifying firing rates different than the baseline before initial movements. To improve corrective movement decoding, we demonstrate that a state-dependent decoder incorporating the population firing rates at the initiation of correction improved performance, highlighting the diverse neural features of corrective movements. In summary, we show neural differences between initial and corrective submovements and how the neural activity encodes specific combinations of velocity and position. These findings are inconsistent with assumptions that neural correlations with kinematic features are global and independent, emphasizing that traditional methods often fall short in describing these diverse neural processes for online corrective movements.

A brain machine interface framework for exploring proactive control of smart environments

Brain machine interfaces (BMIs) can substantially improve the quality of life of elderly or disabled people. However, performing complex action sequences with a BMI system is onerous because it requires issuing commands sequentially. Fundamentally different from this, we have designed a BMI system that reads out mental planning activity and issues commands in a proactive manner. To demonstrate this, we recorded brain activity from freely-moving monkeys performing an instructed task and decoded it with an energy-efficient, small and mobile field-programmable gate array hardware decoder triggering real-time action execution on smart devices. Core of this is an adaptive decoding algorithm that can compensate for the day-by-day neuronal signal fluctuations with minimal re-calibration effort. We show that open-loop planning-ahead control is possible using signals from primary and pre-motor areas leading to significant time-gain in the execution of action sequences. This novel approach provides, thus, a stepping stone towards improved and more humane control of different smart environments with mobile brain machine interfaces.

Event detection and classification from multimodal time series with application to neural data

The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

Learning leaves a memory trace in motor cortex

How are we able to learn new behaviors without disrupting previously learned ones? To understand how the brain achieves this, we used a brain-computer interface (BCI) learning paradigm, which enables us to detect the presence of a memory of one behavior while performing another. We found that learning to use a new BCI map altered the neural activity that monkeys produced when they returned to using a familiar BCI map in a way that was specific to the learning experience. That is, learning left a "memory trace" in the primary motor cortex. This memory trace coexisted with proficient performance under the familiar map, primarily by altering neural activity in dimensions that did not impact behavior. Forming memory traces might be how the brain is able to provide for the joint learning of multiple behaviors without interference.

Motor cortex retains and reorients neural dynamics during motor imagery

The most prominent characteristic of motor cortex is its activation during movement execution, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioural and imaging studies, it is unknown how the specific activity patterns and temporal dynamics in motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people who retain some residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population activity into three orthogonal subspaces, where one was similarly active during both action and imagery, and the others were active only during a single task type-action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamic features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by reorienting the components related to motor output and/or feedback into a unique, output-null imagery subspace.

Robust Decoding of Upper-Limb Movement Direction Under Cognitive Distraction With Invariant Patterns in Embedding Manifold

Motor brain-computer interfaces (BCIs) have gained growing research interest in motor rehabilitation, restoration, and prostheses control. Decoding upper-limb movement direction with noninvasive BCIs has been extensively investigated. However, few of them address the intervention of cognitive distraction that impairs decoding performance in practice. In this study, we propose a novel decoding model with invariant patterns in embedding manifold on a mixed dataset pooled from electroencephalograph (EEG) signals under different attentional states. We reconstruct an embedding low-dimensional manifold that intrinsically characterizes movements of the upper limb and transfer patterns of neural activities decomposed from brain functional connectivity (FC) to the manifold subspace to further preserve movement-related information. Experimental results showed that the proposed decoding model had higher robustness on the mixed dataset of attentive and distracted states compared to the baseline method. Our research provides insights into modeling a uniform underlying mechanism of movement-related EEG signals and can help enhance the practicability of BCI systems under real-world situations.

Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces

Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method to measure instability in neural data without needing to label user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

Objective.Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales.Approach.Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior.Main results.We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior.Significance.Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.

Modeling and dissociation of intrinsic and input-driven neural population dynamics underlying behavior

Neural dynamics can reflect intrinsic dynamics or dynamic inputs, such as sensory inputs or inputs from other brain regions. To avoid misinterpreting temporally structured inputs as intrinsic dynamics, dynamical models of neural activity should account for measured inputs. However, incorporating measured inputs remains elusive in joint dynamical modeling of neural-behavioral data, which is important for studying neural computations of behavior. We first show how training dynamical models of neural activity while considering behavior but not input or input but not behavior may lead to misinterpretations. We then develop an analytical learning method for linear dynamical models that simultaneously accounts for neural activity, behavior, and measured inputs. The method provides the capability to prioritize the learning of intrinsic behaviorally relevant neural dynamics and dissociate them from both other intrinsic dynamics and measured input dynamics. In data from a simulated brain with fixed intrinsic dynamics that performs different tasks, the method correctly finds the same intrinsic dynamics regardless of the task while other methods can be influenced by the task. In neural datasets from three subjects performing two different motor tasks with task instruction sensory inputs, the method reveals low-dimensional intrinsic neural dynamics that are missed by other methods and are more predictive of behavior and/or neural activity. The method also uniquely finds that the intrinsic behaviorally relevant neural dynamics are largely similar across the different subjects and tasks, whereas the overall neural dynamics are not. These input-driven dynamical models of neural-behavioral data can uncover intrinsic dynamics that may otherwise be missed.

A real-time, high-performance brain-computer interface for finger decoding and quadcopter control

People with paralysis express unmet needs for peer support, leisure activities, and sporting activities. Many within the general population rely on social media and massively multiplayer video games to address these needs. We developed a high-performance finger brain-computer-interface system allowing continuous control of 3 independent finger groups with 2D thumb movements. The system was tested in a human research participant over sequential trials requiring fingers to reach and hold on targets, with an average acquisition rate of 76 targets/minute and completion time of 1.58 ± 0.06 seconds. Performance compared favorably to previous animal studies, despite a 2-fold increase in the decoded degrees-of-freedom (DOF). Finger positions were then used for 4-DOF velocity control of a virtual quadcopter, demonstrating functionality over both fixed and random obstacle courses. This approach shows promise for controlling multiple-DOF end-effectors, such as robotic fingers or digital interfaces for work, entertainment, and socialization.

Identifying distinct neural features between the initial and corrective phases of precise reaching using AutoLFADS

Many initial movements require subsequent corrective movements, but how motor cortex transitions to make corrections and how similar the encoding is to initial movements is unclear. In our study, we explored how the brain's motor cortex signals both initial and corrective movements during a precision reaching task. We recorded a large population of neurons from two male rhesus macaques across multiple sessions to examine the neural firing rates during not only initial movements but also subsequent corrective movements. AutoLFADS, an auto-encoder-based deep-learning model, was applied to provide a clearer picture of neurons' activity on individual corrective movements across sessions. Decoding of reach velocity generalized poorly from initial to corrective submovements. Unlike initial movements, it was challenging to predict the velocity of corrective movements using traditional linear methods in a single, global neural space. We identified several locations in the neural space where corrective submovements originated after the initial reaches, signifying firing rates different than the baseline before initial movements. To improve corrective movement decoding, we demonstrate that a state-dependent decoder incorporating the population firing rates at the initiation of correction improved performance, highlighting the diverse neural features of corrective movements. In summary, we show neural differences between initial and corrective submovements and how the neural activity encodes specific combinations of velocity and position. These findings are inconsistent with assumptions that neural correlations with kinematic features are global and independent, emphasizing that traditional methods often fall short in describing these diverse neural processes for online corrective movements.

Significance Statement

We analyzed submovement neural population dynamics during precision reaching. Using an auto- encoder-based deep-learning model, AutoLFADS, we examined neural activity on a single-trial basis. Our study shows distinct neural dynamics between initial and corrective submovements. We demonstrate the existence of unique neural features within each submovement class that encode complex combinations of position and reach direction. Our study also highlights the benefit of state-specific decoding strategies, which consider the neural firing rates at the onset of any given submovement, when decoding complex motor tasks such as corrective submovements.

Distinct manifold encoding of navigational information in the subiculum and hippocampus

The subiculum (SUB) plays a crucial role in spatial navigation and encodes navigational information differently from the hippocampal CA1 area. However, the representation of subicular population activity remains unknown. Here, we investigated the neuronal population activity recorded extracellularly from the CA1 and SUB of rats performing T-maze and open-field tasks. The trajectory of population activity in both areas was confined to low-dimensional neural manifolds homoeomorphic to external space. The manifolds conveyed position, speed, and future path information with higher decoding accuracy in the SUB than in the CA1. The manifolds exhibited common geometry across rats and regions for the CA1 and SUB and between tasks in the SUB. During post-task ripples in slow-wave sleep, population activity represented reward locations/events more frequently in the SUB than in CA1. Thus, the CA1 and SUB encode information distinctly into the neural manifolds that underlie navigational information processing during wakefulness and sleep.

Endovascular Brain-Computer Interfaces in Poststroke Paralysis

Stroke is a leading cause of paralysis, most frequently affecting the upper limbs and vocal folds. Despite recent advances in care, stroke recovery invariably reaches a plateau, after which there are permanent neurological impairments. Implantable brain-computer interface devices offer the potential to bypass permanent neurological lesions. They function by (1) recording neural activity, (2) decoding the neural signal occurring in response to volitional motor intentions, and (3) generating digital control signals that may be used to control external devices. While brain-computer interface technology has the potential to revolutionize neurological care, clinical translation has been limited. Endovascular arrays present a novel form of minimally invasive brain-computer interface devices that have been deployed in human subjects during early feasibility studies. This article provides an overview of endovascular brain-computer interface devices and critically evaluates the patient with stroke as an implant candidate. Future opportunities are mapped, along with the challenges arising when decoding neural activity following infarction. Limitations arise when considering intracerebral hemorrhage and motor cortex lesions; however, future directions are outlined that aim to address these challenges.

Decoding motor plans using a closed-loop ultrasonic brain-machine interface

Brain-machine interfaces (BMIs) enable people living with chronic paralysis to control computers, robots and more with nothing but thought. Existing BMIs have trade-offs across invasiveness, performance, spatial coverage and spatiotemporal resolution. Functional ultrasound (fUS) neuroimaging is an emerging technology that balances these attributes and may complement existing BMI recording technologies. In this study, we use fUS to demonstrate a successful implementation of a closed-loop ultrasonic BMI. We streamed fUS data from the posterior parietal cortex of two rhesus macaque monkeys while they performed eye and hand movements. After training, the monkeys controlled up to eight movement directions using the BMI. We also developed a method for pretraining the BMI using data from previous sessions. This enabled immediate control on subsequent days, even those that occurred months apart, without requiring extensive recalibration. These findings establish the feasibility of ultrasonic BMIs, paving the way for a new class of less-invasive (epidural) interfaces that generalize across extended time periods and promise to restore function to people with neurological impairments.

Unsupervised Neural Manifold Alignment for Stable Decoding of Movement from Cortical Signals

The stable decoding of movement parameters using neural activity is crucial for the success of brain-machine interfaces (BMIs). However, neural activity can be unstable over time, leading to changes in the parameters used for decoding movement, which can hinder accurate movement decoding. To tackle this issue, one approach is to transfer neural activity to a stable, low-dimensional manifold using dimensionality reduction techniques and align manifolds across sessions by maximizing correlations of the manifolds. However, the practical use of manifold stabilization techniques requires knowledge of the true subject intentions such as target direction or behavioral state. To overcome this limitation, an automatic unsupervised algorithm is proposed that determines movement target intention before manifold alignment in the presence of manifold rotation and scaling across sessions. This unsupervised algorithm is combined with a dimensionality reduction and alignment method to overcome decoder instabilities. The effectiveness of the BMI stabilizer method is represented by decoding the two-dimensional (2D) hand velocity of two rhesus macaque monkeys during a center-out-reaching movement task. The performance of the proposed method is evaluated using correlation coefficient and R-squared measures, demonstrating higher decoding performance compared to a state-of-the-art unsupervised BMI stabilizer. The results offer benefits for the automatic determination of movement intents in long-term BMI decoding. Overall, the proposed method offers a promising automatic solution for achieving stable and accurate movement decoding in BMI applications.

Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication

Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to use. In this paper, we propose a method that enables self-recalibration of communication iBCIs without interrupting the user. Our method leverages large language models (LMs) to automatically correct errors in iBCI outputs. The self-recalibration process uses these corrected outputs ("pseudo-labels") to continually update the iBCI decoder online. Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. CORP achieved a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. Notably, this is the longest-running iBCI stability demonstration involving a human participant. Our results provide the first evidence for long-term stabilization of a plug-and-play, high-performance communication iBCI, addressing a major barrier for the clinical translation of iBCIs.

Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months

Brain-computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dysarthria due to ALS operates computer applications with six intuitive speech commands via a chronic electrocorticographic (ECoG) implant over the ventral sensorimotor cortex. Speech commands are accurately detected and decoded (median accuracy: 90.59%) throughout a 3-month study period without model retraining or recalibration. Use of the BCI does not require exogenous timing cues, enabling the participant to issue self-paced commands at will. These results demonstrate that a chronically implanted ECoG-based speech BCI can reliably control assistive devices over long time periods with only initial model training and calibration, supporting the feasibility of unassisted home use.