Decoding complete reach and grasp actions from local primary motor cortex populations

Hand position fields of neurons in the premotor cortex of macaques during natural reaching

While hippocampus represents spatial information through place cells for body navigation, whether motor areas employ a similar framework to guide hand reaching remains unknown. Here, we investigate tuning properties in dorsal premotor cortex (PMd) during naturalistic reach-and-grasp tasks in four monkeys. We find that 22% (132/601) of PMd neurons increase firing rates when the monkey's hand occupies specific positions in space, forming the position fields. These cells represent the hand position highly efficiently, achieving ~80% accuracy for decoding hand trajectories with only 50 most dedicated position tuned cells ( ~ 10% of all recorded neurons). The hand position is co-represented with hand moving direction, speed, and reward location in the same population of PMd neurons, forming a mixed-selective framework to integrate positional and kinematic information. Our findings suggest field-like positional coding may be a mechanism shared across brain regions for spatial representation in goal-directed movements, including body navigation and forelimb reaching.

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

A deep learning-based approach for unbiased kinematic analysis in CNS injury

Traumatic spinal cord injury (SCI) is a devastating condition that impacts over 300,000 individuals in the US alone. Depending on the severity of the injury, SCI can lead to varying degrees of sensorimotor deficits and paralysis. Despite advances in our understanding of the underlying pathological mechanisms of SCI and the identification of promising molecular targets for repair and functional restoration, few therapies have made it into clinical use. To improve the success rate of clinical translation, more robust, sensitive, and reproducible means of functional assessment are required. The gold standards for the evaluation of locomotion in rodents with SCI are the Basso Beattie Bresnahan (BBB) scale and Basso Mouse Scale (BMS). To overcome the shortcomings of current methods, we developed two separate markerless kinematic analysis paradigms in mice, MotorBox and MotoRater, based on deep-learning algorithms generated with the DeepLabCut open-source toolbox. The MotorBox system uses an originally designed, custom-made chamber, and the MotoRater system was implemented on a commercially available MotoRater device. We validated the MotorBox and MotoRater systems by comparing them with the traditional BMS test and extracted metrics of movement and gait that can provide an accurate and sensitive representation of mouse locomotor function post-injury, while eliminating investigator bias and variability. The integration of MotorBox and/or MotoRater assessments with BMS scoring will provide a much wider range of information on specific aspects of locomotion, ensuring the accuracy, rigor, and reproducibility of behavioral outcomes after SCI.

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Understanding the dynamical transformation of neural activity to behavior requires new capabilities to nonlinearly model, dissociate and prioritize behaviorally relevant neural dynamics and test hypotheses about the origin of nonlinearity. We present dissociative prioritized analysis of dynamics (DPAD), a nonlinear dynamical modeling approach that enables these capabilities with a multisection neural network architecture and training approach. Analyzing cortical spiking and local field potential activity across four movement tasks, we demonstrate five use-cases. DPAD enabled more accurate neural-behavioral prediction. It identified nonlinear dynamical transformations of local field potentials that were more behavior predictive than traditional power features. Further, DPAD achieved behavior-predictive nonlinear neural dimensionality reduction. It enabled hypothesis testing regarding nonlinearities in neural-behavioral transformation, revealing that, in our datasets, nonlinearities could largely be isolated to the mapping from latent cortical dynamics to behavior. Finally, DPAD extended across continuous, intermittently sampled and categorical behaviors. DPAD provides a powerful tool for nonlinear dynamical modeling and investigation of neural-behavioral data.

Event detection and classification from multimodal time series with application to neural data

The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

Objective.Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales.Approach.Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior.Main results.We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior.Significance.Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.

Low frequency independent components: Internal neuromarkers linking cortical LFPs to behavior

Local field potentials (LFPs) in the primate motor cortex have been shown to reflect information related to volitional movements. However, LFPs are composite signals that receive contributions from multiple neural sources, producing a complex mix of component signals. Using a blind source separation approach, we examined the components of neural activity recorded using multielectrode arrays in motor areas of macaque monkeys during a grasping and lifting task. We found a set of independent components in the low-frequency LFP with high temporal and spatial consistency associated with each task stage. We observed that ICs often arise from electrodes distributed across multiple cortical areas and provide complementary information to external behavioral markers, specifically in task stage detection and trial alignment. Taken together, our results show that it is possible to separate useful independent components of the LFP associated with specific task-related events, potentially representing internal markers of transition between cortical network states.

Motor actions are spatially organized in motor and dorsal premotor cortex

Frontal motor areas are central to controlling voluntary movements. In non-human primates, the motor areas contain independent, somatotopic, representations of the forelimb (i.e., motor maps). But are the neural codes for actions spatially organized within those forelimb representations? Addressing this question would provide insight into the poorly understood structure-function relationships of the cortical motor system. Here, we tackle the problem using high-resolution optical imaging and motor mapping in motor (M1) and dorsal premotor (PMd) cortex. Two macaque monkeys performed an instructed reach-to-grasp task while cortical activity was recorded with intrinsic signal optical imaging (ISOI). The spatial extent of activity in M1 and PMd was then quantified in relation to the forelimb motor maps, which we obtained from the same hemisphere with intracortical microstimulation. ISOI showed that task-related activity was concentrated in patches that collectively overlapped <40% of the M1 and PMd forelimb representations. The spatial organization of the patches was consistent across task conditions despite small variations in forelimb use. Nevertheless, the largest condition differences in forelimb use were reflected in the magnitude of cortical activity. Distinct time course profiles from patches in arm zones and patches in hand zones suggest functional differences within the forelimb representations. The results collectively support an organizational framework wherein the forelimb representations contain subzones enriched with neurons tuned for specific actions. Thus, the often-overlooked spatial dimension of neural activity appears to be an important organizing feature of the neural code in frontal motor areas.

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Deep learning-based markerless tracking has revolutionized studies of animal behavior. Yet the generalizability of trained models tends to be limited, as new training data typically needs to be generated manually for each setup or visual environment. With each model trained from scratch, researchers track distinct landmarks and analyze the resulting kinematic data in idiosyncratic ways. Moreover, due to inherent limitations in manual annotation, only a sparse set of landmarks are typically labeled. To address these issues, we developed an approach, which we term GlowTrack, for generating orders of magnitude more training data, enabling models that generalize across experimental contexts. We describe: a) a high-throughput approach for producing hidden labels using fluorescent markers; b) a multi-camera, multi-light setup for simulating diverse visual conditions; and c) a technique for labeling many landmarks in parallel, enabling dense tracking. These advances lay a foundation for standardized behavioral pipelines and more complete scrutiny of movement.

Neural Decoding for Intracortical Brain-Computer Interfaces

Brain-computer interfaces have revolutionized the field of neuroscience by providing a solution for paralyzed patients to control external devices and improve the quality of daily life. To accurately and stably control effectors, it is important for decoders to recognize an individual's motor intention from neural activity either by noninvasive or intracortical neural recording. Intracortical recording is an invasive way of measuring neural electrical activity with high temporal and spatial resolution. Herein, we review recent developments in neural signal decoding methods for intracortical brain-computer interfaces. These methods have achieved good performance in analyzing neural activity and controlling robots and prostheses in nonhuman primates and humans. For more complex paradigms in motor rehabilitation or other clinical applications, there remains more space for further improvements of decoders.

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales. Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical subspace identification method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and spike-LFP population activity recorded during a naturalistic reach and grasp behavior. We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower computational cost while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity. Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest.

Robust cortical encoding of 3D tongue shape during feeding in macaques

Dexterous tongue deformation underlies eating, drinking, and speaking. The orofacial sensorimotor cortex has been implicated in the control of coordinated tongue kinematics, but little is known about how the brain encodes-and ultimately drives-the tongue's 3D, soft-body deformation. Here we combine a biplanar x-ray video technology, multi-electrode cortical recordings, and machine-learning-based decoding to explore the cortical representation of lingual deformation. We trained long short-term memory (LSTM) neural networks to decode various aspects of intraoral tongue deformation from cortical activity during feeding in male Rhesus monkeys. We show that both lingual movements and complex lingual shapes across a range of feeding behaviors could be decoded with high accuracy, and that the distribution of deformation-related information across cortical regions was consistent with previous studies of the arm and hand.

Learnable latent embeddings for joint behavioural and neural analysis

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1-3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3-5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Non-human primate models and systems for gait and neurophysiological analysis

Brain-computer interfaces (BCIs) have garnered extensive interest and become a groundbreaking technology to restore movement, tactile sense, and communication in patients. Prior to their use in human subjects, clinical BCIs require rigorous validation and verification (V&V). Non-human primates (NHPs) are often considered the ultimate and widely used animal model for neuroscience studies, including BCIs V&V, due to their proximity to humans. This literature review summarizes 94 NHP gait analysis studies until 1 June, 2022, including seven BCI-oriented studies. Due to technological limitations, most of these studies used wired neural recordings to access electrophysiological data. However, wireless neural recording systems for NHPs enabled neuroscience research in humans, and many on NHP locomotion, while posing numerous technical challenges, such as signal quality, data throughout, working distance, size, and power constraint, that have yet to be overcome. Besides neurological data, motion capture (MoCap) systems are usually required in BCI and gait studies to capture locomotion kinematics. However, current studies have exclusively relied on image processing-based MoCap systems, which have insufficient accuracy (error: ≥4° and 9 mm). While the role of the motor cortex during locomotion is still unclear and worth further exploration, future BCI and gait studies require simultaneous, high-speed, accurate neurophysiological, and movement measures. Therefore, the infrared MoCap system which has high accuracy and speed, together with a high spatiotemporal resolution neural recording system, may expand the scope and improve the quality of the motor and neurophysiological analysis in NHPs.

Direct Discriminative Decoder Models for Analysis of High-Dimensional Dynamical Neural Data

With the accelerated development of neural recording technology over the past few decades, research in integrative neuroscience has become increasingly reliant on data analysis methods that are scalable to high-dimensional recordings and computationally tractable. Latent process models have shown promising results in estimating the dynamics of cognitive processes using individual models for each neuron's receptive field. However, scaling these models to work on high-dimensional neural recordings remains challenging. Not only is it impractical to build receptive field models for individual neurons of a large neural population, but most neural data analyses based on individual receptive field models discard the local history of neural activity, which has been shown to be critical in the accurate inference of the underlying cognitive processes. Here, we propose a novel, scalable latent process model that can directly estimate cognitive process dynamics without requiring precise receptive field models of individual neurons or brain nodes. We call this the direct discriminative decoder (DDD) model. The DDD model consists of (1) a discriminative process that characterizes the conditional distribution of the signal to be estimated, or state, as a function of both the current neural activity and its local history, and (2) a state transition model that characterizes the evolution of the state over a longer time period. While this modeling framework inherits advantages of existing latent process modeling methods, its computational cost is tractable. More important, the solution can incorporate any information from the history of neural activity at any timescale in computing the estimate of the state process. There are many choices in building the discriminative process, including deep neural networks or gaussian processes, which adds to the flexibility of the framework. We argue that these attributes of the proposed methodology, along with its applicability to different modalities of neural data, make it a powerful tool for high-dimensional neural data analysis. We also introduce an extension of these methods, called the discriminative-generative decoder (DGD). The DGD includes both discriminative and generative processes in characterizing observed data. As a result, we can combine physiological correlates like behavior with neural data to better estimate underlying cognitive processes. We illustrate the methods, including steps for inference and model identification, and demonstrate applications to multiple data analysis problems with high-dimensional neural recordings. The modeling results demonstrate the computational and modeling advantages of the DDD and DGD methods.

Motor cortical output for skilled forelimb movement is selectively distributed across projection neuron classes

The interaction of descending neocortical outputs and subcortical premotor circuits is critical for shaping skilled movements. Two broad classes of motor cortical output projection neurons provide input to many subcortical motor areas: pyramidal tract (PT) neurons, which project throughout the neuraxis, and intratelencephalic (IT) neurons, which project within the cortex and subcortical striatum. It is unclear whether these classes are functionally in series or whether each class carries distinct components of descending motor control signals. Here, we combine large-scale neural recordings across all layers of motor cortex with cell type-specific perturbations to study cortically dependent mouse motor behaviors: kinematically variable manipulation of a joystick and a kinematically precise reach-to-grasp. We find that striatum-projecting IT neuron activity preferentially represents amplitude, whereas pons-projecting PT neurons preferentially represent the variable direction of forelimb movements. Thus, separable components of descending motor cortical commands are distributed across motor cortical projection cell classes.

Functional cortical localization of tongue movements using corticokinematic coherence with a deep learning-assisted motion capture system

Corticokinematic coherence (CKC) between magnetoencephalographic and movement signals using an accelerometer is useful for the functional localization of the primary sensorimotor cortex (SM1). However, it is difficult to determine the tongue CKC because an accelerometer yields excessive magnetic artifacts. Here, we introduce a novel approach for measuring the tongue CKC using a deep learning-assisted motion capture system with videography, and compare it with an accelerometer in a control task measuring finger movement. Twelve healthy volunteers performed rhythmical side-to-side tongue movements in the whole-head magnetoencephalographic system, which were simultaneously recorded using a video camera and examined using a deep learning-assisted motion capture system. In the control task, right finger CKC measurements were simultaneously evaluated via motion capture and an accelerometer. The right finger CKC with motion capture was significant at the movement frequency peaks or its harmonics over the contralateral hemisphere; the motion-captured CKC was 84.9% similar to that with the accelerometer. The tongue CKC was significant at the movement frequency peaks or its harmonics over both hemispheres. The CKC sources of the tongue were considerably lateral and inferior to those of the finger. Thus, the CKC with deep learning-assisted motion capture can evaluate the functional localization of the tongue SM1.

Exploring Cognition with Brain-Machine Interfaces

Traditional brain-machine interfaces decode cortical motor commands to control external devices. These commands are the product of higher-level cognitive processes, occurring across a network of brain areas, that integrate sensory information, plan upcoming motor actions, and monitor ongoing movements. We review cognitive signals recently discovered in the human posterior parietal cortex during neuroprosthetic clinical trials. These signals are consistent with small regions of cortex having a diverse role in cognitive aspects of movement control and body monitoring, including sensorimotor integration, planning, trajectory representation, somatosensation, action semantics, learning, and decision making. These variables are encoded within the same population of cells using structured representations that bind related sensory and motor variables, an architecture termed partially mixed selectivity. Diverse cognitive signals provide complementary information to traditional motor commands to enable more natural and intuitive control of external devices.

Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface

Modern brain-machine interfaces can return function to people with paralysis, but current upper extremity brain-machine interfaces are unable to reproduce control of individuated finger movements. Here, for the first time, we present a real-time, high-speed, linear brain-machine interface in nonhuman primates that utilizes intracortical neural signals to bridge this gap. We created a non-prehensile task that systematically individuates two finger groups, the index finger and the middle-ring-small fingers combined. During online brain control, the ReFIT Kalman filter could predict individuated finger group movements with high performance. Next, training ridge regression decoders with individual movements was sufficient to predict untrained combined movements and vice versa. Finally, we compared the postural and movement tuning of finger-related cortical activity to find that individual cortical units simultaneously encode multiple behavioral dimensions. Our results suggest that linear decoders may be sufficient for brain-machine interfaces to execute high-dimensional tasks with the performance levels required for naturalistic neural prostheses.

Measuring and modeling the motor system with machine learning

The utility of machine learning in understanding the motor system is promising a revolution in how to collect, measure, and analyze data. The field of movement science already elegantly incorporates theory and engineering principles to guide experimental work, and in this review we discuss the growing use of machine learning: from pose estimation, kinematic analyses, dimensionality reduction, and closed-loop feedback, to its use in understanding neural correlates and untangling sensorimotor systems. We also give our perspective on new avenues, where markerless motion capture combined with biomechanical modeling and neural networks could be a new platform for hypothesis-driven research.

A Turntable Setup for Testing Visual and Tactile Grasping Movements in Non-human Primates

Grasping movements are some of the most common movements primates do every day. They are important for social interactions as well as picking up objects or food. Usually, these grasping movements are guided by vision but proprioceptive and haptic inputs contribute greatly. Since grasping behaviors are common and easy to motivate, they represent an ideal task for understanding the role of different brain areas during planning and execution of complex voluntary movements in primates. For experimental purposes, a stable and repeatable presentation of the same object as well as the variation of objects is important in order to understand the neural control of movement generation. This is even more the case when investigating the role of different senses for movement planning, where objects need to be presented in specific sensory modalities. We developed a turntable setup for non-human primates (macaque monkeys) to investigate visually and tactually guided grasping movements with an option to easily exchange objects. The setup consists of a turntable that can fit six different objects and can be exchanged easily during the experiment to increase the number of presented objects. The object turntable is connected to a stepper motor through a belt system to automate rotation and hence object presentation. By increasing the distance between the turntable and the stepper motor, metallic components of the stepper motor are kept at a distance to the actual recording setup, which allows using a magnetic-based data glove to track hand kinematics. During task execution, the animal sits in the dark and is instructed to grasp the object in front of it. Options to turn on a light above the object allow for visual presentation of the objects, while the object can also remain in the dark for exclusive tactile exploration. A red LED is projected onto the object by a one-way mirror that serves as a grasp cue instruction for the animal to start grasping the object. By comparing kinematic data from the magnetic-based data glove with simultaneously recorded neural signals, this setup enables the systematic investigation of neural population activity involved in the neural control of hand grasping movements.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Multiscale low-dimensional motor cortical state dynamics predict naturalistic reach-and-grasp behavior

Motor function depends on neural dynamics spanning multiple spatiotemporal scales of population activity, from spiking of neurons to larger-scale local field potentials (LFP). How multiple scales of low-dimensional population dynamics are related in control of movements remains unknown. Multiscale neural dynamics are especially important to study in naturalistic reach-and-grasp movements, which are relatively under-explored. We learn novel multiscale dynamical models for spike-LFP network activity in monkeys performing naturalistic reach-and-grasps. We show low-dimensional dynamics of spiking and LFP activity exhibited several principal modes, each with a unique decay-frequency characteristic. One principal mode dominantly predicted movements. Despite distinct principal modes existing at the two scales, this predictive mode was multiscale and shared between scales, and was shared across sessions and monkeys, yet did not simply replicate behavioral modes. Further, this multiscale mode's decay-frequency explained behavior. We propose that multiscale, low-dimensional motor cortical state dynamics reflect the neural control of naturalistic reach-and-grasp behaviors.

MacaquePose: A Novel "In the Wild" Macaque Monkey Pose Dataset for Markerless Motion Capture

Video-based markerless motion capture permits quantification of an animal's pose and motion, with a high spatiotemporal resolution in a naturalistic context, and is a powerful tool for analyzing the relationship between the animal's behaviors and its brain functions. Macaque monkeys are excellent non-human primate models, especially for studying neuroscience. Due to the lack of a dataset allowing training of a deep neural network for the macaque's markerless motion capture in the naturalistic context, it has been challenging to apply this technology for macaques-based studies. In this study, we created MacaquePose, a novel open dataset with manually labeled body part positions (keypoints) for macaques in naturalistic scenes, consisting of >13,000 images. We also validated the application of the dataset by training and evaluating an artificial neural network with the dataset. The results indicated that the keypoint estimation performance of the trained network was close to that of a human-level. The dataset will be instrumental to train/test the neural networks for markerless motion capture of the macaques and developments of the algorithms for the networks, contributing establishment of an innovative platform for behavior analysis for non-human primates for neuroscience and medicine, as well as other fields using macaques as a model organism.

From Neurons to Cognition: Technologies for Precise Recording of Neural Activity Underlying Behavior

Understanding how brain activity encodes information and controls behavior is a long-standing question in neuroscience. This complex problem requires converging efforts from neuroscience and engineering, including technological solutions to perform high-precision and large-scale recordings of neuronal activity in vivo as well as unbiased methods to reliably measure and quantify behavior. Thanks to advances in genetics, molecular biology, engineering, and neuroscience, in recent decades, a variety of optical imaging and electrophysiological approaches for recording neuronal activity in awake animals have been developed and widely applied in the field. Moreover, sophisticated computer vision and machine learning algorithms have been developed to analyze animal behavior. In this review, we provide an overview of the current state of technology for neuronal recordings with a focus on optical and electrophysiological methods in rodents. In addition, we discuss areas that future technological development will need to cover in order to further our understanding of the neural activity underlying behavior.

Reconstruction of natural images from responses of primate retinal ganglion cells

The visual message conveyed by a retinal ganglion cell (RGC) is often summarized by its spatial receptive field, but in principle also depends on the responses of other RGCs and natural image statistics. This possibility was explored by linear reconstruction of natural images from responses of the four numerically-dominant macaque RGC types. Reconstructions were highly consistent across retinas. The optimal reconstruction filter for each RGC - its visual message - reflected natural image statistics, and resembled the receptive field only when nearby, same-type cells were included. ON and OFF cells conveyed largely independent, complementary representations, and parasol and midget cells conveyed distinct features. Correlated activity and nonlinearities had statistically significant but minor effects on reconstruction. Simulated reconstructions, using linear-nonlinear cascade models of RGC light responses that incorporated measured spatial properties and nonlinearities, produced similar results. Spatiotemporal reconstructions exhibited similar spatial properties, suggesting that the results are relevant for natural vision.

Decoding Kinematic Information From Primary Motor Cortex Ensemble Activities Using a Deep Canonical Correlation Analysis

The control of arm movements through intracortical brain–machine interfaces (BMIs) mainly relies on the activities of the primary motor cortex (M1) neurons and mathematical models that decode their activities. Recent research on decoding process attempts to not only improve the performance but also simultaneously understand neural and behavioral relationships. In this study, we propose an efficient decoding algorithm using a deep canonical correlation analysis (DCCA), which maximizes correlations between canonical variables with the non-linear approximation of mappings from neuronal to canonical variables via deep learning. We investigate the effectiveness of using DCCA for finding a relationship between M1 activities and kinematic information when non-human primates performed a reaching task with one arm. Then, we examine whether using neural activity representations from DCCA improves the decoding performance through linear and non-linear decoders: a linear Kalman filter (LKF) and a long short-term memory in recurrent neural networks (LSTM-RNN). We found that neural representations of M1 activities estimated by DCCA resulted in more accurate decoding of velocity than those estimated by linear canonical correlation analysis, principal component analysis, factor analysis, and linear dynamical system. Decoding with DCCA yielded better performance than decoding the original FRs using LSTM-RNN (6.6 and 16.0% improvement on average for each velocity and position, respectively; Wilcoxon rank sum test, p < 0.05). Thus, DCCA can identify the kinematics-related canonical variables of M1 activities, thus improving the decoding performance. Our results may help advance the design of decoding models for intracortical BMIs.

Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review

Brain-machine interfaces (BMIs) record and translate neural activity into a control signal for assistive or other devices. Intracortical microelectrode arrays (MEAs) enable high degree-of-freedom BMI control for complex tasks by providing fine-resolution neural recording. However, chronically implanted MEAs are subject to a dynamic in vivo environment where transient or systematic disruptions can interfere with neural recording and degrade BMI performance. Typically, neural implant failure modes have been categorized as biological, material, or mechanical. While this categorization provides insight into a disruption's causal etiology, it is less helpful for understanding degree of impact on BMI function or possible strategies for compensation. Therefore, we propose a complementary classification framework for intracortical recording disruptions that is based on duration of impact on BMI performance and requirement for and responsiveness to interventions: (1) Transient disruptions interfere with recordings on the time scale of minutes to hours and can resolve spontaneously; (2) Reversible disruptions cause persistent interference in recordings but the root cause can be remedied by an appropriate intervention; (3) Irreversible compensable disruptions cause persistent or progressive decline in signal quality, but their effects on BMI performance can be mitigated algorithmically; and (4) Irreversible non-compensable disruptions cause permanent signal loss that is not amenable to remediation or compensation. This conceptualization of intracortical BMI disruption types is useful for highlighting specific areas for potential hardware improvements and also identifying opportunities for algorithmic interventions. We review recording disruptions that have been reported for MEAs and demonstrate how biological, material, and mechanical mechanisms of disruption can be further categorized according to their impact on signal characteristics. Then we discuss potential compensatory protocols for each of the proposed disruption classes. Specifically, transient disruptions may be minimized by using robust neural decoder features, data augmentation methods, adaptive machine learning models, and specialized signal referencing techniques. Statistical Process Control methods can identify reparable disruptions for rapid intervention. In-vivo diagnostics such as impedance spectroscopy can inform neural feature selection and decoding models to compensate for irreversible disruptions. Additional compensatory strategies for irreversible disruptions include information salvage techniques, data augmentation during decoder training, and adaptive decoding methods to down-weight damaged channels.

A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces

The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.

Chaotic behaviour of EEG responses with an identical grasp posture

Individuals with severe neuromuscular ailments can benefit from restoring their grasp activities with a brain-controlled upper-limb neuroprosthesis. EEG signals can be utilized as the driving source, and to implement natural human-like grasping abilities. Although good accuracy has already been achieved in classifying the various grasp patterns for specific sets of objects, unseen objects are still a hurdle in real-life implementation. Generalization of grasp patterns should be explored without any prior knowledge of the objects. In this regard, the similarity of motor imagery for different objects requiring similar grasp pattern can be utilized. It is also necessary to identify the brain regions that exhibit prominent distinguishability during different grasp patterns. In this study, we propose a chaos-based method to decode the motor imagery of two quite similar Power grasp patterns-cylindrical and spherical-for holding various objects. Three distinct suitable objects were chosen for each of the two patterns, and a 29-channel EEG was taken of 18 healthy participants to explore motor imagery for grasping the objects. Nonlinear correlation dimension was employed on the EEG data, at sub-band levels α, upper β, and γ, to analyse the distinguishability, as well as the similarity of grasp patterns for the objects. ANOVA was subsequently performed on the obtained CD parameters to identify the contribution of each electrode channel. Furthermore, using an SVM classifier, more than 80% accuracy was obtained in classifying the grasping patterns at the upper β sub-band. The outcome may lead to identification of optimum feature sets of motor imagery from specific brain regions for random objects grasps.

The Discriminative Kalman Filter for Bayesian Filtering with Nonlinear and Nongaussian Observation Models

The Kalman filter provides a simple and efficient algorithm to compute the posterior distribution for state-space models where both the latent state and measurement models are linear and gaussian. Extensions to the Kalman filter, including the extended and unscented Kalman filters, incorporate linearizations for models where the observation model p ( observation | state ) is nonlinear. We argue that in many cases, a model for p ( state | observation ) proves both easier to learn and more accurate for latent state estimation. Approximating p ( state | observation ) as gaussian leads to a new filtering algorithm, the discriminative Kalman filter (DKF), which can perform well even when p ( observation | state ) is highly nonlinear and/or nongaussian. The approximation, motivated by the Bernstein-von Mises theorem, improves as the dimensionality of the observations increases. The DKF has computational complexity similar to the Kalman filter, allowing it in some cases to perform much faster than particle filters with similar precision, while better accounting for nonlinear and nongaussian observation models than Kalman-based extensions. When the observation model must be learned from training data prior to filtering, off-the-shelf nonlinear and nonparametric regression techniques can provide a gaussian model for p ( observation | state ) that cleanly integrates with the DKF. As part of the BrainGate2 clinical trial, we successfully implemented gaussian process regression with the DKF framework in a brain-computer interface to provide real-time, closed-loop cursor control to a person with a complete spinal cord injury. In this letter, we explore the theory underlying the DKF, exhibit some illustrative examples, and outline potential extensions.

Neural Representation of Observed, Imagined, and Attempted Grasping Force in Motor Cortex of Individuals with Chronic Tetraplegia

Hybrid kinetic and kinematic intracortical brain-computer interfaces (iBCIs) have the potential to restore functional grasping and object interaction capabilities in individuals with tetraplegia. This requires an understanding of how kinetic information is represented in neural activity, and how this representation is affected by non-motor parameters such as volitional state (VoS), namely, whether one observes, imagines, or attempts an action. To this end, this work investigates how motor cortical neural activity changes when three human participants with tetraplegia observe, imagine, and attempt to produce three discrete hand grasping forces with the dominant hand. We show that force representation follows the same VoS-related trends as previously shown for directional arm movements; namely, that attempted force production recruits more neural activity compared to observed or imagined force production. Additionally, VoS-modulated neural activity to a greater extent than grasping force. Neural representation of forces was lower than expected, possibly due to compromised somatosensory pathways in individuals with tetraplegia, which have been shown to influence motor cortical activity. Nevertheless, attempted forces (but not always observed or imagined forces) could be decoded significantly above chance, thereby potentially providing relevant information towards the development of a hybrid kinetic and kinematic iBCI.

Ipsilateral primary motor cortex and behavioral compensation after stroke: a case series study

Arm motor recovery after stroke is mainly attributed to reorganization of the primary motor cortex (M1). While M1 contralateral to the paretic arm (cM1) is critical for recovery, the role of ipsilateral M1 (iM1) is still inconclusive. Whether iM1 activity is related to recovery, behavioral compensation, or both is still far from settled. We hypothesized that the magnitude of iM1 activity in chronic stroke survivors will increase or decrease in direct proportion to the degree that movements of the paretic arm are compensated. Movement kinematics (VICON, Oxford Metrics) and functional MRI data (3T MR system) were collected in 11 patients before and after a 4-week training designed to improve motor control of the paretic arm and decrease compensatory trunk recruitment. Twelve matched controls underwent similar evaluations and training. Relationships between iM1 activity and trunk motion were analyzed. At baseline, patients exhibited increased iM1 activity (p = 0.001) and relied more on trunk movement (p = 0.02) than controls. These two variables were directly and significantly related in patients (r = 0.74, p = 0.01) but not in controls (r = 0.28, p = 0.4). After training, patients displayed a significant reduction in iM1 activity (p = 0.008) and a trend toward decreased trunk use (p = 0.1). The relationship between these two variables remained significant (r = 0.66, p = 0.03) and different from controls (r = 0.26, p = 0.4). Our preliminary results suggest that iM1 may play a role in compensating for brain damage rather than directly gaining control of the paretic arm. However, we recommend caution in interpreting these results until more work is completed.

Decoding speech from spike-based neural population recordings in secondary auditory cortex of non-human primates

Direct electronic communication with sensory areas of the neocortex is a challenging ambition for brain-computer interfaces. Here, we report the first successful neural decoding of English words with high intelligibility from intracortical spike-based neural population activity recorded from the secondary auditory cortex of macaques. We acquired 96-channel full-broadband population recordings using intracortical microelectrode arrays in the rostral and caudal parabelt regions of the superior temporal gyrus (STG). We leveraged a new neural processing toolkit to investigate the choice of decoding algorithm, neural preprocessing, audio representation, channel count, and array location on neural decoding performance. The presented spike-based machine learning neural decoding approach may further be useful in informing future encoding strategies to deliver direct auditory percepts to the brain as specific patterns of microstimulation.

A versatile robotic platform for the design of natural, three-dimensional reaching and grasping tasks in monkeys

OBJECTIVE

Translational studies on motor control and neurological disorders require detailed monitoring of sensorimotor components of natural limb movements in relevant animal models. However, available experimental tools do not provide a sufficiently rich repertoire of behavioral signals. Here, we developed a robotic platform that enables the monitoring of kinematics, interaction forces, and neurophysiological signals during user-defined upper limb tasks for monkeys.

APPROACH

We configured the platform to position instrumented objects in a three-dimensional workspace and provide an interactive dynamic force-field.

MAIN RESULTS

We show the relevance of our platform for fundamental and translational studies with three example applications. First, we study the kinematics of natural grasp in response to variable interaction forces. We then show simultaneous and independent encoding of kinematic and forces in single unit intra-cortical recordings from sensorimotor cortical areas. Lastly, we demonstrate the relevance of our platform to develop clinically relevant brain computer interfaces in a kinematically unconstrained motor task.

SIGNIFICANCE

Our versatile control structure does not depend on the specific robotic arm used and allows for the design and implementation of a variety of tasks that can support both fundamental and translational studies of motor control.

Altered Recruitment of Motor Cortex Neuronal Activity During the Grasping Phase of Skilled Reaching in a Chronic Rat Model of Unilateral Parkinsonism

Parkinson's disease causes prominent difficulties in the generation and execution of voluntary limb movements, including regulation of distal muscles and coordination of proximal and distal movement components to achieve accurate grasping. Difficulties with manual dexterity have a major impact on activities of daily living. We used extracellular single neuron recordings to investigate the neural underpinnings of parkinsonian movement deficits in the motor cortex of chronic unilateral 6-hydroxydopamine lesion male rats performing a skilled reach-to-grasp task the. Both normal movements and parkinsonian deficits in this task have striking homology to human performance. In lesioned animals there were several differences in the activity of cortical neurons during reaches by the affected limb compared with control rats. These included an increase in proportions of neurons showing rate decreases, along with increased amplitude of their average rate-decrease response at specific times during the reach, suggesting a shift in the balance of net excitation and inhibition of cortical neurons; a significant increase in the duration of rate-increase responses, which could result from reduced coupling of cortical activity to specific movement components; and changes in the timing and incidence of neurons with pure rate-increase or biphasic responses, particularly at the end of reach when grasping would normally be occurring. The changes in cortical activity may account for the deficits that occur in skilled distal motor control following dopamine depletion, and highlight the need for treatment strategies targeted toward modulating cortical mechanisms for fine distal motor control in patients.SIGNIFICANCE STATEMENT We show for the first time in a chronic lesion rat model of Parkinson's disease movement deficits that there are specific changes in motor cortex neuron activity associated with the grasping phase of a skilled motor task. Such changes provide a possible mechanism underpinning the problems with manual dexterity seen in Parkinson's patients and highlight the need for treatment strategies targeted toward distal motor control.

Context-dependent limb movement encoding in neuronal populations of motor cortex

Neuronal networks of the mammalian motor cortex (M1) are important for dexterous control of limb joints. Yet it remains unclear how encoding of joint movement in M1 depends on varying environmental contexts. Using calcium imaging we measured neuronal activity in layer 2/3 of the M1 forelimb region while mice grasped regularly or irregularly spaced ladder rungs during locomotion. We found that population coding of forelimb joint movements is sparse and varies according to the flexibility demanded from individual joints in the regular and irregular context, even for equivalent grasping actions across conditions. This context-dependence of M1 encoding emerged during task learning, fostering higher precision of grasping actions, but broke apart upon silencing of projections from secondary motor cortex (M2). These findings suggest that M1 exploits information from M2 to adapt encoding of joint movements to the flexibility demands of distinct familiar contexts, thereby increasing the accuracy of motor output.

Network activity in primary motor cortex (M1) controls dexterous limb movements. Here, the authors show that the M1 population code varies according to contextual motor demands that are conveyed via the secondary motor cortex (M2).

Utilizing High-Density Electroencephalography and Motion Capture Technology to Characterize Sensorimotor Integration While Performing Complex Actions

Studies of sensorimotor integration often use sensory stimuli that require a simple motor response, such as a reach or a grasp. Recent advances in neural recording techniques, motion capture technologies, and time-synchronization methods enable studying sensorimotor integration using more complex sensory stimuli and performed actions. Here, we demonstrate that prehensile actions that require using complex sensory instructions for manipulating different objects can be characterized using high-density electroencephalography and motion capture systems. In 20 participants, we presented stimuli in different sensory modalities (visual, auditory) containing different contextual information about the object with which to interact. Neural signals recorded near motor cortex and posterior parietal cortex discharged based on both the instruction delivered and object manipulated. Additionally, kinematics of the wrist movements could be discriminated between participants. These findings demonstrate a proof-of-concept behavioral paradigm for studying sensorimotor integration of multidimensional sensory stimuli to perform complex movements. The designed framework will prove vital for studying neural control of movements in clinical populations in which sensorimotor integration is impaired due to information no longer being communicated correctly between brain regions (e.g. stroke). Such a framework is the first step towards developing a neural rehabilitative system for restoring function more effectively.

odMLtables: A User-Friendly Approach for Managing Metadata of Neurophysiological Experiments

An essential aspect of scientific reproducibility is a coherent and complete acquisition of metadata along with the actual data of an experiment. The high degree of complexity and heterogeneity of neuroscience experiments requires a rigorous management of the associated metadata. The odML framework represents a solution to organize and store complex metadata digitally in a hierarchical format that is both human and machine readable. However, this hierarchical representation of metadata is difficult to handle when metadata entries need to be collected and edited manually during the daily routines of a laboratory. With odMLtables, we present an open-source software solution that enables users to collect, manipulate, visualize, and store metadata in tabular representations (in xls or csv format) by providing functionality to convert these tabular collections to the hierarchically structured metadata format odML, and to either extract or merge subsets of a complex metadata collection. With this, odMLtables bridges the gap between handling metadata in an intuitive way that integrates well with daily lab routines and commonly used software products on the one hand, and the implementation of a complete, well-defined metadata collection for the experiment in a standardized format on the other hand. We demonstrate usage scenarios of the odMLtables tools in common lab routines in the context of metadata acquisition and management, and show how the tool can assist in exploring published datasets that provide metadata in the odML format.

Prediction of Forelimb EMGs and Movement Phases from Corticospinal Signals in the Rat During the Reach-to-Pull Task

Brain-computer interfaces access the volitional command signals from various brain areas in order to substitute for the motor functions lost due to spinal cord injury or disease. As the final common pathway of the central nervous system (CNS) outputs, the descending tracts of the spinal cord offer an alternative site to extract movement-related command signals. Using flexible 2D microelectrode arrays, we have recorded the corticospinal tract (CST) signals in rats during a reach-to-pull task. The CST activity was then classified by the forelimb movement phases into two or three classes in a training dataset and cross validated in a test set. The average classification accuracies were [Formula: see text] (min: [Formula: see text] to max: [Formula: see text]) and [Formula: see text] (min: 43% to max: 71%) for two-class and three-class cases, respectively. The forelimb flexor and extensor EMG envelopes were also predicted from the CST signals using linear regression. The average correlation coefficient between the actual and predicted EMG signals was [Formula: see text] [Formula: see text], whereas the highest correlation was 0.81 for the biceps EMG. Although the forelimb motor function cannot be explained completely by the CST activity alone, the success rates obtained in reconstructing the EMG signals support the feasibility of a spinal-cord-computer interface as a concept.

Decoding Grasping Movements from the Parieto-Frontal Reaching Circuit in the Nonhuman Primate

Prehension movements typically include a reaching phase, guiding the hand toward the object, and a grip phase, shaping the hand around it. The dominant view posits that these components rely upon largely independent parieto-frontal circuits: a dorso-medial circuit involved in reaching and a dorso-lateral circuit involved in grasping. However, mounting evidence suggests a more complex arrangement, with dorso-medial areas contributing to both reaching and grasping. To investigate the role of the dorso-medial reaching circuit in grasping, we trained monkeys to reach-and-grasp different objects in the dark and determined if hand configurations could be decoded from functional magnetic resonance imaging (MRI) responses obtained from the reaching and grasping circuits. Indicative of their established role in grasping, object-specific grasp decoding was found in anterior intraparietal (AIP) area, inferior parietal lobule area PFG and ventral premotor region F5 of the lateral grasping circuit, and primary motor cortex. Importantly, the medial reaching circuit also conveyed robust grasp-specific information, as evidenced by significant decoding in parietal reach regions (particular V6A) and dorsal premotor region F2. These data support the proposed role of dorso-medial "reach" regions in controlling aspects of grasping and demonstrate the value of complementing univariate with more sensitive multivariate analyses of functional MRI (fMRI) data in uncovering information coding in the brain.

Brain-Machine Interfaces: Powerful Tools for Clinical Treatment and Neuroscientific Investigations

Brain-machine interfaces (BMIs) have exploded in popularity in the past decade. BMIs, also called brain-computer interfaces, provide a direct link between the brain and a computer, usually to control an external device. BMIs have a wide array of potential clinical applications, ranging from restoring communication to people unable to speak due to amyotrophic lateral sclerosis or a stroke, to restoring movement to people with paralysis from spinal cord injury or motor neuron disease, to restoring memory to people with cognitive impairment. Because BMIs are controlled directly by the activity of prespecified neurons or cortical areas, they also provide a powerful paradigm with which to investigate fundamental questions about brain physiology, including neuronal behavior, learning, and the role of oscillations. This article reviews the clinical and neuroscientific applications of BMIs, with a primary focus on motor BMIs.

Fear conditioning and extinction induce opposing changes in dendritic spine remodeling and somatic activity of layer 5 pyramidal neurons in the mouse motor cortex

Multiple brain regions including the amygdala and prefrontal cortex are crucial for modulating fear conditioning and extinction. The primary motor cortex is known to participate in the planning, control, and execution of voluntary movements. Whether and how the primary motor cortex is involved in modulating freezing responses related to fear conditioning and extinction remains unclear. Here we show that inactivation of the mouse primary motor cortex impairs both the acquisition and extinction of freezing responses induced by auditory-cued fear conditioning. Fear conditioning significantly increases the elimination of dendritic spines on apical dendrites of layer 5 pyramidal neurons in the motor cortex. These eliminated spines are further apart from each other than expected from random distribution along dendrites. On the other hand, fear extinction causes the formation of new spines that are located near the site of spines eliminated previously after fear conditioning. We further show that fear conditioning decreases and fear extinction increases somatic activities of layer 5 pyramidal neurons in the motor cortex respectively. Taken together, these findings indicate fear conditioning and extinction induce opposing changes in synaptic connections and somatic activities of layer 5 pyramidal neurons in the primary motor cortex, a cortical region important for the acquisition and extinction of auditory-cued conditioned freezing responses.

Decoding Native Cortical Representations for Flexion and Extension at Upper Limb Joints Using Electrocorticography

Brain-machine interface (BMI) researchers have traditionally focused on modeling endpoint reaching tasks to provide the control of neurally driven prosthetic arms. Most previous research has focused on achieving an endpoint control through a Cartesian-coordinate-centered approach. However, a joint-centered approach could potentially be used to intuitively control a wide range of limb movements. We systematically investigated the feasibility of discriminating between flexion and extension of different upper limb joints using electrocorticography(ECoG) recordings from sensorimotor cortex. Four subjects implanted with macro-ECoG (10-mm spacing), high-density ECoG (5-mm spacing), and/or micro-ECoG arrays (0.9-mm spacing and 4 mm × 4 mm coverage), performed randomly cued flexions or extensions of the fingers, wrist, or elbow contralateral to the implanted hemisphere. We trained a linear model to classify six movements using averaged high-gamma power (70-110 Hz) modulations at different latencies with respect to movement onset, and within a time interval restricted to flexion or extension at each joint. Offline decoding models for each subject classified these movements with accuracies of 62%-83%. Our results suggest that the widespread ECoG coverage of sensorimotor cortex could allow a whole limb BMI to sample native cortical representations in order to control flexion and extension at multiple joints.

Closed-loop cortical control of virtual reach and posture using Cartesian and joint velocity commands

OBJECTIVE

Brain-computer interfaces (BCIs) are a promising technology for the restoration of function to people with paralysis, especially for controlling coordinated reaching. Typical BCI studies decode Cartesian endpoint velocities as commands, but human arm movements might be better controlled in a joint-based coordinate frame, which may match underlying movement encoding in the motor cortex. A better understanding of BCI controlled reaching by people with paralysis may lead to performance improvements in brain-controlled assistive devices.

APPROACH

Two intracortical BCI participants in the BrainGate2 pilot clinical trial performed a visual 3D endpoint virtual reality reaching task using two decoders: Cartesian and joint velocity. Task performance metrics (i.e. success rate and path efficiency) and single feature and population tuning were compared across the two decoder conditions. The participants also demonstrated the first BCI control of a fourth dimension of reaching, the arm's swivel angle, in a 4D posture matching task.

MAIN RESULTS

Both users achieved significantly higher success rates using Cartesian velocity control, and joint controlled trajectories were more variable and significantly more curved. Neural tuning analyses showed that most single feature activity was best described by a Cartesian kinematic encoding model, and population analyses revealed only slight differences in aggregate activity between the decoder conditions. Simulations of a BCI user reproduced trajectory features seen during closed-loop joint control when assuming only Cartesian-tuned features passed through a joint decoder. With minimal training, both participants controlled the virtual arm's swivel angle to complete a 4D posture matching task, and achieved significantly higher success using a Cartesian  +  swivel velocity decoder compared to a joint velocity decoder.

SIGNIFICANCE

These results suggest that Cartesian velocity command interfaces may provide better BCI control of arm movements than other kinematic variables, even in 4D posture tasks with swivel angle targets.