Directional information from neuronal ensembles in the primate orofacial sensorimotor cortex

The speech neuroprosthesis

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

Biomechanical and Cortical Control of Tongue Movements During Chewing and Swallowing

Tongue function is vital for chewing and swallowing and lingual dysfunction is often associated with dysphagia. Better treatment of dysphagia depends on a better understanding of hyolingual morphology, biomechanics, and neural control in humans and animal models. Recent research has revealed significant variation among animal models in morphology of the hyoid chain and suprahyoid muscles which may be associated with variation in swallowing mechanisms. The recent deployment of XROMM (X-ray Reconstruction of Moving Morphology) to quantify 3D hyolingual kinematics has revealed new details on flexion and roll of the tongue during chewing in animal models, movements similar to those used by humans. XROMM-based studies of swallowing in macaques have falsified traditional hypotheses of mechanisms of tongue base retraction during swallowing, and literature review suggests that other animal models may employ a diversity of mechanisms of tongue base retraction. There is variation among animal models in distribution of hyolingual proprioceptors but how that might be related to lingual mechanics is unknown. In macaque monkeys, tongue kinematics-shape and movement-are strongly encoded in neural activity in orofacial primary motor cortex, giving optimism for development of brain-machine interfaces for assisting recovery of lingual function after stroke. However, more research on hyolingual biomechanics and control is needed for technologies interfacing the nervous system with the hyolingual apparatus to become a reality.

Primary somatosensory cortex organization for engineering artificial somatosensation

Somatosensory deficits from stroke, spinal cord injury, or other neurologic damage can lead to a significant degree of functional impairment. The primary (SI) and secondary (SII) somatosensory cortices encode information in a medial to lateral organization. SI is generally organized topographically, with more discrete cortical representations of specific body regions. SII regions corresponding to anatomical areas are less discrete and may represent a more functional rather than topographic organization. Human somatosensory research continues to map cortical areas of sensory processing with efforts primarily focused on hand and upper extremity information in SI. However, research into SII and other body regions is lacking. In this review, we synthesize the current state of knowledge regarding the cortical organization of human somatosensation and discuss potential applications for brain computer interface. In addition to accurate individualized mapping of cortical somatosensation, further research is required to uncover the neurophysiological mechanisms of how somatosensory information is encoded in the cortex.

Multiple regions of sensorimotor cortex encode bite force and gape

The precise control of bite force and gape is vital for safe and effective breakdown and manipulation of food inside the oral cavity during feeding. Yet, the role of the orofacial sensorimotor cortex (OSMcx) in the control of bite force and gape is still largely unknown. The aim of this study was to elucidate how individual neurons and populations of neurons in multiple regions of OSMcx differentially encode bite force and static gape when subjects (Macaca mulatta) generated different levels of bite force at varying gapes. We examined neuronal activity recorded simultaneously from three microelectrode arrays implanted chronically in the primary motor (MIo), primary somatosensory (SIo), and cortical masticatory (CMA) areas of OSMcx. We used generalized linear models to evaluate encoding properties of individual neurons and utilized dimensionality reduction techniques to decompose population activity into components related to specific task parameters. Individual neurons encoded bite force more strongly than gape in all three OSMCx areas although bite force was a better predictor of spiking activity in MIo vs. SIo. Population activity differentiated between levels of bite force and gape while preserving task-independent temporal modulation across the behavioral trial. While activation patterns of neuronal populations were comparable across OSMCx areas, the total variance explained by task parameters was context-dependent and differed across areas. These findings suggest that the cortical control of static gape during biting may rely on computations at the population level whereas the strong encoding of bite force at the individual neuron level allows for the precise and rapid control of bite force.

A high-performance neuroprosthesis for speech decoding and avatar control

Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive1. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.

Robust cortical encoding of 3D tongue shape during feeding in macaques

Dexterous tongue deformation underlies eating, drinking, and speaking. The orofacial sensorimotor cortex has been implicated in the control of coordinated tongue kinematics, but little is known about how the brain encodes-and ultimately drives-the tongue's 3D, soft-body deformation. Here we combine a biplanar x-ray video technology, multi-electrode cortical recordings, and machine-learning-based decoding to explore the cortical representation of lingual deformation. We trained long short-term memory (LSTM) neural networks to decode various aspects of intraoral tongue deformation from cortical activity during feeding in male Rhesus monkeys. We show that both lingual movements and complex lingual shapes across a range of feeding behaviors could be decoded with high accuracy, and that the distribution of deformation-related information across cortical regions was consistent with previous studies of the arm and hand.

Cerebral control of swallowing: An update on neurobehavioral evidence

This review aims to update the current knowledge on the cerebral control of swallowing. We review data from both animal and human studies spanning across the fields of neuroanatomy, neurophysiology and neuroimaging to evaluate advancements in our understanding in the brain's role in swallowing. Studies have collectively shown that swallowing is mediated by multiple distinct cortical and subcortical regions and that lesions to these regions can result in dysphagia. These regions are functionally connected in separate groups within and between the two hemispheres. While hemispheric dominance for swallowing has been reported in most human studies, the laterality is inconsistent across individuals. Moreover, there is a shift in activation location and laterality between swallowing preparation and execution, although such activation changes are less well-defined than that for limb motor control. Finally, we discussed recent neurostimulation treatments that may be beneficial for dysphagia after brain injury through promoting the reorganization of the swallowing neural network.

Beyond language: The unspoken sensory-motor representation of the tongue in non-primates, non-human and human primates

The English idiom "on the tip of my tongue" commonly acknowledges that something is known, but it cannot be immediately brought to mind. This phrase accurately describes sensorimotor functions of the tongue, which are fundamental for many tongue-related behaviors (e.g., speech), but often neglected by scientific research. Here, we review a wide range of studies conducted on non-primates, non-human and human primates with the aim of providing a comprehensive description of the cortical representation of the tongue's somatosensory inputs and motor outputs across different phylogenetic domains. First, we summarize how the properties of passive non-noxious mechanical stimuli are encoded in the putative somatosensory tongue area, which has a conserved location in the ventral portion of the somatosensory cortex across mammals. Second, we review how complex self-generated actions involving the tongue are represented in more anterior regions of the putative somato-motor tongue area. Finally, we describe multisensory response properties of the primate and non-primate tongue area by also defining how the cytoarchitecture of this area is affected by experience and deafferentation.

The Association between Age-Related Changes in Oral Neuromechanics and Alzheimer's Disease

The global population of 80 years and older is predicted to reach 437 million by 2050. As overall brain structure and function progressively degrades, older and younger adults show differences in sensorimotor performance and brain activity in the sensorimotor regions. Oral sensorimotor functions are an important area of focus in natural aging and Alzheimer’s Disease (AD) because oral health issues are commonly found in both elderly and AD populations. While human behavioral studies on changes in oral sensorimotor functions abound, very little is known about their neuronal correlates in normal and pathological aging.

Combination of jaw and tongue movement training influences neuroplasticity of corticomotor pathways in humans

Since humans in daily life perform multiple motor behaviors that often involve the simultaneous activation of both jaw and tongue muscles, it is essential to understand the effects of combined orofacial sensorimotor tasks on plasticity in corticomotor pathways. Moreover, to establish novel rehabilitation programs for patients, it is important to clarify the possible interrelationships in corticomotor excitability between jaw and tongue motor control. The aim of this study was to examine the effect of a combination of a repetitive tooth bite task (TBT) and a repetitive tongue lift task (TLT) on corticomotor excitability of the tongue and jaw muscles as assessed by transcranial magnetic stimulation (TMS). Sixteen healthy individuals participated in three kinds of training tasks consisting of 41-min TBT, 41-min TLT, and 82-min TBT + TLT. Motor-evoked potentials (MEPs) from the tongue muscle, masseter muscle, and first dorsal interosseous muscle were measured before and after the training tasks. The amplitude of tongue MEPs after training with TLT and TLT + TBT, and masseter MEPs after training with TBT and TLT + TBT, were significantly higher than before training (P < 0.05). Tongue MEPs and masseter MEPs were significantly higher after TLT + TBT than after TBT or TLT (P < 0.05). The present results suggest that a task combining both jaw and tongue movement training is associated with a greater degree of neuroplasticity in the corticomotor control of jaw and tongue muscles than either task alone.

Primary sensorimotor cortex exhibits complex dependencies of spike-field coherence on neuronal firing rates, field power, and behavior

Spike-field coherence (SFC) is widely used to assess cortico-cortical interactions during sensorimotor behavioral tasks by measuring the consistency of the relative phases between the spike train of a neuron and the concurrent local field potentials (LFPs). Interpretations of SFC as a measure of functional connectivity are complicated by theoretical work suggesting that estimates of SFC depend on overall neuronal activity. We evaluated the dependence of SFC on neuronal firing rates, LFP power, and behavior in the primary motor (MIo) and primary somatosensory (SIo) areas of the orofacial sensorimotor cortex of monkeys ( Macaca mulatta) during performance of a tongue-protrusion task. Although we occasionally observed monotonically increasing linear relationships between coherence and firing rate, we most often found highly complex, nonmonotonic relationships in both SIo and MIo and sometimes even found that coherence decreased with increasing firing rate. The lack of linear relationships was also true for both LFP power and tongue-protrusive force. Moreover, the ratio between maximal firing rate and the firing rate at peak coherence deviated significantly from unity, indicating that MIo and SIo neurons achieved maximal SFC at a submaximal level of spiking. Overall, these results point to complex relationships of SFC to firing rates, LFP power, and behavior during sensorimotor cortico-cortical interactions: coherence is a measure of functional connectivity whose magnitude is not a mere monotonic reflection of changes in firing rate, LFP power, or the relevantly controlled behavioral parameter. NEW & NOTEWORTHY The concern that estimates of spike-field coherence depend on the firing rates of single neurons has influenced analytical methods employed by experimental studies investigating the functional interactions between cortical areas. Our study shows that the overwhelming majority of the estimated spike-field coherence exhibited complex relations with firing rates of neurons in the orofacial sensorimotor cortex. The lack of monotonic relations was also evident after testing the influence of local field potential power and force on spike-field coherence.

Human Sensorimotor Cortex Control of Directly Measured Vocal Tract Movements during Vowel Production

During speech production, we make vocal tract movements with remarkable precision and speed. Our understanding of how the human brain achieves such proficient control is limited, in part due to the challenge of simultaneously acquiring high-resolution neural recordings and detailed vocal tract measurements. To overcome this challenge, we combined ultrasound and video monitoring of the supralaryngeal articulators (lips, jaw, and tongue) with electrocorticographic recordings from the cortical surface of 4 subjects (3 female, 1 male) to investigate how neural activity in the ventral sensory-motor cortex (vSMC) relates to measured articulator movement kinematics (position, speed, velocity, acceleration) during the production of English vowels. We found that high-gamma activity at many individual vSMC electrodes strongly encoded the kinematics of one or more articulators, but less so for vowel formants and vowel identity. Neural population decoding methods further revealed the structure of kinematic features that distinguish vowels. Encoding of articulator kinematics was sparsely distributed across time and primarily occurred during the time of vowel onset and offset. In contrast, encoding was low during the steady-state portion of the vowel, despite sustained neural activity at some electrodes. Significant representations were found for all kinematic parameters, but speed was the most robust. These findings enabled by direct vocal tract monitoring demonstrate novel insights into the representation of articulatory kinematic parameters encoded in the vSMC during speech production.SIGNIFICANCE STATEMENT Speaking requires precise control and coordination of the vocal tract articulators (lips, jaw, and tongue). Despite the impressive proficiency with which humans move these articulators during speech production, our understanding of how the brain achieves such control is rudimentary, in part because the movements themselves are difficult to observe. By simultaneously measuring speech movements and the neural activity that gives rise to them, we demonstrate how neural activity in sensorimotor cortex produces complex, coordinated movements of the vocal tract.

Animal Models for Dysphagia Studies: What Have We Learnt So Far

Research using animal models has contributed significantly to realizing the goal of understanding dysfunction and improving the care of patients who suffer from dysphagia. But why should other researchers and the clinicians who see patients day in and day out care about this work? Results from studies of animal models have the potential to change and grow how we think about dysphagia research and practice in general, well beyond applying specific results to human studies. Animal research provides two key contributions to our understanding of dysphagia. The first is a more complete characterization of the physiology of both normal and pathological swallow than is possible in human subjects. The second is suggesting of specific, physiological, targets for development and testing of treatment interventions to improve dysphagia outcomes.

Exploring the effects of synchronous pharyngeal electrical stimulation with swallowing carbonated water on cortical excitability in the human pharyngeal motor system

BACKGROUND

Previous reports have revealed that excitation of human pharyngeal motor cortex can be induced by pharyngeal electrical stimulation (PES) and swallowing carbonated water (CW). This study investigated whether combining PES with swallowing (of still water, SW or CW) can potentiate this excitation in either cortical and/or brain stem areas assessed with transcranial and transcutaneous magnetic stimulation (TMS).

METHODS

Fourteen healthy volunteers participated and were intubated with an intraluminal catheter to record pharyngeal electromyography and deliver PES. Each participant underwent baseline corticopharyngeal, hand and craniobulbar motor-evoked potential (MEP) measurements. Subjects were then randomized to receive each of four 10-min interventions (PES only, ShamPES+CW, PES+CW, and PES+SW). Corticobulbar, craniobulbar and hand MEPs were then remeasured for up to 60 min and data analyzed using anova and post hoc t-tests.

KEY RESULTS

A two-way rmanova for Interventions × Time-point showed a significant corticopharyngeal interaction (p = 0.010). One-way anova with post hoc t-tests indicated significant cortical changes with PES only at 45 (p = 0.038) and 60 min (p = 0.023) and ShamPES+CW immediately (p = 0.008) but not with PES+CW or PES+SW. By contrast, there were immediate craniobulbar amplitude changes only with PES+CW (p = 0.020) which were not sustained.

CONCLUSIONS & INFERENCES

We conclude that only PES produced long-term changes in corticopharyngeal excitability whereas combination stimuli were less effective. Our data suggest that PES alone rather than in combination, may be better for the patients who have difficulty in performing voluntary swallows.

Primary motor and sensory cortical areas communicate via spatiotemporally coordinated networks at multiple frequencies

Skilled movements rely on sensory information to shape optimal motor responses, for which the sensory and motor cortical areas are critical. How these areas interact to mediate sensorimotor integration is largely unknown. Here, we measure intercortical coherence between the orofacial motor (MIo) and somatosensory (SIo) areas of cortex as monkeys learn to generate tongue-protrusive force. We report that coherence between MIo and SIo is reciprocal and that neuroplastic changes in coherence gradually emerge over a few days. These functional networks of coherent spiking and local field potentials exhibit frequency-specific spatiotemporal properties. During force generation, theta coherence (2-6 Hz) is prominent and exhibited by numerous paired signals; before or after force generation, coherence is evident in alpha (6-13 Hz), beta (15-30 Hz), and gamma (30-50 Hz) bands, but the functional networks are smaller and weaker. Unlike coherence in the higher frequency bands, the distribution of the phase at peak theta coherence is bimodal with peaks near 0° and ±180°, suggesting that communication between somatosensory and motor areas is coordinated temporally by the phase of theta coherence. Time-sensitive sensorimotor integration and plasticity may rely on coherence of local and large-scale functional networks for cortical processes to operate at multiple temporal and spatial scales.

High-Resolution, Non-Invasive Imaging of Upper Vocal Tract Articulators Compatible with Human Brain Recordings

A complete neurobiological understanding of speech motor control requires determination of the relationship between simultaneously recorded neural activity and the kinematics of the lips, jaw, tongue, and larynx. Many speech articulators are internal to the vocal tract, and therefore simultaneously tracking the kinematics of all articulators is nontrivial—especially in the context of human electrophysiology recordings. Here, we describe a noninvasive, multi-modal imaging system to monitor vocal tract kinematics, demonstrate this system in six speakers during production of nine American English vowels, and provide new analysis of such data. Classification and regression analysis revealed considerable variability in the articulator-to-acoustic relationship across speakers. Non-negative matrix factorization extracted basis sets capturing vocal tract shapes allowing for higher vowel classification accuracy than traditional methods. Statistical speech synthesis generated speech from vocal tract measurements, and we demonstrate perceptual identification. We demonstrate the capacity to predict lip kinematics from ventral sensorimotor cortical activity. These results demonstrate a multi-modal system to non-invasively monitor articulator kinematics during speech production, describe novel analytic methods for relating kinematic data to speech acoustics, and provide the first decoding of speech kinematics from electrocorticography. These advances will be critical for understanding the cortical basis of speech production and the creation of vocal prosthetics.

Cytoarchitecture and cortical connections of the anterior insula and adjacent frontal motor fields in the rhesus monkey

The cytoarchitecture and cortical connections of the ventral motor region are investigated using Nissl, and NeuN staining methods and the fluorescent retrograde tract tracing technique in the rhesus monkey. On the basis of gradual laminar differentiation, it is shown that the ventral motor region stems from the ventral proisocortical area (anterior insula and dorsal Sylvian opercular region). The cytoarchitecture of the ventral motor region is shown to progress in three lines, as we have recently shown for the dorsal motor region. Namely, root (anterior insular and dorsal Sylvian opercular area ProM), belt (ventral premotor cortex) and core (precentral motor cortex) lines. This stepwise architectonic organization is supported by the overall patterns of corticocortical connections. Areas in each line are sequentially interconnected (intralineal connections) and all lines are interconnected (interlinear connections). Moreover, root areas, as well as some of the belt areas of the ventral and dorsal trend are interconnected. The ventral motor region is also connected with the ventral somatosensory areas in a topographic manner. The root and belt areas of ventral motor region are connected with paralimbic, multimodal and prefrontal (outer belt) areas. In contrast, the core area has a comparatively more restricted pattern of corticocortical connections. This architectonic and connectional organization is consistent in part, with the functional organization of the ventral motor region as reported in behavioral and neuroimaging studies which include the mediation of facial expression and emotion, communication, phonic articulation, and language in human.

A motor cortex circuit for motor planning and movement

Activity in motor cortex predicts specific movements seconds before they occur, but how this preparatory activity relates to upcoming movements is obscure. We dissected the conversion of preparatory activity to movement within a structured motor cortex circuit. An anterior lateral region of the mouse cortex (a possible homologue of premotor cortex in primates) contains equal proportions of intermingled neurons predicting ipsi- or contralateral movements, yet unilateral inactivation of this cortical region during movement planning disrupts contralateral movements. Using cell-type-specific electrophysiology, cellular imaging and optogenetic perturbation, we show that layer 5 neurons projecting within the cortex have unbiased laterality. Activity with a contralateral population bias arises specifically in layer 5 neurons projecting to the brainstem, and only late during movement planning. These results reveal the transformation of distributed preparatory activity into movement commands within hierarchically organized cortical circuits.

Sequential evolution of cortical activity and effective connectivity of swallowing using fMRI

Swallowing consists of a hierarchical sequence of primary motor and somatosensory processes. The temporal interplay of different phases is complex and clinical disturbances frequent. Of interest was the temporal interaction of the swallowing network. Time resolution optimized functional magnetic resonance imaging was used to describe the temporal sequence of representation sites of swallowing and their functional connectivity. Sixteen young healthy volunteers were investigated who swallowed 2 ml of water 20 times per run with a repetition time for functional imaging of 514 ms. After applying the general linear model approach to identify activation magnitude in preselected regions of interest repeated measures analysis of variance (rmANOVA) was used to detect relevant effects on lateralization, time, and onset. Furthermore, dynamic causal modeling (DCM) was applied to uncover where the input enters the model and the way in which the cortical regions are connected. The temporal analysis revealed a successive activation starting at the premotor cortex, supplementary motor area (SMA), and bilateral thalamus, followed by the primary sensorimotor cortex, the posterior insula, and cerebellum and culminating with activation in the pons shortly before subsiding. The rmANOVA revealed that activation was lateralized initially to the left hemisphere and gradually moved to the right hemisphere over time. The group random effects DCM analysis resulted in a most likely model that consisted of inputs to SMA and M1S1, bidirectionally connected, and a one-way connection from M1S1 to the posterior insula.

Dynamics of neural population responses in prefrontal cortex indicate changes of mind on single trials

Decision making is a complex process in which different sources of information are combined into a decision variable (DV) that guides action [1, 2]. Neurophysiological studies have typically sought insight into the dynamics of the decision-making process and its neural mechanisms through statistical analysis of large numbers of trials from sequentially recorded single neurons or small groups of neurons [3-6]. However, detecting and analyzing the DV on individual trials has been challenging [7]. Here we show that by recording simultaneously from hundreds of units in prearcuate gyrus of macaque monkeys performing a direction discrimination task, we can predict the monkey's choices with high accuracy and decode DV dynamically as the decision unfolds on individual trials. This advance enabled us to study changes of mind (CoMs) that occasionally happen before the final commitment to a decision [8-10]. On individual trials, the decoded DV varied significantly over time and occasionally changed its sign, identifying a potential CoM. Interrogating the system by random stopping of the decision-making process during the delay period after stimulus presentation confirmed the validity of identified CoMs. Importantly, the properties of the candidate CoMs also conformed to expectations based on prior theoretical and behavioral studies [8]: they were more likely to go from an incorrect to a correct choice, they were more likely for weak and intermediate stimuli than for strong stimuli, and they were more likely earlier in the trial. We suggest that simultaneous recording of large neural populations provides a good estimate of DV and explains idiosyncratic aspects of the decision-making process that were inaccessible before.

Modulation dynamics in the orofacial sensorimotor cortex during motor skill acquisition

The orofacial sensorimotor cortex is known to play a role in motor learning. However, how motor learning changes the dynamics of neuronal activity and whether these changes differ between orofacial primary motor (MIo) and somatosensory (SIo) cortices remain unknown. To address these questions, we used chronically implanted microelectrode arrays to track learning-induced changes in the activity of simultaneously recorded neurons in MIo and SIo as two naive monkeys (Macaca mulatta) were trained in a novel tongue-protrusion task. Over a period of 8-12 d, the monkeys showed behavioral improvements in task performance that were accompanied by rapid and long-lasting changes in neuronal responses in MIo and SIo occurring in parallel: (1) increases in the proportion of task-modulated neurons, (2) increases in the mutual information between tongue-protrusive force and spiking activity, (3) reductions in the across-trial firing rate variability, and (4) transient increases in coherent firing of neuronal pairs. More importantly, the time-resolved mutual information in MIo and SIo exhibited temporal alignment. While showing parallel changes, MIo neurons exhibited a bimodal distribution of peak correlation lag times between spiking activity and force, whereas SIo neurons showed a unimodal distribution. Moreover, coherent activity between pairs of MIo neurons was higher and centered around force onset compared with pairwise coherence of SIo neurons. Overall, the results suggest that the neuroplasticity in MIo and SIo occurring in parallel serves as a substrate for linking sensation and movement during sensorimotor learning, whereas the differing dynamic organizations reflect specific ways to control movement parameters as learning progresses.