The dynamic and task-dependent representational transformation between the motor and sensory systems during speech production

Whole-brain dynamics of articulatory, acoustic and semantic speech representations

Speech production is a complex process that traverses several representations, from the meaning of spoken words (semantic), through the movement of articulatory muscles (articulatory) and, ultimately, to the produced audio waveform (acoustic). In this study, we identify how these different representations of speech are spatially and temporally distributed throughout the depth of the brain. Intracranial neural data is recorded from 15 participants, across 1647 electrode contacts, while overtly speaking 100 unique words. We find a bilateral spatial distribution for all three representations, with a more widespread and temporally dynamic distribution in the left compared to the right hemisphere. The articulatory and acoustic representations share a similar spatial distribution surrounding the Sylvian fissure, while the semantic representation is more widely distributed across the brain in a mostly distinct network. These results highlight the distributed nature of the speech production neural process and the potential of non-motor representations for speech brain-computer interfaces.

Neural representation of sensorimotor features in language-motor areas during auditory and visual perception

Speech processing involves a complex interplay between sensory and motor systems in the brain, essential for early language development. Recent studies have extended this sensory-motor interaction to visual word processing, emphasizing the connection between reading and handwriting during literacy acquisition. Here we show how language-motor areas encode motoric and sensory features of language stimuli during auditory and visual perception, using functional magnetic resonance imaging (fMRI) combined with representational similarity analysis. Chinese-speaking adults completed tasks involving the perception of spoken syllables and written characters, alongside syllable articulation and finger writing tasks to localize speech-motor and writing-motor areas. We found that both language-motor and sensory areas generally encode production-related motoric features across modalities, indicating cooperative interactions between motor and sensory systems. Notably, sensory encoding within sensorimotor areas was observed during auditory speech perception, but not in visual character perception. These findings underscore the dual encoding capacities of language-motor areas, revealing both shared and distinct neural representation patterns across modalities, which may be linked to innate sensory-motor mechanisms and modality-specific processing demands. Our results shed light on the sensorimotor integration mechanisms underlying language perception, highlighting the importance of a cross-modality perspective.

Decoding Single and Paired Phonemes Using 7T Functional MRI

Several studies have shown that mouth movements related to the pronunciation of individual phonemes are represented in the sensorimotor cortex. This would theoretically allow for brain computer interfaces that are capable of decoding continuous speech by training classifiers based on the activity in the sensorimotor cortex related to the production of individual phonemes. To address this, we investigated the decodability of trials with individual and paired phonemes (pronounced consecutively with one second interval) using activity in the sensorimotor cortex. Fifteen participants pronounced 3 different phonemes and 3 combinations of two of the same phonemes in a 7T functional MRI experiment. We confirmed that support vector machine (SVM) classification of single and paired phonemes was possible. Importantly, by combining classifiers trained on single phonemes, we were able to classify paired phonemes with an accuracy of 53% (33% chance level), demonstrating that activity of isolated phonemes is present and distinguishable in combined phonemes. A SVM searchlight analysis showed that the phoneme representations are widely distributed in the ventral sensorimotor cortex. These findings provide insights about the neural representations of single and paired phonemes. Furthermore, it supports the notion that speech BCI may be feasible based on machine learning algorithms trained on individual phonemes using intracranial electrode grids.

The representation of noun-verb distinction in left posterior middle temporal gyrus: evidence from representation similarity analyses

Nouns and verbs are fundamental grammatical building blocks of languages. A key question is whether and where the noun-verb division was represented in the brain. Previous studies mainly used univariate analyses to examine this issue. However, the interpretation of activated brain regions in univariate analyses may be confounded with general cognitive processing and/or confounding variables. We addressed these limitations by using partial representation similarity analysis (RSA) of Chinese nouns and verbs with different levels of imageability. Participants were asked to complete the 1-back grammatical class probe (GCP; an explicit measure) and the 1-back word probe (WP; an implicit measure) tasks while undergoing functional magnetic resonance imaging. RSA results showed that the activation pattern in the left posterior middle temporal gyrus (LpMTG) was significantly correlated with the grammatical class representational dissimilarity matrix in the GCP task after eliminating the potential confounding variables. Moreover, the LpMTG did not overlap with the frontal-parietal regions that were activated by verbs vs. nouns or the task effect (CRP vs. WP) in univariate analyses. These results highlight the role of LpMTG in distinguishing nouns from verbs rather than general cognitive processing.

Intended Speech Classification with EEG Signals Based on a Temporal Attention Mechanism: A Study of Mandarin Vowels

Speech brain-machine interfaces (BCIs) offer an effective means for patients with voice disorders to communicate, and research on decoding electroencephalography (EEG) signals related to intended speech can help to understand the mechanisms of language production in the brain. This study classified the intended speech EEG signals of four Chinese vowels, utilizing a dataset collected from 10 participants. A proposed TA-EEGNet model was employed, incorporating a temporal attention module. The model achieved an accuracy of 49.47%, surpassing other prevalent EEG classification models. The average accuracy in the binary classification of vowels was 69.83%. The vowels /u/ and /ü/ were classified with the lowest accuracy, suggesting difficulties in classifying vowels with similar articulatory movements based on intended speech EEG signals. Furthermore, the research analyzed the classification performance using data of different brain regions. The results showed that the auditory cortex, Broca's and Wernicke's areas, prefrontal cortex, and motor cortex outperformed the sensory cortex, indicating their contributions in the intended speech process of Mandarin vowels. Results also showed left hemisphere dominance. These findings contribute to the study of the neural mechanisms underlying speech production and articulatory movements, emphasizing the potential of speech BCIs to improve communication for people with speech disorders.

Decoding kinematic information from beta-band motor rhythms of speech motor cortex: a methodological/analytic approach using concurrent speech movement tracking and magnetoencephalography

Introduction

Articulography and functional neuroimaging are two major tools for studying the neurobiology of speech production. Until now, however, it has generally not been feasible to use both in the same experimental setup because of technical incompatibilities between the two methodologies.

Methods

Here we describe results from a novel articulography system dubbed Magneto-articulography for the Assessment of Speech Kinematics (MASK), which is technically compatible with magnetoencephalography (MEG) brain scanning systems. In the present paper we describe our methodological and analytic approach for extracting brain motor activities related to key kinematic and coordination event parameters derived from time-registered MASK tracking measurements. Data were collected from 10 healthy adults with tracking coils on the tongue, lips, and jaw. Analyses targeted the gestural landmarks of reiterated utterances/ipa/ and /api/, produced at normal and faster rates.

Results

The results show that (1) Speech sensorimotor cortex can be reliably located in peri-rolandic regions of the left hemisphere; (2) mu (8-12 Hz) and beta band (13-30 Hz) neuromotor oscillations are present in the speech signals and contain information structures that are independent of those present in higher-frequency bands; and (3) hypotheses concerning the information content of speech motor rhythms can be systematically evaluated with multivariate pattern analytic techniques.

Discussion

These results show that MASK provides the capability, for deriving subject-specific articulatory parameters, based on well-established and robust motor control parameters, in the same experimental setup as the MEG brain recordings and in temporal and spatial co-register with the brain data. The analytic approach described here provides new capabilities for testing hypotheses concerning the types of kinematic information that are encoded and processed within specific components of the speech neuromotor system.

Functional and structural abnormalities of the speech disorders: a multimodal activation likelihood estimation meta-analysis

Speech disorders are associated with different degrees of functional and structural abnormalities. However, the abnormalities associated with specific disorders, and the common abnormalities shown by all disorders, remain unclear. Herein, a meta-analysis was conducted to integrate the results of 70 studies that compared 1843 speech disorder patients (dysarthria, dysphonia, stuttering, and aphasia) to 1950 healthy controls in terms of brain activity, functional connectivity, gray matter, and white matter fractional anisotropy. The analysis revealed that compared to controls, the dysarthria group showed higher activity in the left superior temporal gyrus and lower activity in the left postcentral gyrus. The dysphonia group had higher activity in the right precentral and postcentral gyrus. The stuttering group had higher activity in the right inferior frontal gyrus and lower activity in the left inferior frontal gyrus. The aphasia group showed lower activity in the bilateral anterior cingulate gyrus and left superior frontal gyrus. Across the four disorders, there were concurrent lower activity, gray matter, and fractional anisotropy in motor and auditory cortices, and stronger connectivity between the default mode network and frontoparietal network. These findings enhance our understanding of the neural basis of speech disorders, potentially aiding clinical diagnosis and intervention.

Neural representations of the content and production of human vocalization

Speech, as the spoken form of language, is fundamental for human communication. The phenomenon of covert inner speech implies functional independence of speech content and motor production. However, it remains unclear how a flexible mapping between speech content and production is achieved on the neural level. To address this, we recorded magnetoencephalography in humans performing a rule-based vocalization task. On each trial, vocalization content (one of two vowels) and production form (overt or covert) were instructed independently. Using multivariate pattern analysis, we found robust neural information about vocalization content and production, mostly originating from speech areas of the left hemisphere. Production signals dynamically transformed upon presentation of the content cue, whereas content signals remained largely stable throughout the trial. In sum, our results show dissociable neural representations of vocalization content and production in the human brain and provide insights into the neural dynamics underlying human vocalization.

Dual-stream cortical pathways mediate sensory prediction

Predictions are constantly generated from diverse sources to optimize cognitive functions in the ever-changing environment. However, the neural origin and generation process of top-down induced prediction remain elusive. We hypothesized that motor-based and memory-based predictions are mediated by distinct descending networks from motor and memory systems to the sensory cortices. Using functional magnetic resonance imaging (fMRI) and a dual imagery paradigm, we found that motor and memory upstream systems activated the auditory cortex in a content-specific manner. Moreover, the inferior and posterior parts of the parietal lobe differentially relayed predictive signals in motor-to-sensory and memory-to-sensory networks. Dynamic causal modeling of directed connectivity revealed selective enabling and modulation of connections that mediate top-down sensory prediction and ground the distinctive neurocognitive basis of predictive processing.

The nested hierarchy of overt, mouthed, and imagined speech activity evident in intracranial recordings

Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts.

Speech Kinematics and Coordination Measured With an MEG-Compatible Speech Tracking System

Articulography and functional neuroimaging are two major tools for studying the neurobiology of speech production. Until recently, however, it has generally not been possible to use both in the same experimental setup because of technical incompatibilities between the two methodologies. Here we describe results from a novel articulography system dubbed Magneto-articulography for the Assessment of Speech Kinematics (MASK), which we used to derive kinematic profiles of oro-facial movements during speech. MASK was used to characterize speech kinematics in two healthy adults, and the results were compared to measurements from a separate participant with a conventional Electromagnetic Articulography (EMA) system. Analyses targeted the gestural landmarks of reiterated utterances /ipa/, /api/ and /pataka/. The results demonstrate that MASK reliably characterizes key kinematic and movement coordination parameters of speech motor control. Since these parameters are intrinsically registered in time with concurrent magnetoencephalographic (MEG) measurements of neuromotor brain activity, this methodology paves the way for innovative cross-disciplinary studies of the neuromotor control of human speech production, speech development, and speech motor disorders.

Mapping Specific Mental Content during Musical Imagery

Humans can mentally represent auditory information without an external stimulus, but the specificity of these internal representations remains unclear. Here, we asked how similar the temporally unfolding neural representations of imagined music are compared to those during the original perceived experience. We also tested whether rhythmic motion can influence the neural representation of music during imagery as during perception. Participants first memorized six 1-min-long instrumental musical pieces with high accuracy. Functional MRI data were collected during: 1) silent imagery of melodies to the beat of a visual metronome; 2) same but while tapping to the beat; and 3) passive listening. During imagery, inter-subject correlation analysis showed that melody-specific temporal response patterns were reinstated in right associative auditory cortices. When tapping accompanied imagery, the melody-specific neural patterns were reinstated in more extensive temporal-lobe regions bilaterally. These results indicate that the specific contents of conscious experience are encoded similarly during imagery and perception in the dynamic activity of auditory cortices. Furthermore, rhythmic motion can enhance the reinstatement of neural patterns associated with the experience of complex sounds, in keeping with models of motor to sensory influences in auditory processing.

Efference copy in kinesthetic perception: a copy of what is it?

A number of notions in the fields of motor control and kinesthetic perception have been used without clear definitions. In this review, we consider definitions for efference copy, percept, and sense of effort based on recent studies within the physical approach, which assumes that the neural control of movement is based on principles of parametric control and involves defining time-varying profiles of spatial referent coordinates for the effectors. The apparent redundancy in both motor and perceptual processes is reconsidered based on the principle of abundance. Abundance of efferent and afferent signals is viewed as the means of stabilizing both salient action characteristics and salient percepts formalized as stable manifolds in high-dimensional spaces of relevant elemental variables. This theoretical scheme has led recently to a number of novel predictions and findings. These include, in particular, lower accuracy in perception of variables produced by elements involved in a multielement task compared with the same elements in single-element tasks, dissociation between motor and perceptual effects of muscle coactivation, force illusions induced by muscle vibration, and errors in perception of unintentional drifts in performance. Taken together, these results suggest that participation of efferent signals in perception frequently involves distorted copies of actual neural commands, particularly those to antagonist muscles. Sense of effort is associated with such distorted efferent signals. Distortions in efference copy happen spontaneously and can also be caused by changes in sensory signals, e.g., those produced by muscle vibration.