The cortical computations underlying feedback control in vocal production

Spatiotemporal Mapping of Auditory Onsets during Speech Production

The human auditory cortex is organized according to the timing and spectral characteristics of speech sounds during speech perception. During listening, the posterior superior temporal gyrus is organized according to onset responses, which segment acoustic boundaries in speech, and sustained responses, which further process phonological content. When we speak, the auditory system is actively processing the sound of our own voice to detect and correct speech errors in real time. This manifests in neural recordings as suppression of auditory responses during speech production compared with perception, but whether this differentially affects the onset and sustained temporal profiles is not known. Here, we investigated this question using intracranial EEG recorded from seventeen pediatric, adolescent, and adult patients with medication-resistant epilepsy while they performed a reading/listening task. We identified onset and sustained responses to speech in the bilateral auditory cortex and observed a selective suppression of onset responses during speech production. We conclude that onset responses provide a temporal landmark during speech perception that is redundant with forward prediction during speech production and are therefore suppressed. Phonological feature tuning in these "onset suppression" electrodes remained stable between perception and production. Notably, auditory onset responses and phonological feature tuning were present in the posterior insula during both speech perception and production, suggesting an anatomically and functionally separate auditory processing zone that we believe to be involved in multisensory integration during speech perception and feedback control.

Effects of attentional instructions on the behavioral and neural mechanisms of speech auditory feedback control

The present study investigated how instructions for paying attention to auditory feedback may affect speech error detection and sensorimotor control. Electroencephalography (EEG) and speech signals were recorded from 21 neurologically intact adult subjects while they produced the speech vowel sound /a/ and received randomized ±100 cents pitch-shift alterations in their real-time auditory feedback. Subjects were instructed to pay attention to their auditory feedback and press a button to indicate whether they detected a pitch-shift stimulus during trials. Data for this group was compared with 22 matched subjects who completed the same speech task under altered auditory feedback condition without attentional instructions. Results revealed a significantly smaller magnitude of speech compensations in the attentional-instruction vs. no-instruction group and a positive linear association between the magnitude of compensations and P2 event-related potential (ERP) amplitudes. In addition, we found that the amplitude of P2 ERP component was significantly larger in the attentional-instruction vs. no-instruction group. Source localization analysis showed that this effect was accounted for by significantly stronger neural activities in the right hemisphere insula, precentral gyrus, postcentral gyrus, transverse temporal gyrus, and superior temporal gyrus in the attentional-instruction group. These findings suggest that attentional instructions may enhance speech auditory feedback error detection, and subsequently improve sensorimotor control via generating more stable speech outputs (i.e., smaller compensations) in response to pitch-shift alterations. Our data are informative for advancing theoretical models and motivating targeted interventions with a focus on the role of attentional instructions for improving treatment outcomes in patients with motor speech disorders.

Processing of auditory feedback in perisylvian and insular cortex

When we speak, we not only make movements with our mouth, lips, and tongue, but we also hear the sound of our own voice. Thus, speech production in the brain involves not only controlling the movements we make, but also auditory and sensory feedback. Auditory responses are typically suppressed during speech production compared to perception, but how this manifests across space and time is unclear. Here we recorded intracranial EEG in seventeen pediatric, adolescent, and adult patients with medication-resistant epilepsy who performed a reading/listening task to investigate how other auditory responses are modulated during speech production. We identified onset and sustained responses to speech in bilateral auditory cortex, with a selective suppression of onset responses during speech production. Onset responses provide a temporal landmark during speech perception that is redundant with forward prediction during speech production. Phonological feature tuning in these "onset suppression" electrodes remained stable between perception and production. Notably, the posterior insula responded at sentence onset for both perception and production, suggesting a role in multisensory integration during feedback control.

Multiple processes of vocal sensory-motor interaction in primate auditory cortex

Sensory-motor interactions in the auditory system play an important role in vocal self-monitoring and control. These result from top-down corollary discharges, relaying predictions about vocal timing and acoustics. Recent evidence suggests such signals may be two distinct processes, one suppressing neural activity during vocalization and another enhancing sensitivity to sensory feedback, rather than a single mechanism. Single-neuron recordings have been unable to disambiguate due to overlap of motor signals with sensory inputs. Here, we sought to disentangle these processes in marmoset auditory cortex during production of multi-phrased 'twitter' vocalizations. Temporal responses revealed two timescales of vocal suppression: temporally-precise phasic suppression during phrases and sustained tonic suppression. Both components were present within individual neurons, however, phasic suppression presented broadly regardless of frequency tuning (gating), while tonic was selective for vocal frequencies and feedback (prediction). This suggests that auditory cortex is modulated by concurrent corollary discharges during vocalization, with different computational mechanisms.

Parallel Encoding of Speech in Human Frontal and Temporal Lobes

Models of speech perception are centered around a hierarchy in which auditory representations in the thalamus propagate to primary auditory cortex, then to the lateral temporal cortex, and finally through dorsal and ventral pathways to sites in the frontal lobe. However, evidence for short latency speech responses and low-level spectrotemporal representations in frontal cortex raises the question of whether speech-evoked activity in frontal cortex strictly reflects downstream processing from lateral temporal cortex or whether there are direct parallel pathways from the thalamus or primary auditory cortex to the frontal lobe that supplement the traditional hierarchical architecture. Here, we used high-density direct cortical recordings, high-resolution diffusion tractography, and hemodynamic functional connectivity to evaluate for evidence of direct parallel inputs to frontal cortex from low-level areas. We found that neural populations in the frontal lobe show speech-evoked responses that are synchronous or occur earlier than responses in the lateral temporal cortex. These short latency frontal lobe neural populations encode spectrotemporal speech content indistinguishable from spectrotemporal encoding patterns observed in the lateral temporal lobe, suggesting parallel auditory speech representations reaching temporal and frontal cortex simultaneously. This is further supported by white matter tractography and functional connectivity patterns that connect the auditory nucleus of the thalamus (medial geniculate body) and the primary auditory cortex to the frontal lobe. Together, these results support the existence of a robust pathway of parallel inputs from low-level auditory areas to frontal lobe targets and illustrate long-range parallel architecture that works alongside the classical hierarchical speech network model.

Temporal scaling of motor cortical dynamics reveals hierarchical control of vocal production

Neocortical activity is thought to mediate voluntary control over vocal production, but the underlying neural mechanisms remain unclear. In a highly vocal rodent, the male Alston's singing mouse, we investigate neural dynamics in the orofacial motor cortex (OMC), a structure critical for vocal behavior. We first describe neural activity that is modulated by component notes (~100 ms), probably representing sensory feedback. At longer timescales, however, OMC neurons exhibit diverse and often persistent premotor firing patterns that stretch or compress with song duration (~10 s). Using computational modeling, we demonstrate that such temporal scaling, acting through downstream motor production circuits, can enable vocal flexibility. These results provide a framework for studying hierarchical control circuits, a common design principle across many natural and artificial systems.

When direction matters: Neural correlates of interlimb coordination of rhythm and beat

In a previous experiment, we found evidence for a bodily hierarchy governing interlimb coordination of rhythm and beat, using five effectors: 1) Left foot, 2) Right foot, 3) Left hand, 4) Right hand and 5) Voice. The hierarchy implies that, during simultaneous rhythm and beat performance and using combinations of two of these effectors, executing the task by performing the rhythm with an effector that has a higher number than the beat effector is significantly easier than vice versa. To investigate the neural underpinnings of this proposed bodily hierarchy, we here scanned 46 professional musicians using fMRI as they performed a rhythmic pattern with one effector while keeping the beat with another. The conditions combined the voice and the right hand (V + RH), the right hand and the left hand (RH + LH), and the left hand and the right foot (LH + RF). Each effector combination was performed with and against the bodily hierarchy. Going against the bodily hierarchy increased tapping errors significantly and also increased activity in key brain areas functionally associated with top-down sensorimotor control and bottom-up feedback processing, such as the cerebellum and SMA. Conversely, going with the bodily hierarchy engaged areas functionally associated with the default mode network and regions involved in emotion processing. Theories of general brain function that hold prediction as a key principle, propose that action and perception are governed by the brain's attempt to minimise prediction error at different levels in the brain. Following this viewpoint, our results indicate that going against the hierarchy induces stronger prediction errors, while going with the hierarchy allows for a higher degree of automatization. Our results also support the notion of a bodily hierarchy in motor control that prioritizes certain conductive and supportive tapping roles in specific effector combinations.

Test-Retest Reliability of Behavioral Assays of Feedforward and Feedback Auditory-Motor Control of Voice and Articulation

PURPOSE

Behavioral assays of feedforward and feedback auditory-motor control of voice and articulation frequently are used to make inferences about underlying neural mechanisms and to study speech development and disorders. However, no studies have examined the test-retest reliability of such measures, which is critical for rigorous study of auditory-motor control. Thus, the purpose of the present study was to assess the reliability of assays of feedforward and feedback control in voice versus articulation domains.

METHOD

Twenty-eight participants (14 cisgender women, 12 cisgender men, one transgender man, one transmasculine/nonbinary) who denied any history of speech, hearing, or neurological impairment were measured for responses to predictable versus unexpected auditory feedback perturbations of vocal (fundamental frequency, fo) and articulatory (first formant, F1) acoustic parameters twice, with 3-6 weeks between sessions. Reliability was measured with intraclass correlations.

RESULTS

Opposite patterns of reliability were observed for fo and F1; fo reflexive responses showed good reliability and fo adaptive responses showed poor reliability, whereas F1 reflexive responses showed poor reliability and F1 adaptive responses showed moderate reliability. However, a criterion-referenced categorical measurement of fo adaptive responses as typical versus atypical showed substantial test-retest agreement.

CONCLUSIONS

Individual responses to some behavioral assays of auditory-motor control of speech should be interpreted with caution, which has implications for several fields of research. Additional research is needed to establish reliable criterion-referenced measures of F1 adaptive responses as well as fo and F1 reflexive responses. Furthermore, the opposite patterns of test-retest reliability observed for voice versus articulation add to growing evidence for differences in underlying neural control mechanisms.

Speaker-induced Suppression in EEG during a Naturalistic Reading and Listening Task

Speaking elicits a suppressed neural response when compared with listening to others' speech, a phenomenon known as speaker-induced suppression (SIS). Previous research has focused on investigating SIS at constrained levels of linguistic representation, such as the individual phoneme and word level. Here, we present scalp EEG data from a dual speech perception and production task where participants read sentences aloud then listened to playback of themselves reading those sentences. Playback was separated into immediate repetition of the previous trial and randomized repetition of a former trial to investigate if forward modeling of responses during passive listening suppresses the neural response. Concurrent EMG was recorded to control for movement artifact during speech production. In line with previous research, ERP analyses at the sentence level demonstrated suppression of early auditory components of the EEG for production compared with perception. To evaluate whether linguistic abstractions (in the form of phonological feature tuning) are suppressed during speech production alongside lower-level acoustic information, we fit linear encoding models that predicted scalp EEG based on phonological features, EMG activity, and task condition. We found that phonological features were encoded similarly between production and perception. However, this similarity was only observed when controlling for movement by using the EMG response as an additional regressor. Our results suggest that SIS operates at a sensory representational level and is dissociated from higher order cognitive and linguistic processing that takes place during speech perception and production. We also detail some important considerations when analyzing EEG during continuous speech production.

Spatiotemporal dynamics characterise spectral connectivity profiles of continuous speaking and listening

Speech production and perception are fundamental processes of human cognition that both rely on intricate processing mechanisms that are still poorly understood. Here, we study these processes by using magnetoencephalography (MEG) to comprehensively map connectivity of regional brain activity within the brain and to the speech envelope during continuous speaking and listening. Our results reveal not only a partly shared neural substrate for both processes but also a dissociation in space, delay, and frequency. Neural activity in motor and frontal areas is coupled to succeeding speech in delta band (1 to 3 Hz), whereas coupling in the theta range follows speech in temporal areas during speaking. Neural connectivity results showed a separation of bottom-up and top-down signalling in distinct frequency bands during speaking. Here, we show that frequency-specific connectivity channels for bottom-up and top-down signalling support continuous speaking and listening. These findings further shed light on the complex interplay between different brain regions involved in speech production and perception.

DIVA Meets EEG: Model Validation Using Formant-Shift Reflex

The neurocomputational model 'Directions into Velocities of Articulators' (DIVA) was developed to account for various aspects of normal and disordered speech production and acquisition. The neural substrates of DIVA were established through functional magnetic resonance imaging (fMRI), providing physiological validation of the model. This study introduces DIVA_EEG an extension of DIVA that utilizes electroencephalography (EEG) to leverage the high temporal resolution and broad availability of EEG over fMRI. For the development of DIVA_EEG, EEG-like signals were derived from original equations describing the activity of the different DIVA maps. Synthetic EEG associated with the utterance of syllables was generated when both unperturbed and perturbed auditory feedback (first formant perturbations) were simulated. The cortical activation maps derived from synthetic EEG closely resembled those of the original DIVA model. To validate DIVA_EEG, the EEG of individuals with typical voices (N = 30) was acquired during an altered auditory feedback paradigm. The resulting empirical brain activity maps significantly overlapped with those predicted by DIVA_EEG. In conjunction with other recent model extensions, DIVA_EEG lays the foundations for constructing a complete neurocomputational framework to tackle vocal and speech disorders, which can guide model-driven personalized interventions.

Impaired Speaking-Induced Suppression in Alzheimer's Disease

Alzheimer's disease (AD) is a neurodegenerative disease involving cognitive impairment and abnormalities in speech and language. Here, we examine how AD affects the fidelity of auditory feedback predictions during speaking. We focus on the phenomenon of speaking-induced suppression (SIS), the auditory cortical responses' suppression during auditory feedback processing. SIS is determined by subtracting the magnitude of auditory cortical responses during speaking from listening to playback of the same speech. Our state feedback control (SFC) model of speech motor control explains SIS as arising from the onset of auditory feedback matching a prediction of that feedback onset during speaking, a prediction that is absent during passive listening to playback of the auditory feedback. Our model hypothesizes that the auditory cortical response to auditory feedback reflects the mismatch with the prediction: small during speaking, large during listening, with the difference being SIS. Normally, during speaking, auditory feedback matches its predictions, then SIS will be large. Any reductions in SIS will indicate inaccuracy in auditory feedback prediction not matching the actual feedback. We investigated SIS in AD patients [n = 20; mean (SD) age, 60.77 (10.04); female (%), 55.00] and healthy controls [n = 12; mean (SD) age, 63.68 (6.07); female (%), 83.33] through magnetoencephalography (MEG)-based functional imaging. We found a significant reduction in SIS at ∼100 ms in AD patients compared with healthy controls (linear mixed effects model, F (1,57.5) = 6.849, p = 0.011). The results suggest that AD patients generate inaccurate auditory feedback predictions, contributing to abnormalities in AD speech.

Functional connectivity between parietal and temporal lobes mediates internal forward models during speech production

Internal forward models hypothesize functional links between motor and sensory systems for predicting the consequences of actions. Recently, the cascaded theory proposes that somatosensory estimation in the inferior parietal lobe (IPL) can be a relay computational structure, converting motor signals into predictions of auditory consequences in a serial processing manner during speech production. The study used fMRI with functional connectivity (FC) analyses to investigate the proposed cascaded processes using three speech tasks: overt articulation (OA), silent articulation (SA) and imagined articulation (IA). The FC results showed that connectivity between aIPL and STG was increased in OA compared with SA, suggesting that the relationship between somatosensory and auditory estimations can be modulated by speech tasks. Moreover, stronger connectivity between IFGoper and pIPL, and between pIPL and STG were observed in SA and IA compared with OA. These results are consistent with a cascaded process in the internal forward models.

Neural dynamics in the rodent motor cortex enables flexible control of vocal timing

Neocortical activity is thought to mediate voluntary control over vocal production, but the underlying neural mechanisms remain unclear. In a highly vocal rodent, the Alston's singing mouse, we investigate neural dynamics in the orofacial motor cortex (OMC), a structure critical for vocal behavior. We first describe neural activity that is modulated by component notes (approx. 100 ms), likely representing sensory feedback. At longer timescales, however, OMC neurons exhibit diverse and often persistent premotor firing patterns that stretch or compress with song duration (approx. 10 s). Using computational modeling, we demonstrate that such temporal scaling, acting via downstream motor production circuits, can enable vocal flexibility. These results provide a framework for studying hierarchical control circuits, a common design principle across many natural and artificial systems.

Continuous theta burst stimulation over left and right supramarginal gyri demonstrates their involvement in auditory feedback control of vocal production

The supramarginal gyrus (SMG) has been implicated in auditory-motor integration for vocal production. However, whether the SMG is bilaterally or unilaterally involved in auditory feedback control of vocal production in a causal manner remains unclear. The present event-related potential (ERP) study investigated the causal roles of the left and right SMG to auditory-vocal integration using neuronavigated continuous theta burst stimulation (c-TBS). Twenty-four young adults produced sustained vowel phonations and heard their voice unexpectedly pitch-shifted by ±200 cents after receiving active or sham c-TBS over the left or right SMG. As compared to sham stimulation, c-TBS over the left or right SMG led to significantly smaller vocal compensations for pitch perturbations that were accompanied by smaller cortical P2 responses. Moreover, no significant differences were found in the vocal and ERP responses when comparing active c-TBS over the left vs. right SMG. These findings provide neurobehavioral evidence for a causal influence of both the left and right SMG on auditory feedback control of vocal production. Decreased vocal compensations paralleled by reduced P2 responses following c-TBS over the bilateral SMG support their roles for auditory-motor transformation in a bottom-up manner: receiving auditory feedback information and mediating vocal compensations for feedback errors.

Identifying distinct latent classes of pitch-shift response consistency: Evidence from manipulating the predictability of shift direction

Auditory feedback plays an important role in regulating our vocal pitch. When pitch shifts suddenly appear in auditory feedback, the majority of the responses are opposing, correcting for the mismatch between perceived pitch and actual pitch. However, research has indicated that following responses to auditory perturbation could be common. This study attempts to explore the ways individual speakers would respond to pitch perturbation (using an opposing response or a following response) from trial to trial. Thirty-six native speakers of Mandarin produced the vowel /a/ while receiving perturbed pitch at a random time (500 ~ 700 ms) after vocal onset for a duration of 200 ms. Three blocks of 30 trials that differed in the pitch-shift stimulus direction were recorded in a randomized order: (a) the down-only condition where pitch was shifted downwards 250 cents; (b) the up-only condition where pitch was shifted upwards 250 cents; and (c) the random condition where downshifts and upshifts occurred randomly and were equally likely. The participants were instructed to ignore the pitch shifts. Results from the latent class analysis show that at the individual level across trials, 57% of participants were switchers, 28% were opposers, and 15% were followers. Our results support that speakers produce a mix of opposing and following responses when they respond to perturbed pitch. Specifically, the proportion of followers was conditional on the expectancy of pitch-shift stimulus direction: More followers were observed when the pitch-shift stimulus direction was predictable. Closer inspection of the levels of response consistency in different time phases shows that a particular mechanism (opposing or following) was initially implemented; the two mechanisms may alternate in the middle phase; and then finally, the pitch-shift response was featured as a particular mechanism near the end phase.

Dynamic auditory contributions to error detection revealed in the discrimination of Same and Different syllable pairs

During speech production auditory regions operate in concert with the anterior dorsal stream to facilitate online error detection. As the dorsal stream also is known to activate in speech perception, the purpose of the current study was to probe the role of auditory regions in error detection during auditory discrimination tasks as stimuli are encoded and maintained in working memory. A priori assumptions are that sensory mismatch (i.e., error) occurs during the discrimination of Different (mismatched) but not Same (matched) syllable pairs. Independent component analysis was applied to raw EEG data recorded from 42 participants to identify bilateral auditory alpha rhythms, which were decomposed across time and frequency to reveal robust patterns of event related synchronization (ERS; inhibition) and desynchronization (ERD; processing) over the time course of discrimination events. Results were characterized by bilateral peri-stimulus alpha ERD transitioning to alpha ERS in the late trial epoch, with ERD interpreted as evidence of working memory encoding via Analysis by Synthesis and ERS considered evidence of speech-induced-suppression arising during covert articulatory rehearsal to facilitate working memory maintenance. The transition from ERD to ERS occurred later in the left hemisphere in Different trials than in Same trials, with ERD and ERS temporally overlapping during the early post-stimulus window. Results were interpreted to suggest that the sensory mismatch (i.e., error) arising from the comparison of the first and second syllable elicits further processing in the left hemisphere to support working memory encoding and maintenance. Results are consistent with auditory contributions to error detection during both encoding and maintenance stages of working memory, with encoding stage error detection associated with stimulus concordance and maintenance stage error detection associated with task-specific retention demands.

A voice without a mouth no more: The neurobiology of language and consciousness

Most research on the neurobiology of language ignores consciousness and vice versa. Here, language, with an emphasis on inner speech, is hypothesised to generate and sustain self-awareness, i.e., higher-order consciousness. Converging evidence supporting this hypothesis is reviewed. To account for these findings, a 'HOLISTIC' model of neurobiology of language, inner speech, and consciousness is proposed. It involves a 'core' set of inner speech production regions that initiate the experience of feeling and hearing words. These take on affective qualities, deriving from activation of associated sensory, motor, and emotional representations, involving a largely unconscious dynamic 'periphery', distributed throughout the whole brain. Responding to those words forms the basis for sustained network activity, involving 'default mode' activation and prefrontal and thalamic/brainstem selection of contextually relevant responses. Evidence for the model is reviewed, supporting neuroimaging meta-analyses conducted, and comparisons with other theories of consciousness made. The HOLISTIC model constitutes a more parsimonious and complete account of the 'neural correlates of consciousness' that has implications for a mechanistic account of mental health and wellbeing.

Timing of brain entrainment to the speech envelope during speaking, listening and self-listening

⁠This study investigates the dynamics of speech envelope tracking during speech production, listening and self-listening. We use a paradigm in which participants listen to natural speech (Listening), produce natural speech (Speech Production), and listen to the playback of their own speech (Self-Listening), all while their neural activity is recorded with EEG. After time-locking EEG data collection and auditory recording and playback, we used a Gaussian copula mutual information measure to estimate the relationship between information content in the EEG and auditory signals. In the 2-10 Hz frequency range, we identified different latencies for maximal speech envelope tracking during speech production and speech perception. Maximal speech tracking takes place approximately 110 ms after auditory presentation during perception and 25 ms before vocalisation during speech production. These results describe a specific timeline for speech tracking in speakers and listeners in line with the idea of a speech chain and hence, delays in communication.

Auditory and somatosensory feedback mechanisms of laryngeal and articulatory speech motor control

PURPOSE

Speech production is a complex motor task involving multiple subsystems. The relationships between these subsystems need to be comprehensively investigated to understand the underlying mechanisms of speech production. The goal of this paper is to examine the differential contributions of 1) auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory speech production subsystems on speech motor control at an individual speaker level using altered auditory and somatosensory feedback paradigms.

METHODS

Twenty young adults completed speaking tasks in which sudden and unpredictable auditory and physical perturbations were applied to the laryngeal and articulatory speech production subsystems. Auditory perturbations were applied to laryngeal or articulatory acoustic features of speech. Physical perturbations were applied to the larynx and the jaw. Pearson-product moment correlation coefficients were calculated between 1) auditory and somatosensory reflexive responses to investigate relationships between auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory reflexive responses as well as acuity measures to investigate the relationship between auditory-motor features of laryngeal and articulatory subsystems.

RESULTS

No statistically significant correlations were found concerning the relationships between auditory and somatosensory feedback. No statistically significant correlations were found between auditory-motor features in the laryngeal and articulatory control subsystems.

CONCLUSION

Results suggest that the laryngeal and articulatory speech production subsystems operate with differential auditory and somatosensory feedback control mechanisms. The outcomes suggest that current models of speech motor control should consider decoupling laryngeal and articulatory domains to better model speech motor control processes.

LaDIVA: A neurocomputational model providing laryngeal motor control for speech acquisition and production

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA's laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

Music in the brain

Music is ubiquitous across human cultures - as a source of affective and pleasurable experience, moving us both physically and emotionally - and learning to play music shapes both brain structure and brain function. Music processing in the brain - namely, the perception of melody, harmony and rhythm - has traditionally been studied as an auditory phenomenon using passive listening paradigms. However, when listening to music, we actively generate predictions about what is likely to happen next. This enactive aspect has led to a more comprehensive understanding of music processing involving brain structures implicated in action, emotion and learning. Here we review the cognitive neuroscience literature of music perception. We show that music perception, action, emotion and learning all rest on the human brain's fundamental capacity for prediction - as formulated by the predictive coding of music model. This Review elucidates how this formulation of music perception and expertise in individuals can be extended to account for the dynamics and underlying brain mechanisms of collective music making. This in turn has important implications for human creativity as evinced by music improvisation. These recent advances shed new light on what makes music meaningful from a neuroscientific perspective.

Temporal specificity of abnormal neural oscillations during phonatory events in laryngeal dystonia

Laryngeal dystonia is a debilitating disorder of voicing in which the laryngeal muscles are intermittently in spasm resulting in involuntary interruptions during speech. The central pathophysiology of laryngeal dystonia, underlying computational impairments in vocal motor control, remains poorly understood. Although prior imaging studies have found aberrant activity in the CNS during phonation in patients with laryngeal dystonia, it is not known at what timepoints during phonation these abnormalities emerge and what function may be impaired. To investigate this question, we recruited 22 adductor laryngeal dystonia patients (15 female, age range = 28.83-72.46 years) and 18 controls (eight female, age range = 27.40-71.34 years). We leveraged the fine temporal resolution of magnetoencephalography to monitor neural activity around glottal movement onset, subsequent voice onset and after the onset of pitch feedback perturbations. We examined event-related beta-band (12-30 Hz) and high-gamma-band (65-150 Hz) neural oscillations. Prior to glottal movement onset, we observed abnormal frontoparietal motor preparatory activity. After glottal movement onset, we observed abnormal activity in the somatosensory cortex persisting through voice onset. Prior to voice onset and continuing after, we also observed abnormal activity in the auditory cortex and the cerebellum. After pitch feedback perturbation onset, we observed no differences between controls and patients in their behavioural responses to the perturbation. But in patients, we did find abnormal activity in brain regions thought to be involved in the auditory feedback control of vocal pitch (premotor, motor, somatosensory and auditory cortices). Our study results confirm the abnormal processing of somatosensory feedback that has been seen in other studies. However, there were several remarkable findings in our study. First, patients have impaired vocal motor activity even before glottal movement onset, suggesting abnormal movement preparation. These results are significant because (i) they occur before movement onset, abnormalities in patients cannot be ascribed to deficits in vocal performance and (ii) they show that neural abnormalities in laryngeal dystonia are more than just abnormal responses to sensory feedback during phonation as has been hypothesized in some previous studies. Second, abnormal auditory cortical activity in patients begins even before voice onset, suggesting abnormalities in setting up auditory predictions before the arrival of auditory feedback at voice onset. Generally, activation abnormalities identified in key brain regions within the speech motor network around various phonation events not only provide temporal specificity to neuroimaging phenotypes in laryngeal dystonia but also may serve as potential therapeutic targets for neuromodulation.

Controversy over the temporal cortical terminations of the left arcuate fasciculus: a reappraisal

The arcuate fasciculus has been considered a major dorsal fronto-temporal white matter pathway linking frontal language production regions with auditory perception in the superior temporal gyrus, the so-called Wernicke's area. In line with this tradition, both historical and contemporary models of language function have assigned primacy to superior temporal projections of the arcuate fasciculus. However, classical anatomical descriptions and emerging behavioural data are at odds with this assumption. On one hand, fronto-temporal projections to Wernicke's area may not be unique to the arcuate fasciculus. On the other hand, dorsal stream language deficits have been reported also for damage to middle, inferior and basal temporal gyri which may be linked to arcuate disconnection. These findings point to a reappraisal of arcuate projections in the temporal lobe. Here, we review anatomical and functional evidence regarding the temporal cortical terminations of the left arcuate fasciculus by incorporating dissection and tractography findings with stimulation data using cortico-cortical evoked potentials and direct electrical stimulation mapping in awake patients. Firstly, we discuss the fibers of the arcuate fasciculus projecting to the superior temporal gyrus and the functional rostro-caudal gradient in this region where both phonological encoding and auditory-motor transformation may be performed. Caudal regions within the temporoparietal junction may be involved in articulation and associated with temporoparietal projections of the third branch of the superior longitudinal fasciculus, while more rostral regions may support encoding of acoustic phonetic features, supported by arcuate fibres. We then move to examine clinical data showing that multimodal phonological encoding is facilitated by projections of the arcuate fasciculus to superior, but also middle, inferior and basal temporal regions. Hence, we discuss how projections of the arcuate fasciculus may contribute to acoustic (middle-posterior superior and middle temporal gyri), visual (posterior inferior temporal/fusiform gyri comprising the visual word form area) and lexical (anterior-middle inferior temporal/fusiform gyri in the basal temporal language area) information in the temporal lobe to be processed, encoded and translated into a dorsal phonological route to the frontal lobe. Finally, we point out surgical implications for this model in terms of the prediction and avoidance of neurological deficit.

Impairment of speech auditory feedback error detection and motor correction in post-stroke aphasia

INTRODUCTION

The present study investigated how damage to left-hemisphere brain networks affects the ability for speech auditory feedback error detection and motor correction in post-stroke aphasia.

METHODS

34 individuals with left-hemisphere stroke and 25 neurologically intact age-matched control participants performed two randomized experimental tasks in which their online speech auditory feedback was altered using externally induced pitch-shift stimuli: 1) vocalization of a steady speech vowel sound /a/, and 2) listening to the playback of the same self-produced vowel vocalizations. Randomized control condition trials were interleaved in between vocalization and listening tasks where no pitch-shift stimuli were delivered. Following each trial, participants pressed a button to indicate whether they detected a pitch-shift error in their speech auditory feedback during vocalization and listening tasks.

RESULTS

Our data analysis revealed that speech auditory feedback error detection accuracy rate was significantly lower in the stroke compared with control participants, irrespective of the experimental task (i.e. vocalization vs. listening) and trial condition (i.e. pitch-shifted vs. no-pitch-shift). We found that this effect was associated with the reduced magnitude of speech compensation in the early phase of responses at 150-200 ms following the onset of pitch-shift stimuli in stroke participants. In addition, motor speech compensation deficit in the stroke group was correlated with lower scores on speech repetition tasks as an index of language impairment resulting from aphasia.

CONCLUSIONS

These findings provide evidence that left-hemisphere stroke is associated with impaired speech auditory feedback error processing, and such deficits account for specific aspects of language impairment in aphasia.

Processing of Degraded Speech in Brain Disorders

The speech we hear every day is typically "degraded" by competing sounds and the idiosyncratic vocal characteristics of individual speakers. While the comprehension of "degraded" speech is normally automatic, it depends on dynamic and adaptive processing across distributed neural networks. This presents the brain with an immense computational challenge, making degraded speech processing vulnerable to a range of brain disorders. Therefore, it is likely to be a sensitive marker of neural circuit dysfunction and an index of retained neural plasticity. Considering experimental methods for studying degraded speech and factors that affect its processing in healthy individuals, we review the evidence for altered degraded speech processing in major neurodegenerative diseases, traumatic brain injury and stroke. We develop a predictive coding framework for understanding deficits of degraded speech processing in these disorders, focussing on the "language-led dementias"-the primary progressive aphasias. We conclude by considering prospects for using degraded speech as a probe of language network pathophysiology, a diagnostic tool and a target for therapeutic intervention.

Speech timing changes accompany speech entrainment in aphasia

BACKGROUND

Prior speech entrainment studies, where individuals with non-fluent aphasia mimic an audio-visual model, suggest speech entrainment improves speech fluency, as indexed by various linguistic measures (e.g., the total number of different words produced per minute). Here, more precise speech timing adjustments accompanying entrained speech were studied and compared to spontaneous speech to determine how these temporal variables relate to the fluency inducing effects of speech entrainment in aphasia.

METHODS

Thirty-one left hemisphere stroke survivors classified with fluent or non-fluent speech were audio-video recorded as they described a picture and during speech entrainment. Speech fluency was documented using the Western Aphasia Battery-Revised. Acoustic measures of speech timing included total number of syllables, speech rate, articulatory rate, silent pause frequency and duration. Standard descriptive statistics and a two-factor mixed model analysis of variance were used to investigate group, task, and 'group x task' interaction effects.

FINDINGS

All acoustic measures of speech timing differentiated the fluent and nonfluent groups except for silent pause frequency. Differences between speech entrainment and spontaneous speech were found for most acoustic measures of speech timing and speaker groups, yet the direction of the effect varied. Stroke survivors classified with non-fluent aphasia improved speech fluency such that speech entrainment elicited pause adjustments facilitating more typical speech timing in comparison to spontaneous speech.

CONCLUSION

Overall, findings provide further evidence of the impact of speech entrainment on measures of speech timing to help individuals with non-fluent aphasia to practice speaking more fluently. Practicing speaking more fluently may ultimately impact perceptual judgments of speech naturalness and social acceptance for persons with aphasia.

Efference copy in kinesthetic perception: a copy of what is it?

A number of notions in the fields of motor control and kinesthetic perception have been used without clear definitions. In this review, we consider definitions for efference copy, percept, and sense of effort based on recent studies within the physical approach, which assumes that the neural control of movement is based on principles of parametric control and involves defining time-varying profiles of spatial referent coordinates for the effectors. The apparent redundancy in both motor and perceptual processes is reconsidered based on the principle of abundance. Abundance of efferent and afferent signals is viewed as the means of stabilizing both salient action characteristics and salient percepts formalized as stable manifolds in high-dimensional spaces of relevant elemental variables. This theoretical scheme has led recently to a number of novel predictions and findings. These include, in particular, lower accuracy in perception of variables produced by elements involved in a multielement task compared with the same elements in single-element tasks, dissociation between motor and perceptual effects of muscle coactivation, force illusions induced by muscle vibration, and errors in perception of unintentional drifts in performance. Taken together, these results suggest that participation of efferent signals in perception frequently involves distorted copies of actual neural commands, particularly those to antagonist muscles. Sense of effort is associated with such distorted efferent signals. Distortions in efference copy happen spontaneously and can also be caused by changes in sensory signals, e.g., those produced by muscle vibration.

Distortions of the Efferent Copy during Force Perception: A Study of Force Drifts and Effects of Muscle Vibration

We used a finger force matching task to explore the role of efferent signals in force perception. Healthy, young participants performed accurate force production tasks at different force levels with the index and middle fingers of one hand (task-hand). They received visual feedback during an early part of each trial only. After the feedback was turned off, the force drifted toward lower magnitudes. After 5 s of the drift, the participants matched the force with the same finger pair of the other hand (match-hand). The match-hand consistently overshot the task-hand force by a magnitude invariant over the initial force levels. During force matching, both hands were lifted and lowered smoothly to estimate their referent coordinate (RC) and apparent stiffness values. These trials were performed without muscle vibration and under vibration applied to the finger/hand flexors or extensors of the task-hand or match-hand. Effects of vibration were seen in the match-hand only; they were the same during vibration of flexors and extensors. We interpret the vibration-induced effects as consequences of using distorted copies of the central commands to the task-hand during force matching. In particular, using distorted copies of the RC for the antagonist muscle group could account for the differences between the task-hand and match-hand. We conclude that efferent signals may be distorted before their participation in the perceptual process. Such distortions emerge spontaneously and may be amplified by the response of sensory endings to muscle vibration combined over both agonist and antagonist muscle groups.

Error in the Superior Temporal Gyrus? A Systematic Review and Activation Likelihood Estimation Meta-Analysis of Speech Production Studies

Evidence for perceptual processing in models of speech production is often drawn from investigations in which the sound of a talker's voice is altered in real time to induce "errors." Methods of acoustic manipulation vary but are assumed to engage the same neural network and psychological processes. This paper aims to review fMRI and PET studies of altered auditory feedback and assess the strength of the evidence these studies provide for a speech error correction mechanism. Studies included were functional neuroimaging studies of speech production in neurotypical adult humans, using natural speech errors or one of three predefined speech manipulation techniques (frequency altered feedback, delayed auditory feedback, and masked auditory feedback). Seventeen studies met the inclusion criteria. In a systematic review, we evaluated whether each study (1) used an ecologically valid speech production task, (2) controlled for auditory activation caused by hearing the perturbation, (3) statistically controlled for multiple comparisons, and (4) measured behavioral compensation correlating with perturbation. None of the studies met all four criteria. We then conducted an activation likelihood estimation meta-analysis of brain coordinates from 16 studies that reported brain responses to manipulated over unmanipulated speech feedback, using the GingerALE toolbox. These foci clustered in bilateral superior temporal gyri, anterior to cortical fields typically linked to error correction. Within the limits of our analysis, we conclude that existing neuroimaging evidence is insufficient to determine whether error monitoring occurs in the posterior superior temporal gyrus regions proposed by models of speech production.

Real-time feedback control of voice in cochlear implant recipients

OBJECTIVES

To evaluate feedback-dependent vocal control in cochlear implant patients using pitch-shifted auditory feedback.

METHODS

Twenty-three CI recipients with at least 6 months of implant experience were enrolled. Vocal recordings were performed while subjects repeated the vowel /e/ and vocal signals were altered in real-time using a digital effects processor to introduce a pitch-shift, presented back to subjects using headphones. Recordings were analyzed to determine pitch changes following the pitch-shifted feedback, and results compared to the magnitude of the shift as well as patient demographics.

RESULTS

Consistent with previous results, CI patients' voices had higher pitches with their implant turned off, a change explainable by increases in vocal loudness without the CI. CI patients rapidly compensated for pitch-shifted feedback by changing their vocal pitch, but only for larger shifts. Considerable inter-subject variability was present, and weakly correlated with the duration of implant experience and implant sound thresholds.

CONCLUSIONS

CI patients, like normal hearing individuals, are capable of real-time feedback-dependent control of their vocal pitch. However, CI patients are less sensitive to small feedback changes, possibly a result of courser CI frequency precision, and may explain poorer than normal vocal control in these patients.

LEVEL OF EVIDENCE

Level 3b.

Learning Speech Production and Perception through Sensorimotor Interactions

Action and perception are closely linked in many behaviors necessitating a close coordination between sensory and motor neural processes so as to achieve a well-integrated smoothly evolving task performance. To investigate the detailed nature of these sensorimotor interactions, and their role in learning and executing the skilled motor task of speaking, we analyzed ECoG recordings of responses in the high-γ band (70-150 Hz) in human subjects while they listened to, spoke, or silently articulated speech. We found elaborate spectrotemporally modulated neural activity projecting in both "forward" (motor-to-sensory) and "inverse" directions between the higher-auditory and motor cortical regions engaged during speaking. Furthermore, mathematical simulations demonstrate a key role for the forward projection in "learning" to control the vocal tract, beyond its commonly postulated predictive role during execution. These results therefore offer a broader view of the functional role of the ubiquitous forward projection as an important ingredient in learning, rather than just control, of skilled sensorimotor tasks.

Nonverbal auditory communication - Evidence for integrated neural systems for voice signal production and perception

While humans have developed a sophisticated and unique system of verbal auditory communication, they also share a more common and evolutionarily important nonverbal channel of voice signaling with many other mammalian and vertebrate species. This nonverbal communication is mediated and modulated by the acoustic properties of a voice signal, and is a powerful - yet often neglected - means of sending and perceiving socially relevant information. From the viewpoint of dyadic (involving a sender and a signal receiver) voice signal communication, we discuss the integrated neural dynamics in primate nonverbal voice signal production and perception. Most previous neurobiological models of voice communication modelled these neural dynamics from the limited perspective of either voice production or perception, largely disregarding the neural and cognitive commonalities of both functions. Taking a dyadic perspective on nonverbal communication, however, it turns out that the neural systems for voice production and perception are surprisingly similar. Based on the interdependence of both production and perception functions in communication, we first propose a re-grouping of the neural mechanisms of communication into auditory, limbic, and paramotor systems, with special consideration for a subsidiary basal-ganglia-centered system. Second, we propose that the similarity in the neural systems involved in voice signal production and perception is the result of the co-evolution of nonverbal voice production and perception systems promoted by their strong interdependence in dyadic interactions.

Auditory Feedback Control of Vocal Pitch in Spasmodic Dysphonia

OBJECTIVES/HYPOTHESIS

Hearing plays an important role in the maintenance of vocal control in normal individuals. In patients with spasmodic dysphonia (SD), however, the ability to maintain sustained control of phonation is impaired. The origins of SD are unknown, and it is unclear whether auditory feedback-dependent vocal control is compromised in these patients.

STUDY DESIGN

Prospective case-control study.

METHODS

We tested 15 SD patients and 11 age-matched controls. Voice recordings were performed while subjects repeated the vowel /e/ and auditory feedback of their vocal sounds was altered in real-time to introduce a pitch-shift (±2 semitones), presented back to subjects using headphones. Recordings were analyzed to determine voice changes following the pitch-shifted feedback. Results were further compared with patient demographics and subjective measures of dysphonia, including the Voice Handicap Index (VHI).

RESULTS

Despite considerable pitch variability and vocal breaks, SD patients exhibited significantly higher average vocal pitch compensation than control subjects. SD patients also exhibited greater variability than controls. However, there were no significant correlations between vocal compensation and patient demographics, although there was a significant inverse correlation with VHI.

CONCLUSIONS

In this pilot study, patients with SD exhibited increased sensitivity to altered auditory feedback during sustained phonation. These results are consistent with recent theories of SD as a disorder of sensory-motor feedback processing, and suggest possible avenues for future investigation.

LEVEL OF EVIDENCE

3 Laryngoscope, 131:2070-2075, 2021.

Compensation to real-time temporal auditory feedback perturbation depends on syllable position

Auditory feedback perturbations involving spectral shifts indicated a crucial contribution of auditory feedback to planning and execution of speech. However, much less is known about the contribution of auditory feedback with respect to temporal properties of speech. The current study aimed at providing insight into the representation of temporal properties of speech and the relevance of auditory feedback for speech timing. Real-time auditory feedback perturbations were applied in the temporal domain, viz., stretching and compressing of consonant-consonant-vowel (CCV) durations in onset + nucleus vs vowel-consonant-consonant (VCC) durations in nucleus + coda. Since CCV forms a gesturally more cohesive and stable structure than VCC, greater articulatory adjustments to nucleus + coda (VCC) perturbation were expected. The results show that speakers compensate for focal temporal feedback alterations. Responses to VCC perturbation were greater than to CCV perturbation, suggesting less deformability of onsets when confronted with temporally perturbed auditory feedback. Further, responses to CCV perturbation rather reflected within-trial reactive compensation, whereas VCC compensation was more pronounced and indicative of adaptive behavior. Accordingly, planning and execution of temporal properties of speech are indeed guided by auditory feedback, but the precise nature of the reaction to perturbations is linked to the structural position in the syllable and the associated feedforward timing strategies.

Auditory representation of learned sound sequences in motor regions of the macaque brain

Human speech production requires the ability to couple motor actions with their auditory consequences. Nonhuman primates might not have speech because they lack this ability. To address this question, we trained macaques to perform an auditory-motor task producing sound sequences via hand presses on a newly designed device ("monkey piano"). Catch trials were interspersed to ascertain the monkeys were listening to the sounds they produced. Functional MRI was then used to map brain activity while the animals listened attentively to the sound sequences they had learned to produce and to two control sequences, which were either completely unfamiliar or familiar through passive exposure only. All sounds activated auditory midbrain and cortex, but listening to the sequences that were learned by self-production additionally activated the putamen and the hand and arm regions of motor cortex. These results indicate that, in principle, monkeys are capable of forming internal models linking sound perception and production in motor regions of the brain, so this ability is not special to speech in humans. However, the coupling of sounds and actions in nonhuman primates (and the availability of an internal model supporting it) seems not to extend to the upper vocal tract, that is, the supralaryngeal articulators, which are key for the production of speech sounds in humans. The origin of speech may have required the evolution of a "command apparatus" similar to the control of the hand, which was crucial for the evolution of tool use.

Modulation of vocal pitch control through high-definition transcranial direct current stimulation of the left ventral motor cortex

Neural interactions between sensorimotor integration mechanisms play critical roles in voice motor control. We investigated how high-definition transcranial direct current stimulation (HD-tDCS) of the left ventral motor cortex modulates neural mechanisms of sensorimotor integration during voice motor control. HD-tDCS was performed during speech vowel production in an altered auditory feedback (AAF) paradigm in response to upward and downward pitch-shift stimuli. In one experiment, two groups received either anodal or cathodal 2 milliamp (mA) HD-tDCS to the left ventral motor cortex while a third group received sham (placebo) stimulation. In a second experiment, two groups received either 1 mA or 2 mA cathodal HD-tDCS to the left ventral motor cortex. Results of the first experiment indicated that the magnitude of vocal compensation was significantly reduced following anodal and cathodal HD-tDCS only in responses to downward pitch-shift AAF stimuli, with stronger effects associated with cathodal HD-tDCS. However, no such effect was observed following sham stimulation. Results of the second experiment indicate that there is not a differential effect of modulation from 1 mA versus 2 mA. Further, these results replicate the directional finding of the first experiment for vocal compensation in response to downward pitch-shift only. These findings suggest that neurostimulation of the left ventral motor cortex modulates sensorimotor mechanisms underlying voice motor control. We speculate that this effect is associated with the increased contribution of feedforward motor mechanisms, leading to reduced compensatory speech responses to AAF.

Dissociation of Unit Activity and Gamma Oscillations during Vocalization in Primate Auditory Cortex

Vocal production is a sensory-motor process in which auditory self-monitoring is used to ensure accurate communication. During vocal production, the auditory cortex of both humans and animals is suppressed, a phenomenon that plays an important role in self-monitoring and vocal motor control. However, the underlying neural mechanisms of this vocalization-induced suppression are unknown. γ-band oscillations (>25 Hz) have been implicated a variety of cortical functions and are thought to arise from activity of local inhibitory interneurons, but have not been studied during vocal production. We therefore examined γ-band activity in the auditory cortex of vocalizing marmoset monkeys, of either sex, and found that γ responses increased during vocal production. This increase in γ contrasts with simultaneously recorded suppression of single-unit and multiunit responses. Recorded vocal γ oscillations exhibited two separable components: a vocalization-specific nonsynchronized ("induced") response correlating with vocal suppression, and a synchronized ("evoked") response that was also present during passive sound playback. These results provide evidence for the role of cortical γ oscillations during inhibitory processing. Furthermore, the two distinct components of the γ response suggest possible mechanisms for vocalization-induced suppression, and may correspond to the sensory-motor integration of top-down and bottom-up inputs to the auditory cortex during vocal production.SIGNIFICANCE STATEMENT Vocal communication is important to both humans and animals. In order to ensure accurate information transmission, we must monitor our own vocal output. Surprisingly, spiking activity in the auditory cortex is suppressed during vocal production yet maintains sensitivity to the sound of our own voice ("feedback"). The mechanisms of this vocalization-induced suppression are unknown. Here we show that auditory cortical γ oscillations, which reflect interneuron activity, are actually increased during vocal production, the opposite response of that seen in spiking units. We discuss these results with proposed functions of γ activity during inhibitory sensory processing and coordination of different brain regions, suggesting a role in sensory-motor integration.

Event-related potential correlates of auditory feedback control of vocal production in experienced singers

Considerable evidence has shown that experienced singers are capable of voluntarily suppressing vocal compensations for consistent pitch perturbations in auditory feedback. Our recent behavioral study found that singers also compensated for brief pitch perturbations to a lesser degree than nonsingers in an involuntary manner. In the present event-related potential study, we investigated the neural correlates of involuntary vocal pitch regulation in experienced singers. All participants were instructed to vocalize the vowel sounds while their voice was unexpectedly shifted in pitch by -50 and -200 cents. The results revealed decreased cortical N1 and P2 responses to pitch perturbations and reduced involuntary vocal compensations for singers when compared to nonsingers. Moreover, larger vocal responses were significantly correlated with smaller cortical P2 responses for nonsingers, whereas this brain-behavior relationship did not exist for singers. These findings demonstrate that the cortical processing of involuntary auditory-motor integration for vocal pitch regulation can be shaped as a function of singing experience, suggesting that experienced singers may be less influenced by auditory feedback and rely more on somatosensory feedback or feedforward control as a consequence of singing training as compared to nonsingers.

Limited Pre-Speech Auditory Modulation in Individuals Who Stutter: Data and Hypotheses

Purpose We review and interpret our recent series of studies investigating motor-to-auditory influences during speech movement planning in fluent speakers and speakers who stutter. In those studies, we recorded auditory evoked potentials in response to probe tones presented immediately prior to speaking or at the equivalent time in no-speaking control conditions. As a measure of pre-speech auditory modulation (PSAM), we calculated changes in auditory evoked potential amplitude in the speaking conditions relative to the no-speaking conditions. Whereas adults who do not stutter consistently showed PSAM, this phenomenon was greatly reduced or absent in adults who stutter. The same between-group difference was observed in conditions where participants expected to hear their prerecorded speech played back without actively producing it, suggesting that the speakers who stutter use inefficient forward modeling processes rather than inefficient motor command generation processes. Compared with fluent participants, adults who stutter showed both less PSAM and less auditory-motor adaptation when producing speech while exposed to formant-shifted auditory feedback. Across individual participants, however, PSAM and auditory-motor adaptation did not correlate in the typically fluent group, and they were negatively correlated in the stuttering group. Interestingly, speaking with a consistent 100-ms delay added to the auditory feedback signal-normalized PSAM in speakers who stutter, and there no longer was a between-group difference in this condition. Conclusions Combining our own data with human and animal neurophysiological evidence from other laboratories, we interpret the overall findings as suggesting that (a) speech movement planning modulates auditory processing in a manner that may optimize its tuning characteristics for monitoring feedback during speech production and, (b) in conditions with typical auditory feedback, adults who stutter do not appropriately modulate the auditory system prior to speech onset. Lack of modulation of speakers who stutter may lead to maladaptive feedback-driven movement corrections that manifest themselves as repetitive movements or postural fixations.

Modeling the Role of Sensory Feedback in Speech Motor Control and Learning

Purpose While the speech motor system is sensitive to feedback perturbations, sensory feedback does not seem to be critical to speech motor production. How the speech motor system is able to be so flexible in its use of sensory feedback remains an open question. Method We draw on evidence from a variety of disciplines to summarize current understanding of the sensory systems' role in speech motor control, including both online control and motor learning. We focus particularly on computational models of speech motor control that incorporate sensory feedback, as these models provide clear encapsulations of different theories of sensory systems' function in speech production. These computational models include the well-established directions into velocities of articulators model and computational models that we have been developing in our labs based on the domain-general theory of state feedback control (feedback aware control of tasks in speech model). Results After establishing the architecture of the models, we show that both the directions into velocities of articulators and state feedback control/feedback aware control of tasks models can replicate key behaviors related to sensory feedback in the speech motor system. Although the models agree on many points, the underlying architecture of the 2 models differs in a few key ways, leading to different predictions in certain areas. We cover key disagreements between the models to show the limits of our current understanding and point toward areas where future experimental studies can resolve these questions. Conclusions Understanding the role of sensory information in the speech motor system is critical to understanding speech motor production and sensorimotor learning in healthy speakers as well as in disordered populations. Computational models, with their concrete implementations and testable predictions, are an important tool to understand this process. Comparison of different models can highlight areas of agreement and disagreement in the field and point toward future experiments to resolve important outstanding questions about the speech motor control system.

Decreased Gray-Matter Volume in Insular Cortex as a Correlate of Singers' Enhanced Sensorimotor Control of Vocal Production

Accumulating evidence has shown enhanced sensorimotor control of vocal production as a consequence of extensive singing experience. The neural basis of this ability, however, is poorly understood. Given that the insula mediates motor aspects of vocal production, the present study investigated structural plasticity in insula induced by singing experience and its link to auditory feedback control of vocal production. Voxel-based morphometry (VBM) was used to examine the differences in gray matter (GM) volume in the insula of 21 singers and 21 non-singers. An auditory feedback perturbation paradigm was used to examine the differences in auditory-motor control of vocal production between singers and non-singers. Both groups vocalized sustained vowels while hearing their voice unexpectedly pitch-shifted −50 or −200 cents (200 ms duration). VBM analyses showed that singers exhibited significantly lower GM volumes in the bilateral insula than non-singers. When exposed to pitch perturbations in voice auditory feedback, singers involuntarily compensated for pitch perturbations in voice auditory feedback to a significantly lesser degree than non-singers. Moreover, across the two sizes of pitch perturbations, the magnitudes of vocal compensations were positively correlated with the total regional GM volumes in the bilateral insula. These results indicate that extensive singing training leads to decreased GM volumes in insula and suggest that morphometric plasticity in insula contributes to the enhanced sensorimotor control of vocal production observed in singers.

Neural Basis of Sensorimotor Plasticity in Speech Motor Adaptation

When we speak, we get correlated sensory feedback from speech sounds and from the muscles and soft tissues of the vocal tract. Here we dissociate the contributions of auditory and somatosensory feedback to identify brain networks that underlie the somatic contribution to speech motor learning. The technique uses a robotic device that selectively alters somatosensory inputs in combination with resting-state fMRI scans that reveal learning-related changes in functional connectivity. A partial correlation analysis is used to identify connectivity changes that are not explained by the time course of activity in any other learning-related areas. This analysis revealed changes related to behavioral improvements in movement and separately, to changes in auditory perception: Speech motor adaptation itself was associated with connectivity changes that were primarily in non-motor areas of brain, specifically, to a strengthening of connectivity between auditory and somatosensory cortex and between presupplementary motor area and the inferior parietal lobule. In contrast, connectively changes associated with alterations to auditory perception were restricted to speech motor areas, specifically, primary motor cortex and inferior frontal gyrus. Overall, our findings show that during adaptation, somatosensory inputs result in a broad range of changes in connectivity in areas associated with speech motor control and learning.

The use of intracranial recordings to decode human language: Challenges and opportunities

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production - such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components - using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

Speaking-Induced Suppression of the Auditory Cortex in Humans and Its Relevance to Schizophrenia

Speaking-induced suppression (SIS) is the phenomenon that the sounds one generates by overt speech elicit a smaller neurophysiological response in the auditory cortex than comparable sounds that are externally generated. SIS is a specific example of the more general phenomenon of self-suppression. SIS has been well established in nonhuman animals and is believed to involve the action of corollary discharges. This review summarizes, first, the evidence for SIS in heathy human participants, where it has been most commonly assessed with electroencephalography and/or magnetoencephalography using an experimental paradigm known as "Talk-Listen"; and second, the growing number of Talk-Listen studies that have reported subnormal levels of SIS in patients with schizophrenia. This result is theoretically significant, as it provides a plausible explanation for some of the most distinctive and characteristic symptoms of schizophrenia, namely the first-rank symptoms. In particular, while the failure to suppress the neural consequences of self-generated movements (such as those associated with overt speech) provides a prima facie explanation for delusions of control, the failure to suppress the neural consequences of self-generated inner speech provides a plausible explanation for certain classes of auditory-verbal hallucinations, such as audible thoughts. While the empirical evidence for a relationship between SIS and the first-rank symptoms is currently limited, I predict that future studies with more sensitive experimental designs will confirm its existence. Establishing the existence of a causal, mechanistic relationship would represent a major step forward in our understanding of schizophrenia, which is a necessary precursor to the development of novel treatments.

Intermittent theta burst stimulation over right somatosensory larynx cortex enhances vocal pitch-regulation in nonsingers

While the significance of auditory cortical regions for the development and maintenance of speech motor coordination is well established, the contribution of somatosensory brain areas to learned vocalizations such as singing is less well understood. To address these mechanisms, we applied intermittent theta burst stimulation (iTBS), a facilitatory repetitive transcranial magnetic stimulation (rTMS) protocol, over right somatosensory larynx cortex (S1) and a nonvocal dorsal S1 control area in participants without singing experience. A pitch-matching singing task was performed before and after iTBS to assess corresponding effects on vocal pitch regulation. When participants could monitor auditory feedback from their own voice during singing (Experiment I), no difference in pitch-matching performance was found between iTBS sessions. However, when auditory feedback was masked with noise (Experiment II), only larynx-S1 iTBS enhanced pitch accuracy (50-250 ms after sound onset) and pitch stability (>250 ms after sound onset until the end). Results indicate that somatosensory feedback plays a dominant role in vocal pitch regulation when acoustic feedback is masked. The acoustic changes moreover suggest that right larynx-S1 stimulation affected the preparation and involuntary regulation of vocal pitch accuracy, and that kinesthetic-proprioceptive processes play a role in the voluntary control of pitch stability in nonsingers. Together, these data provide evidence for a causal involvement of right larynx-S1 in vocal pitch regulation during singing.

Neural correlates of abnormal auditory feedback processing during speech production in Alzheimer's disease

Accurate integration of sensory inputs and motor commands is essential to achieve successful behavioral goals. A robust model of sensorimotor integration is the pitch perturbation response, in which speakers respond rapidly to shifts of the pitch in their auditory feedback. In a previous study, we demonstrated abnormal sensorimotor integration in patients with Alzheimer's disease (AD) with an abnormally enhanced behavioral response to pitch perturbation. Here we examine the neural correlates of the abnormal pitch perturbation response in AD patients, using magnetoencephalographic imaging. The participants phonated the vowel /α/ while a real-time signal processor briefly perturbed the pitch (100 cents, 400 ms) of their auditory feedback. We examined the high-gamma band (65-150 Hz) responses during this task. AD patients showed significantly reduced left prefrontal activity during the early phase of perturbation and increased right middle temporal activity during the later phase of perturbation, compared to controls. Activity in these brain regions significantly correlated with the behavioral response. These results demonstrate that impaired prefrontal modulation of speech-motor-control network and additional recruitment of right temporal regions are significant mediators of aberrant sensorimotor integration in patients with AD. The abnormal neural integration mechanisms signify the contribution of cortical network dysfunction to cognitive and behavioral deficits in AD.

Production Variability and Categorical Perception of Vowels Are Strongly Linked

Theoretical models of speech production suggest that the speech motor system (SMS) uses auditory goals to determine errors in its auditory output during vowel production. This type of error calculation indicates that within-speaker production variability of a given vowel is related to the size of the vowel’s auditory goal. However, emerging evidence suggests that the SMS may also take into account perceptual knowledge of vowel categories (in addition to auditory goals) to estimate errors in auditory feedback. In this study, we examined how this mechanism influences within-speaker variability in vowel production. We conducted a study (n = 40 adults), consisting of a vowel categorization task and a vowel production task. The vowel categorization task was designed—based on participant-specific vowels—to estimate the categorical perceptual boundary (CPB) between two front vowels (/ε/ and /æ/). Using the vowel production data of each participant, we calculated a variability-based boundary (VBB) located at the “center of mass” of the two vowels. The inverse of the standard deviation of a vowel distribution was used as the “mass” of the vowel. We found that: (a) categorical boundary was located farther from more variable vowels; and (b) the calculated VBB (i.e., the center of mass of the vowels) significantly and positively correlated with the estimated categorical boundary (r = 0.912 for formants calculated in hertz; r = 0.854 for formants calculated in bark). Overall, our findings support a view that vowel production and vowel perception are strongly and bidirectionally linked.