Auditory-motor adaptation to frequency-altered auditory feedback occurs when participants ignore feedback

Speech-induced suppression and vocal feedback sensitivity in human cortex

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.

Neurophysiological evidence of sensory prediction errors driving speech sensorimotor adaptation

The human sensorimotor system has a remarkable ability to quickly and efficiently learn movements from sensory experience. A prominent example is sensorimotor adaptation, learning that characterizes the sensorimotor system's response to persistent sensory errors by adjusting future movements to compensate for those errors. Despite being essential for maintaining and fine-tuning motor control, mechanisms underlying sensorimotor adaptation remain unclear. A component of sensorimotor adaptation is implicit (i.e., the learner is unaware of the learning process) which has been suggested to result from sensory prediction errors-the discrepancies between predicted sensory consequences of motor commands and actual sensory feedback. However, to date no direct neurophysiological evidence that sensory prediction errors drive adaptation has been demonstrated. Here, we examined prediction errors via magnetoencephalography (MEG) imaging of the auditory cortex during sensorimotor adaptation of speech to altered auditory feedback, an entirely implicit adaptation task. Specifically, we measured how speaking-induced suppression (SIS)--a neural representation of auditory prediction errors--changed over the trials of the adaptation experiment. SIS refers to the suppression of auditory cortical response to speech onset (in particular, the M100 response) to self-produced speech when compared to the response to passive listening to identical playback of that speech. SIS was reduced (reflecting larger prediction errors) during the early learning phase compared to the initial unaltered feedback phase. Furthermore, reduction in SIS positively correlated with behavioral adaptation extents, suggesting that larger prediction errors were associated with more learning. In contrast, such a reduction in SIS was not found in a control experiment in which participants heard unaltered feedback and thus did not adapt. In addition, in some participants who reached a plateau in the late learning phase, SIS increased (reflecting smaller prediction errors), demonstrating that prediction errors were minimal when there was no further adaptation. Together, these findings provide the first neurophysiological evidence for the hypothesis that prediction errors drive human sensorimotor adaptation.

Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors-discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Identifying distinct latent classes of pitch-shift response consistency: Evidence from manipulating the predictability of shift direction

Auditory feedback plays an important role in regulating our vocal pitch. When pitch shifts suddenly appear in auditory feedback, the majority of the responses are opposing, correcting for the mismatch between perceived pitch and actual pitch. However, research has indicated that following responses to auditory perturbation could be common. This study attempts to explore the ways individual speakers would respond to pitch perturbation (using an opposing response or a following response) from trial to trial. Thirty-six native speakers of Mandarin produced the vowel /a/ while receiving perturbed pitch at a random time (500 ~ 700 ms) after vocal onset for a duration of 200 ms. Three blocks of 30 trials that differed in the pitch-shift stimulus direction were recorded in a randomized order: (a) the down-only condition where pitch was shifted downwards 250 cents; (b) the up-only condition where pitch was shifted upwards 250 cents; and (c) the random condition where downshifts and upshifts occurred randomly and were equally likely. The participants were instructed to ignore the pitch shifts. Results from the latent class analysis show that at the individual level across trials, 57% of participants were switchers, 28% were opposers, and 15% were followers. Our results support that speakers produce a mix of opposing and following responses when they respond to perturbed pitch. Specifically, the proportion of followers was conditional on the expectancy of pitch-shift stimulus direction: More followers were observed when the pitch-shift stimulus direction was predictable. Closer inspection of the levels of response consistency in different time phases shows that a particular mechanism (opposing or following) was initially implemented; the two mechanisms may alternate in the middle phase; and then finally, the pitch-shift response was featured as a particular mechanism near the end phase.

LaDIVA: A neurocomputational model providing laryngeal motor control for speech acquisition and production

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

ReActLab: A Custom Framework for Sensorimotor Experiments “in-the-wild”

Over the last few years online platforms for running psychology experiments beyond simple questionnaires and surveys have become increasingly popular. This trend has especially increased after many laboratory facilities had to temporarily avoid in-person data collection following COVID-19-related lockdown regulations. Yet, while offering a valid alternative to in-person experiments in many cases, platforms for online experiments are still not a viable solution for a large part of human-based behavioral research. Two situations in particular pose challenges: First, when the research question requires design features or participant interaction which exceed the customization capability provided by the online platform; and second, when variation among hardware characteristics between participants results in an inadmissible confounding factor. To mitigate the effects of these limitations, we developed ReActLab (Remote Action Laboratory), a framework for programming remote, browser-based experiments using freely available and open-source JavaScript libraries. Since the experiment is run entirely within the browser, our framework allows for portability to any operating system and many devices. In our case, we tested our approach by running experiments using only a specific model of Android tablet. Using ReActLab with this standardized hardware allowed us to optimize our experimental design for our research questions, as well as collect data outside of laboratory facilities without introducing setup variation among participants. In this paper, we describe our framework and show examples of two different experiments carried out with it: one consisting of a visuomotor adaptation task, the other of a visual localization task. Through comparison with results obtained from similar tasks in in-person laboratory settings, we discuss the advantages and limitations for developing browser-based experiments using our framework.

Pediatric Responses to Fundamental and Formant Frequency Altered Auditory Feedback: A Scoping Review

Purpose

The ability to hear ourselves speak has been shown to play an important role in the development and maintenance of fluent and coherent speech. Despite this, little is known about the developing speech motor control system throughout childhood, in particular if and how vocal and articulatory control may differ throughout development. A scoping review was undertaken to identify and describe the full range of studies investigating responses to frequency altered auditory feedback in pediatric populations and their contributions to our understanding of the development of auditory feedback control and sensorimotor learning in childhood and adolescence.

Method

Relevant studies were identified through a comprehensive search strategy of six academic databases for studies that included (a) real-time perturbation of frequency in auditory input, (b) an analysis of immediate effects on speech, and (c) participants aged 18 years or younger.

Results

Twenty-three articles met inclusion criteria. Across studies, there was a wide variety of designs, outcomes and measures used. Manipulations included fundamental frequency (9 studies), formant frequency (12), frequency centroid of fricatives (1), and both fundamental and formant frequencies (1). Study designs included contrasts across childhood, between children and adults, and between typical, pediatric clinical and adult populations. Measures primarily explored acoustic properties of speech responses (latency, magnitude, and variability). Some studies additionally examined the association of these acoustic responses with clinical measures (e.g., stuttering severity and reading ability), and neural measures using electrophysiology and magnetic resonance imaging.

Conclusion

Findings indicated that children above 4 years generally compensated in the opposite direction of the manipulation, however, in several cases not as effectively as adults. Overall, results varied greatly due to the broad range of manipulations and designs used, making generalization challenging. Differences found between age groups in the features of the compensatory vocal responses, latency of responses, vocal variability and perceptual abilities, suggest that maturational changes may be occurring in the speech motor control system, affecting the extent to which auditory feedback is used to modify internal sensorimotor representations. Varied findings suggest vocal control develops prior to articulatory control. Future studies with multiple outcome measures, manipulations, and more expansive age ranges are needed to elucidate findings.

Contributions of Forward-Focused Voice to Audio-Vocal Feedback Measured Using Nasal Accelerometry and Power Spectral Analysis of Vocal Fundamental Frequency

PURPOSE

The spectral powers of the modulations of vocal fundamental frequency (f _o) less than 3 Hz (low-frequency power, LFP) and between 3 and 8 Hz (middle-frequency power, MFP) had been established to indicate the audio-vocal feedback status and vocal efficiency of a speaker, and a resonant voice may enhance the auditory-vocal feedback. This study aims to determine whether the auditory feedback can be augmented by a forward and resonant voice and therefore contribute to the modulations of f _o variability.

METHOD

Vocal signals and accelerometric signals of lateral nasal cartilage were obtained from 27 healthy adults who, respectively, sustained vowels /a/ and /i/ with their habitual speaking voice and with a forward-focused voice under three auditory conditions: natural hearing (N0), high-level noise exposure (N90), and low-level noise exposure (N60). Nasal skin vibrations were measured using a nasal accelerometry to reflect voice resonance status. Vocal intensity and f _o variability were also analyzed to show the auditory-vocal interactions under varied conditions of auditory feedback and voice resonance.

RESULTS

In both N0 and N90 conditions, forward-focused voice showed a significantly lower LFP than the speakers' habitual voice. In addition, LFP of f _o would significantly increase during natural voice production as the voice feedback was greatly masked by high-intensity noise; however, with a forward-focused voice, the noise-induced variation in LFP was significantly decreased. Under N90, MFP significantly decreased during forward-focused voice production compared with that measured during natural voice production. The stability of f _o modulations was not adversely affected by N60.

CONCLUSION

The results support the idea that vocalizing with a forward-focused voice enhance the auditory feedback of the speaker's own voice and, thus, reduce the variability of f _o during sustained phonation, especially when vocalizing in the high noise condition.

The effect of stimulus timing in compensating for pitch perturbation on flat, rising, and falling contours

The purpose of this study was to explore vocal responses to pitch perturbation on the flat, rising, and falling contour made of sequences of level tones in Taiwanese Southern Min. Twenty-two native speakers produced nine disyllabic words (flat: high-high, mid-mid, and low-low tone sequences; rising: mid-high, low-high, and low-mid tone sequences; falling: high-mid, high-low, and mid-low tone sequences). Pitch-shift stimuli (200 ms) appeared at either 100 ms (the beginning of the first syllable) or 400 ms (the beginning of the second syllable) after vocal onset. The participants were asked to ignore the pitch perturbation that appeared via auditory feedback. We found their compensation decreased when both syllables had identical level tones (i.e., the flat contour) but was particularly large when the overall contour was falling. Furthermore, pitch compensation at 100 ms was smaller than at 400 ms for the falling contour, but not for the flat and rising contours. Our results suggest that less susceptibility to pitch perturbation in the initial speech planning process is conditioned by the velocity of overall pitch contour.

Reliability and Efficiency of Pitch-Shifting Plug-Ins in Voice and Hearing Research

PURPOSE

Auditory feedback perturbation with voice pitch manipulation has been widely used in previous studies. There are several hardware and software tools for such manipulations, but audio plug-ins developed for music, movies, and radio applications that operate in digital audio workstations may be extremely beneficial and are easy to use, accessible, and cost effective. However, it is unknown whether these plug-ins can perform similarly to tools that have been described in previous literature. Hence, this study aimed to evaluate the reliability and efficiency of these plug-ins.

METHOD

Six different plug-ins were used at +1 and -1 st pitch shifting with formant correction on and off to pitch shift the sustained /ɑ/ voice recording sample of 12 healthy participants (six cisgender males and six cisgender females). Pitch-shifting accuracy, formant shifting amount, intensity changes, and total latency values were reported.

RESULTS

Some variability was observed between different plug-ins and pitch shift settings. One plug-in managed to perform similarly in all four measured aspects with well-known hardware and software units with 1-cent pitch-shifting accuracy, low latency values, negligible intensity difference, and preserved formants. Other plug-ins performed similarly in some respects.

CONCLUSIONS

Audio plug-ins may be used effectively in pitch-shifting applications. Researchers and clinicians can access these plug-ins easily and test whether the features also fit their aims.

EXPRESS: Don't blame yourself: Conscious source monitoring modulates feedback control during speech production

Sensory feedback plays an important role in speech motor control. One of the main sources of evidence for this are studies where online auditory feedback is perturbed during ongoing speech. In motor control, it is therefore crucial to distinguish between sensory feedback and externally generated sensory events. This is called source monitoring. Previous altered feedback studies have taken non-conscious source monitoring for granted, as automatic responses to altered sensory feedback imply that the feedback changes are processed as self-caused. However, the role of conscious source monitoring is unclear. The current study investigated whether conscious source monitoring modulates responses to unexpected pitch changes in auditory feedback. During a first block, some participants spontaneously attributed the pitch shifts to themselves (self-blamers) while others attributed them to an external source (other-blamers). Before block 2, all participants were informed that the pitch shifts were experimentally induced. The self-blamers then showed a reduction in response magnitude in block 2 compared with block 1, while the other-blamers did not. This suggests that conscious source monitoring modulates responses to altered auditory feedback, such that consciously ascribing feedback to oneself leads to larger compensation responses. These results can be accounted for within the dominant comparator framework, where conscious source monitoring could modulate the gain on sensory feedback. Alternatively, the results can be naturally explained from an inferential framework, where conscious knowledge may bias the priors in a Bayesian process to determine the most likely source of a sensory event.

Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning

The neural mechanisms underlying stuttering remain poorly understood. A large body of work has focused on sensorimotor integration difficulties in individuals who stutter, including recently the capacity for sensorimotor learning. Typically, sensorimotor learning is assessed with adaptation paradigms in which one or more sensory feedback modalities are experimentally perturbed in real time. Our own previous work on speech with perturbed auditory feedback revealed substantial auditory-motor learning limitations in both children and adults who stutter (AWS). It remains unknown, however, which subprocesses of sensorimotor learning are impaired. Indeed, new insights from research on upper limb motor control indicate that sensorimotor learning involves at least two distinct components: (a) an explicit component that includes intentional strategy use and presumably is driven by target error and (b) an implicit component that updates an internal model without awareness of the learner and presumably is driven by sensory prediction error. Here, we attempted to dissociate these components for speech auditory-motor learning in AWS versus adults who do not stutter (AWNS). Our formant-shift auditory-motor adaptation results replicated previous findings that such sensorimotor learning is limited in AWS. Novel findings are that neither control nor stuttering participants reported any awareness of changing their productions in response to the auditory perturbation and that neither group showed systematic drift in auditory target judgments made throughout the adaptation task. These results indicate that speech auditory-motor adaptation to formant-shifted feedback relies exclusively on implicit learning processes. Thus, limited adaptation in AWS reflects poor implicit sensorimotor learning. Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning.

Speech compensation responses and sensorimotor adaptation to formant feedback perturbations

Control of speech formants is important for the production of distinguishable speech sounds and is achieved with both feedback and learned feedforward control. However, it is unclear whether the learning of feedforward control involves the mechanisms of feedback control. Speakers have been shown to compensate for unpredictable transient mid-utterance perturbations of pitch and loudness feedback, demonstrating online feedback control of these speech features. To determine whether similar feedback control mechanisms exist in the production of formants, responses to unpredictable vowel formant feedback perturbations were examined. Results showed similar within-trial compensatory responses to formant perturbations that were presented at utterance onset and mid-utterance. The relationship between online feedback compensation to unpredictable formant perturbations and sensorimotor adaptation to consistent formant perturbations was further examined. Within-trial online compensation responses were not correlated with across-trial sensorimotor adaptation. A detailed analysis of within-trial time course dynamics across trials during sensorimotor adaptation revealed that across-trial sensorimotor adaptation responses did not result from an incorporation of within-trial compensation response. These findings suggest that online feedback compensation and sensorimotor adaptation are governed by distinct neural mechanisms. These findings have important implications for models of speech motor control in terms of how feedback and feedforward control mechanisms are implemented.

Changes in sensorimotor regions of the cerebral cortex in congenital amusia: a case-control study

Perceiving pitch is a central function of the human auditory system; congenital amusia is a disorder of pitch perception. The underlying neural mechanisms of congenital amusia have been actively discussed. However, little attention has been paid to the changes in the motor rain within congenital amusia. In this case-control study, 17 participants with congenital amusia and 14 healthy controls underwent functional magnetic resonance imaging while resting with their eyes closed. A voxel-based degree centrality method was used to identify abnormal functional network centrality by comparing degree centrality values between the congenital amusia group and the healthy control group. We found decreased degree centrality values in the right primary sensorimotor areas in participants with congenital amusia relative to controls, indicating potentially decreased centrality of the corresponding brain regions in the auditory-sensory motor feedback network. We found a significant positive correlation between the degree centrality values and the Montreal Battery of Evaluation of Amusia scores. In conclusion, our study identified novel, hitherto undiscussed candidate brain regions that may partly contribute to or be modulated by congenital amusia. Our evidence supports the view that sensorimotor coupling plays an important role in memory and musical discrimination. The study was approved by the Ethics Committee of the Second Xiangya Hospital, Central South University, China (No. WDX20180101GZ01) on February 9, 2019.

Slowed Compensation Responses to Altered Auditory Feedback in Post-Stroke Aphasia: Implications for Speech Sensorimotor Integration

Developing a clearer understanding of impairments that underlie the behavioral characteristics of aphasia is essential for the development of targeted treatments and will help inform theories of speech motor control. Impairments in sensorimotor integration of speech in individuals with conduction aphasia have previously been implicated in their repetition deficits. However, much less is known about the extent to which these integrative deficits occur outside of conduction aphasia and how this manifests behaviorally in areas other than speech repetition. In this study, we aimed to address these issues by examining the behavioral correlates of speech sensorimotor impairment under altered auditory feedback (AAF) and their relationship with the impaired ability to independently correct for online errors during picture naming in people with aphasia. We found that people with aphasia generate slower vocal compensation response to pitch-shift AAF stimuli compared with controls. However, when the timing of responses was controlled for, no significant difference in the magnitude of vocal pitch compensation was observed between aphasia and control groups. Moreover, no relationship was found between self-correction of naming errors and the timing and magnitude of vocal compensation responses to AAF. These findings suggest that slowed compensation is a potential behavioral marker of impaired sensorimotor integration in aphasia.

Adaptation to pitch-altered feedback is independent of one's own voice pitch sensitivity

Monitoring voice pitch is a fine-tuned process in daily conversations as conveying accurately the linguistic and affective cues in a given utterance depends on the precise control of phonation and intonation. This monitoring is thought to depend on whether the error is treated as self-generated or externally-generated, resulting in either a correction or inflation of errors. The present study reports on two separate paradigms of adaptation to altered feedback to explore whether participants could behave in a more cohesive manner once the error is of comparable size perceptually. The vocal behavior of normal-hearing and fluent speakers was recorded in response to a personalized size of pitch shift versus a non-specific size, one semitone. The personalized size of shift was determined based on the just-noticeable difference in fundamental frequency (F0) of each participant’s voice. Here we show that both tasks successfully demonstrated opposing responses to a constant and predictable F0 perturbation (on from the production onset) but these effects barely carried over once the feedback was back to normal, depicting a pattern that bears some resemblance to compensatory responses. Experiencing a F0 shift that is perceived as self-generated (because it was precisely just-noticeable) is not enough to force speakers to behave more consistently and more homogeneously in an opposing manner. On the contrary, our results suggest that the type of the response as well as the magnitude of the response do not depend in any trivial way on the sensitivity of participants to their own voice pitch. Based on this finding, we speculate that error correction could possibly occur even with a bionic ear, typically even when F0 cues are too subtle for cochlear implant users to detect accurately.

Contribution of sensory memory to speech motor learning

Speech learning requires precise motor control, but it likewise requires transient storage of information to enable the adjustment of upcoming movements based on the success or failure of previous attempts. The contribution of somatic sensory memory for limb position has been documented in work on arm movement; however, in speech, the sensory support for speech production comes from both somatosensory and auditory inputs, and accordingly sensory memory for either or both of sounds and somatic inputs might contribute to learning. In the present study, adaptation to altered auditory feedback was used as an experimental model of speech motor learning. Participants also underwent tests of both auditory and somatic sensory memory. We found that although auditory memory for speech sounds is better than somatic memory for speechlike facial skin deformations, somatic sensory memory predicts adaptation, whereas auditory sensory memory does not. Thus even though speech relies substantially on auditory inputs and in the present manipulation adaptation requires the minimization of auditory error, it is somatic inputs that provide the memory support for learning.NEW & NOTEWORTHY In speech production, almost everyone achieves an exceptionally high level of proficiency. This is remarkable because speech involves some of the smallest and most carefully timed movements of which we are capable. The present paper demonstrates that sensory memory contributes to speech motor learning. Moreover, we report the surprising result that somatic sensory memory predicts speech motor learning, whereas auditory memory does not.

Auditory-Motor Perturbations of Voice Fundamental Frequency: Feedback Delay and Amplification

Purpose Gradual and sudden perturbations of vocal fundamental frequency (f _o), also known as adaptive and reflexive f _o perturbations, are techniques to study the influence of auditory feedback on voice f _o control mechanisms. Previous vocal f _o perturbations have incorporated varied setup-specific feedback delays and amplifications. Here, we investigated the effects of feedback delays (10-100 ms) and amplifications on both adaptive and reflexive f _o perturbation paradigms, encapsulating the variability in equipment-specific delays (3-45 ms) and amplifications utilized in previous experiments. Method Responses to adaptive and reflexive f _o perturbations were recorded in 24 typical speakers for four delay conditions (10, 40, 70, and 100 ms) or three amplification conditions (-10, +5, and +10 dB relative to microphone) in a counterbalanced order. Repeated-measures analyses of variance were carried out on the magnitude of f _o responses to determine the effect of feedback condition. Results There was a statistically significant effect of the level of auditory feedback amplification on the response magnitude during adaptive f _o perturbations, driven by the difference between +10- and -10-dB amplification conditions (hold phase difference: M = 38.3 cents, SD = 51.2 cents; after-effect phase: M = 66.1 cents, SD = 84.6 cents). No other statistically significant effects of condition were found for either paradigm. Conclusions Experimental equipment delays below 100 ms in behavioral paradigms do not affect the results of f _o perturbation paradigms. As there is no statistically significant difference between the response magnitudes elicited by +5- and +10-dB auditory amplification conditions, this study is a confirmation that an auditory feedback amplification of +5 dB relative to microphone is sufficient to elicit robust compensatory responses for f _o perturbation paradigms.

It's About Time: Minimizing Hardware and Software Latencies in Speech Research With Real-Time Auditory Feedback

Purpose Various aspects of speech production related to auditory-motor integration and learning have been examined through auditory feedback perturbation paradigms in which participants' acoustic speech output is experimentally altered and played back via earphones/headphones "in real time." Scientific rigor requires high precision in determining and reporting the involved hardware and software latencies. Many reports in the literature, however, are not consistent with the minimum achievable latency for a given experimental setup. Here, we focus specifically on this methodological issue associated with implementing real-time auditory feedback perturbations, and we offer concrete suggestions for increased reproducibility in this particular line of work. Method Hardware and software latencies as well as total feedback loop latency were measured for formant perturbation studies with the Audapter software. Measurements were conducted for various audio interfaces, desktop and laptop computers, and audio drivers. An approach for lowering Audapter's software latency through nondefault parameter specification was also tested. Results Oft-overlooked hardware-specific latencies were not negligible for some of the tested audio interfaces (adding up to 15 ms). Total feedback loop latencies (including both hardware and software latency) were also generally larger than claimed in the literature. Nondefault parameter values can improve Audapter's own processing latency without negative impact on formant tracking. Conclusions Audio interface selection and software parameter optimization substantially affect total feedback loop latency. Thus, the actual total latency (hardware plus software) needs to be correctly measured and described in all published reports. Future speech research with "real-time" auditory feedback perturbations should increase scientific rigor by minimizing this latency.

Exposure to Auditory Feedback Delay while Speaking Induces Perceptual Habituation but does not Mitigate the Disruptive Effect of Delay on Speech Auditory-motor Learning

Perceiving the sensory consequences of our actions with a delay alters the interpretation of these afferent signals and impacts motor learning. For reaching movements, delayed visual feedback of hand position reduces the rate and extent of visuomotor adaptation, but substantial adaptation still occurs. Moreover, the detrimental effect of visual feedback delay on reach motor learning-selectively affecting its implicit component-can be mitigated by prior habituation to the delay. Auditory-motor learning for speech has been reported to be more sensitive to feedback delay, and it remains unknown whether habituation to auditory delay reduces its negative impact on learning. We investigated whether 30 min of exposure to auditory delay during speaking (a) affects the subjective perception of delay, and (b) mitigates its disruptive effect on speech auditory-motor learning. During a speech adaptation task with real-time perturbation of vowel spectral properties, participants heard this frequency-shifted feedback with no delay, 75 ms delay, or 115 ms delay. In the delay groups, 50% of participants had been exposed to the delay throughout a preceding 30-minute block of speaking whereas the remaining participants completed this block without delay. Although habituation minimized awareness of the delay, no improvement in adaptation to the spectral perturbation was observed. Thus, short-term habituation to auditory feedback delays is not effective in reducing the negative impact of delay on speech auditory-motor adaptation. Combined with previous findings, the strong negative effect of delay and the absence of an influence of delay awareness suggest the involvement of predominantly implicit learning mechanisms in speech.

Event-related potential correlates of auditory feedback control of vocal production in experienced singers

Considerable evidence has shown that experienced singers are capable of voluntarily suppressing vocal compensations for consistent pitch perturbations in auditory feedback. Our recent behavioral study found that singers also compensated for brief pitch perturbations to a lesser degree than nonsingers in an involuntary manner. In the present event-related potential study, we investigated the neural correlates of involuntary vocal pitch regulation in experienced singers. All participants were instructed to vocalize the vowel sounds while their voice was unexpectedly shifted in pitch by -50 and -200 cents. The results revealed decreased cortical N1 and P2 responses to pitch perturbations and reduced involuntary vocal compensations for singers when compared to nonsingers. Moreover, larger vocal responses were significantly correlated with smaller cortical P2 responses for nonsingers, whereas this brain-behavior relationship did not exist for singers. These findings demonstrate that the cortical processing of involuntary auditory-motor integration for vocal pitch regulation can be shaped as a function of singing experience, suggesting that experienced singers may be less influenced by auditory feedback and rely more on somatosensory feedback or feedforward control as a consequence of singing training as compared to nonsingers.

A Simple 3-Parameter Model for Examining Adaptation in Speech and Voice Production

Sensorimotor adaptation experiments are commonly used to examine motor learning behavior and to uncover information about the underlying control mechanisms of many motor behaviors, including speech production. In the speech and voice domains, aspects of the acoustic signal are shifted/perturbed over time via auditory feedback manipulations. In response, speakers alter their production in the opposite direction of the shift so that their perceived production is closer to what they intended. This process relies on a combination of feedback and feedforward control mechanisms that are difficult to disentangle. The current study describes and tests a simple 3-parameter mathematical model that quantifies the relative contribution of feedback and feedforward control mechanisms to sensorimotor adaptation. The model is a simplified version of the DIVA model, an adaptive neural network model of speech motor control. The three fitting parameters of SimpleDIVA are associated with the three key subsystems involved in speech motor control, namely auditory feedback control, somatosensory feedback control, and feedforward control. The model is tested through computer simulations that identify optimal model fits to six existing sensorimotor adaptation datasets. We show its utility in (1) interpreting the results of adaptation experiments involving the first and second formant frequencies as well as fundamental frequency; (2) assessing the effects of masking noise in adaptation paradigms; (3) fitting more than one perturbation dimension simultaneously; (4) examining sensorimotor adaptation at different timepoints in the production signal; and (5) quantitatively predicting responses in one experiment using parameters derived from another experiment. The model simulations produce excellent fits to real data across different types of perturbations and experimental paradigms (mean correlation between data and model fits across all six studies = 0.95 ± 0.02). The model parameters provide a mechanistic explanation for the behavioral responses to the adaptation paradigm that are not readily available from the behavioral responses alone. Overall, SimpleDIVA offers new insights into speech and voice motor control and has the potential to inform future directions of speech rehabilitation research in disordered populations. Simulation software, including an easy-to-use graphical user interface, is publicly available to facilitate the use of the model in future studies.

Decreased Gray-Matter Volume in Insular Cortex as a Correlate of Singers' Enhanced Sensorimotor Control of Vocal Production

Accumulating evidence has shown enhanced sensorimotor control of vocal production as a consequence of extensive singing experience. The neural basis of this ability, however, is poorly understood. Given that the insula mediates motor aspects of vocal production, the present study investigated structural plasticity in insula induced by singing experience and its link to auditory feedback control of vocal production. Voxel-based morphometry (VBM) was used to examine the differences in gray matter (GM) volume in the insula of 21 singers and 21 non-singers. An auditory feedback perturbation paradigm was used to examine the differences in auditory-motor control of vocal production between singers and non-singers. Both groups vocalized sustained vowels while hearing their voice unexpectedly pitch-shifted −50 or −200 cents (200 ms duration). VBM analyses showed that singers exhibited significantly lower GM volumes in the bilateral insula than non-singers. When exposed to pitch perturbations in voice auditory feedback, singers involuntarily compensated for pitch perturbations in voice auditory feedback to a significantly lesser degree than non-singers. Moreover, across the two sizes of pitch perturbations, the magnitudes of vocal compensations were positively correlated with the total regional GM volumes in the bilateral insula. These results indicate that extensive singing training leads to decreased GM volumes in insula and suggest that morphometric plasticity in insula contributes to the enhanced sensorimotor control of vocal production observed in singers.

Pitch Shifting With the Commercially Available Eventide Eclipse: Intended and Unintended Changes to the Speech Signal

Purpose This study details the intended and unintended consequences of pitch shifting with the commercially available Eventide Eclipse. Method Ten vocally healthy participants ( M = 22.0 years; 6 cisgender females, 4 cisgender males) produced a sustained /ɑ/, creating an input signal. This input signal was processed in near real time by the Eventide Eclipse to create an output signal that was either not shifted (0 cents), shifted +100 cents, or shifted -100 cents. Shifts occurred either throughout the entire vocalization or for a 200-ms period after vocal onset. Results Input signals were compared to output signals to examine potential changes. Average pitch-shift magnitudes were within 1 cent of the intended pitch shift. Measured pitch-shift length for intended 200-ms shifts was between 5.9% and 21.7% less than expected, based on the portion of shift selected for measurement. The delay between input and output signals was an average of 11.1 ms. Trials shifted +100 cents had a longer delay than trials shifted -100 or 0 cents. The first 2 formants (F1, F2) shifted in the direction of the pitch shift, with F1 shifting 6.5% and F2 shifting 6.0%. Conclusions The Eventide Eclipse is an accurate pitch-shifting hardware that can be used to explore voice and vocal motor control. The pitch-shifting algorithm shifts all frequencies, resulting in a subsequent change in F1 and F2 during pitch-shifted trials. Researchers using this device should be mindful of stimuli selection to avoid confusion during data interpretation.

Consistency influences altered auditory feedback processing

Previous research on the effect of perturbed auditory feedback in speech production has focused on two types of responses. In the short term, speakers generate compensatory motor commands in response to unexpected perturbations. In the longer term, speakers adapt feedforward motor programmes in response to feedback perturbations, to avoid future errors. The current study investigated the relation between these two types of responses to altered auditory feedback. Specifically, it was hypothesised that consistency in previous feedback perturbations would influence whether speakers adapt their feedforward motor programmes. In an altered auditory feedback paradigm, formant perturbations were applied either across all trials (the consistent condition) or only to some trials, whereas the others remained unperturbed (the inconsistent condition). The results showed that speakers' responses were affected by feedback consistency, with stronger speech changes in the consistent condition compared with the inconsistent condition. Current models of speech-motor control can explain this consistency effect. However, the data also suggest that compensation and adaptation are distinct processes, which are not in line with all current models.

Online Adaptation to Altered Auditory Feedback Is Predicted by Auditory Acuity and Not by Domain-General Executive Control Resources

When a speaker's auditory feedback is altered, he adapts for the perturbation by altering his own production, which demonstrates the role of auditory feedback in speech motor control. In the present study, we explored the role of auditory acuity and executive control in this process. Based on the DIVA model and the major cognitive control models, we expected that higher auditory acuity, and better executive control skills would predict larger adaptation to the alteration. Thirty-six Spanish native speakers performed an altered auditory feedback experiment, executive control (numerical Stroop, Simon and Flanker) tasks, and auditory acuity tasks (loudness, pitch, and melody pattern discrimination). In the altered feedback experiment, participants had to produce the pseudoword “pep” (/pep/) while perceiving their auditory feedback in real time through earphones. The auditory feedback was first unaltered and then progressively altered in F1 and F2 dimensions until maximal alteration (F1 −150 Hz; F2 +300 Hz). The normalized distance of maximal adaptation ranged from 4 to 137 Hz (median of 75 ± 36). The different measures of auditory acuity were significant predictors of adaptation, while individual measures of cognitive function skills (obtained from the executive control tasks) were not. Better auditory discriminators adapted more to the alteration. We conclude that adaptation to altered auditory feedback is very well-predicted by general auditory acuity, as suggested by the DIVA model. In line with the framework of motor-control models, no specific claim on the implication of executive resources in speech motor control can be made.

Auditory-Motor Control of Vocal Production during Divided Attention: Behavioral and ERP Correlates

When people hear unexpected perturbations in auditory feedback, they produce rapid compensatory adjustments of their vocal behavior. Recent evidence has shown enhanced vocal compensations and cortical event-related potentials (ERPs) in response to attended pitch feedback perturbations, suggesting that this reflex-like behavior is influenced by selective attention. Less is known, however, about auditory-motor integration for voice control during divided attention. The present cross-modal study investigated the behavioral and ERP correlates of auditory feedback control of vocal pitch production during divided attention. During the production of sustained vowels, 32 young adults were instructed to simultaneously attend to both pitch feedback perturbations they heard and flashing red lights they saw. The presentation rate of the visual stimuli was varied to produce a low, intermediate, and high attentional load. The behavioral results showed that the low-load condition elicited significantly smaller vocal compensations for pitch perturbations than the intermediate-load and high-load conditions. As well, the cortical processing of vocal pitch feedback was also modulated as a function of divided attention. When compared to the low-load and intermediate-load conditions, the high-load condition elicited significantly larger N1 responses and smaller P2 responses to pitch perturbations. These findings provide the first neurobehavioral evidence that divided attention can modulate auditory feedback control of vocal pitch production.

Adaptation in Mandarin tone production with pitch-shifted auditory feedback: Influence of tonal contrast requirements

We investigated Mandarin speakers' control of lexical tone production with F0-perturbed auditory feedback. Subjects produced high level (T1), mid rising (T2), low dipping (T3), and high falling (T4) tones in conditions with (a) no perturbation, (b) T1 shifted down, (c) T1 shifted down and T3 shifted up, or (d) T1 shifted down and T3 shifted up but without producing other tones. Speakers and new subjects also completed a tone identification task with unaltered and F0-perturbed productions. With only T1 perturbed down, speakers adapted by raising F0 relative to no-perturbation. With simultaneous T1 down and T3 up perturbations, no T1 adaptation occurred, and T3 adaptation occurred only if T2 was also produced. Identification accuracy with stimuli representing adapted productions was comparable to baseline, but with simulated non-adapted productions it was reduced for T2 and T3. Thus, Mandarin speakers' adaptation to F0 perturbations is linguistically constrained and serves to maintain tone contrast.

Top-Down Modulation of Auditory-Motor Integration during Speech Production: The Role of Working Memory

Although working memory (WM) is considered as an emergent property of the speech perception and production systems, the role of WM in sensorimotor integration during speech processing is largely unknown. We conducted two event-related potential experiments with female and male young adults to investigate the contribution of WM to the neurobehavioural processing of altered auditory feedback during vocal production. A delayed match-to-sample task that required participants to indicate whether the pitch feedback perturbations they heard during vocalizations in test and sample sequences matched, elicited significantly larger vocal compensations, larger N1 responses in the left middle and superior temporal gyrus, and smaller P2 responses in the left middle and superior temporal gyrus, inferior parietal lobule, somatosensory cortex, right inferior frontal gyrus, and insula compared with a control task that did not require memory retention of the sequence of pitch perturbations. On the other hand, participants who underwent extensive auditory WM training produced suppressed vocal compensations that were correlated with improved auditory WM capacity, and enhanced P2 responses in the left middle frontal gyrus, inferior parietal lobule, right inferior frontal gyrus, and insula that were predicted by pretraining auditory WM capacity. These findings indicate that WM can enhance the perception of voice auditory feedback errors while inhibiting compensatory vocal behavior to prevent voice control from being excessively influenced by auditory feedback. This study provides the first evidence that auditory-motor integration for voice control can be modulated by top-down influences arising from WM, rather than modulated exclusively by bottom-up and automatic processes.SIGNIFICANCE STATEMENT One outstanding question that remains unsolved in speech motor control is how the mismatch between predicted and actual voice auditory feedback is detected and corrected. The present study provides two lines of converging evidence, for the first time, that working memory cannot only enhance the perception of vocal feedback errors but also exert inhibitory control over vocal motor behavior. These findings represent a major advance in our understanding of the top-down modulatory mechanisms that support the detection and correction of prediction-feedback mismatches during sensorimotor control of speech production driven by working memory. Rather than being an exclusively bottom-up and automatic process, auditory-motor integration for voice control can be modulated by top-down influences arising from working memory.

Sensorimotor learning in children and adults: Exposure to frequency-altered auditory feedback during speech production

Auditory feedback plays an important role in the acquisition of fluent speech; however, this role may change once speech is acquired and individuals no longer experience persistent developmental changes to the brain and vocal tract. For this reason, we investigated whether the role of auditory feedback in sensorimotor learning differs across children and adult speakers. Participants produced vocalizations while they heard their vocal pitch predictably or unpredictably shifted downward one semitone. The participants' vocal pitches were measured at the beginning of each vocalization, before auditory feedback was available, to assess the extent to which the deviant auditory feedback modified subsequent speech motor commands. Sensorimotor learning was observed in both children and adults, with participants' initial vocal pitch increasing following trials where they were exposed to predictable, but not unpredictable, frequency-altered feedback. Participants' vocal pitch was also measured across each vocalization, to index the extent to which the deviant auditory feedback was used to modify ongoing vocalizations. While both children and adults were found to increase their vocal pitch following predictable and unpredictable changes to their auditory feedback, adults produced larger compensatory responses. The results of the current study demonstrate that both children and adults rapidly integrate information derived from their auditory feedback to modify subsequent speech motor commands. However, these results also demonstrate that children and adults differ in their ability to use auditory feedback to generate compensatory vocal responses during ongoing vocalization. Since vocal variability also differed across the children and adult groups, these results also suggest that compensatory vocal responses to frequency-altered feedback manipulations initiated at vocalization onset may be modulated by vocal variability.

Attentional demands modulate sensorimotor learning induced by persistent exposure to changes in auditory feedback

Hearing one's own voice is important for regulating ongoing speech and for mapping speech sounds onto articulator movements. However, it is currently unknown whether attention mediates changes in the relationship between motor commands and their acoustic output, which are necessary as growth and aging inevitably cause changes to the vocal tract. In this study, participants produced vocalizations while they heard their vocal pitch persistently shifted downward one semitone in both single- and dual-task conditions. During the single-task condition, participants vocalized while passively viewing a visual stream. During the dual-task condition, participants vocalized while also monitoring a visual stream for target letters, forcing participants to divide their attention. Participants' vocal pitch was measured across each vocalization, to index the extent to which their ongoing vocalization was modified as a result of the deviant auditory feedback. Smaller compensatory responses were recorded during the dual-task condition, suggesting that divided attention interfered with the use of auditory feedback for the regulation of ongoing vocalizations. Participants' vocal pitch was also measured at the beginning of each vocalization, before auditory feedback was available, to assess the extent to which the deviant auditory feedback was used to modify subsequent speech motor commands. Smaller changes in vocal pitch at vocalization onset were recorded during the dual-task condition, suggesting that divided attention diminished sensorimotor learning. Together, the results of this study suggest that attention is required for the speech motor control system to make optimal use of auditory feedback for the regulation and planning of speech motor commands.

Transfer Effect of Speech-sound Learning on Auditory-motor Processing of Perceived Vocal Pitch Errors

Speech perception and production are intimately linked. There is evidence that speech motor learning results in changes to auditory processing of speech. Whether speech motor control benefits from perceptual learning in speech, however, remains unclear. This event-related potential study investigated whether speech-sound learning can modulate the processing of feedback errors during vocal pitch regulation. Mandarin speakers were trained to perceive five Thai lexical tones while learning to associate pictures with spoken words over 5 days. Before and after training, participants produced sustained vowel sounds while they heard their vocal pitch feedback unexpectedly perturbed. As compared to the pre-training session, the magnitude of vocal compensation significantly decreased for the control group, but remained consistent for the trained group at the post-training session. However, the trained group had smaller and faster N1 responses to pitch perturbations and exhibited enhanced P2 responses that correlated significantly with their learning performance. These findings indicate that the cortical processing of vocal pitch regulation can be shaped by learning new speech-sound associations, suggesting that perceptual learning in speech can produce transfer effects to facilitating the neural mechanisms underlying the online monitoring of auditory feedback regarding vocal production.

Attention modulates cortical processing of pitch feedback errors in voice control

Considerable evidence has shown that unexpected alterations in auditory feedback elicit fast compensatory adjustments in vocal production. Although generally thought to be involuntary in nature, whether these adjustments can be influenced by attention remains unknown. The present event-related potential (ERP) study aimed to examine whether neurobehavioral processing of auditory-vocal integration can be affected by attention. While sustaining a vowel phonation and hearing pitch-shifted feedback, participants were required to either ignore the pitch perturbations, or attend to them with low (counting the number of perturbations) or high attentional load (counting the type of perturbations). Behavioral results revealed no systematic change of vocal response to pitch perturbations irrespective of whether they were attended or not. At the level of cortex, there was an enhancement of P2 response to attended pitch perturbations in the low-load condition as compared to when they were ignored. In the high-load condition, however, P2 response did not differ from that in the ignored condition. These findings provide the first neurophysiological evidence that auditory-motor integration in voice control can be modulated as a function of attention at the level of cortex. Furthermore, this modulatory effect does not lead to a general enhancement but is subject to attentional load.

Auditory feedback of one's own voice is used for high-level semantic monitoring: the "self-comprehension" hypothesis

What would it be like if we said one thing, and heard ourselves saying something else? Would we notice something was wrong? Or would we believe we said the thing we heard? Is feedback of our own speech only used to detect errors, or does it also help to specify the meaning of what we say? Comparator models of self-monitoring favor the first alternative, and hold that our sense of agency is given by the comparison between intentions and outcomes, while inferential models argue that agency is a more fluent construct, dependent on contextual inferences about the most likely cause of an action. In this paper, we present a theory about the use of feedback during speech. Specifically, we discuss inferential models of speech production that question the standard comparator assumption that the meaning of our utterances is fully specified before articulation. We then argue that auditory feedback provides speakers with a channel for high-level, semantic “self-comprehension”. In support of this we discuss results using a method we recently developed called Real-time Speech Exchange (RSE). In our first study using RSE (Lind et al., in press) participants were fitted with headsets and performed a computerized Stroop task. We surreptitiously recorded words they said, and later in the test we played them back at the exact same time that the participants uttered something else, while blocking the actual feedback of their voice. Thus, participants said one thing, but heard themselves saying something else. The results showed that when timing conditions were ideal, more than two thirds of the manipulations went undetected. Crucially, in a large proportion of the non-detected manipulated trials, the inserted words were experienced as self-produced by the participants. This indicates that our sense of agency for speech has a strong inferential component, and that auditory feedback of our own voice acts as a pathway for semantic monitoring. We believe RSE holds great promise as a tool for investigating the role of auditory feedback during speech, and we suggest a number of future studies to serve this purpose.