Enhanced auditory evoked activity to self-generated sounds is mediated by primary and supplementary motor cortices

Hippocampal-entorhinal cognitive maps and cortical motor system represent action plans and their outcomes

Efficiently interacting with the environment requires weighing and selecting among multiple alternative actions based on their associated outcomes. However, the neural mechanisms underlying these processes are still debated. We show that forming relations between arbitrary action-outcome associations involve building a cognitive map. Using an immersive virtual reality paradigm, participants learned 2D abstract motor action-outcome associations and later compared action combinations while their brain activity was monitored with fMRI. We observe a hexadirectional modulation of the activity in entorhinal cortex while participants compared different action plans. Furthermore, hippocampal activity scales with the 2D similarity between outcomes of these action plans. Conversely, the supplementary motor area represents individual actions, showing a stronger response to overlapping action plans. Crucially, the connectivity between hippocampus and supplementary motor area is modulated by the similarity between the action plans, suggesting their complementary roles in action evaluation. These findings provide evidence for the role of cognitive maps in action selection, challenging classical models of memory taxonomy and its neural bases.

Somatosensory influence on auditory cortical response of self-generated sound

Motor execution which results in the generation of sounds attenuates the cortical response to these self-generated sounds. This attenuation has been explained as a result of motor relevant processing. The current study shows that corresponding somatosensory inputs can also change the auditory processing of a self-generated sound. We recorded auditory event-related potentials (ERP) in response to self-generated sounds and assessed how the amount of auditory attenuation changed according to the somatosensory inputs. The sound stimuli were generated by a finger movement that pressed on a virtual object, which was produced by a haptic robotic device. Somatosensory inputs were modulated by changing the stiffness of this virtual object (low and high) in an unpredictable manner. For comparison purposes, we carried out the same test with a computer keyboard, which is conventionally used to induce the auditory attenuation of self-generated sound. While N1 and P2 attenuations were clearly induced in the control condition with the keyboard as has been observed in previous studies, when using the robotic device the amplitude of N1 was found to vary according to the stiffness of the virtual object. The amplitude of N1 in the low stiffness condition was similar to that found using the keyboard for the same condition but not in the high stiffness condition. In addition, P2 attenuation did not differ between stiffness conditions. The waveforms of auditory ERP after 200 ms also differed according to the stiffness conditions. The estimated source of N1 attenuation was located in the right parietal area. These results suggest that somatosensory inputs during movement can modify the auditory processing of self-generated sound. The auditory processing of self-generated sound may represent self-referenced processing like an embodied process or an action-perception mechanism.

The prediction of auditory consequences of own and observed actions: a brain decoding multivariate pattern study

Evidence from the auditory domain suggests that sounds generated by self-performed as well as observed actions are processed differently compared to external sounds. This study aimed to investigate which brain regions are involved in the processing of auditory stimuli generated by actions, addressing the question of whether cerebellar forward models, which are supposed to predict the sensory consequences of self-performed actions, similarly underlie predictions for action observation. We measured brain activity with functional magnetic resonance imaging (fMRI) while participants elicited a sound via button press, observed another person performing this action, or listened to external sounds. By applying multivariate pattern analysis (MVPA), we found evidence for altered processing in the right auditory cortex for sounds following both self-performed and observed actions relative to external sounds. Evidence for the prediction of auditory action consequences was found in the bilateral cerebellum and the right supplementary motor area, but only for self-performed actions. Our results suggest that cerebellar forward models contribute to predictions of sensory consequences only for action performance. While predictions are also generated for action observation, the underlying mechanisms remain to be elucidated.

Sensory attenuation of self-initiated tactile feedback is modulated by stimulus strength and temporal delay in a virtual reality environment

Despite extensive research across various modalities, the precise mechanisms of sensory attenuation (SA) remain debated. Specifically, it remains unclear to what extent SA is influenced by stimulus predictability alone, as opposed to the distinct impact of self-generated actions. Forward models suggest that efference copies of motor commands enable the brain to predict and distinguish anticipated changes in self-initiated sensory input. Predictive processing proposes that predictions about upcoming changes in sensory input are not solely based on efference copies, but rather generated in the form of a generative model integrating external, contextual factors, as well. This study investigated the underlying mechanisms of SA in the tactile domain, specifically examining self-initiation and temporal predictions within a virtual reality (VR) framework. This setup allowed for precise control over sensory feedback in response to movement. Participants (N = 33) engaged in an active condition, moving their hands to elicit a virtual touch. Importantly, visual perception was modified in VR, so that participants touched their rendered-but not physical-hands. The virtual touch triggered the test vibrations on a touch controller (intensities: 0.2, 0.35, 0.5, 0.65, 0.8; in arbitrary units.), the intensity of which was then compared to that of a standard stimulus (intensity: 0.5). In the passive condition, vibrations were presented without movement and were preceded by a visual cue. Further, test vibrations appeared either immediately or after a variable onset delay (700-800ms). Our results revealed a significant effect of the factor "onset delay" on perceived vibration intensity. In addition, we observed interactions between the factors "agency" and "test vibration intensity" and between the factors "agency" and "onset delay," with attenuation effects for immediate vibrations at high intensities and enhancement effects for delayed vibrations at low intensities. These findings emphasize the impact of external, contextual factors and support the notion of a broader, attention-oriented predictive mechanism for the perception of self-initiated stimuli.

Suppression and omission effects in auditory predictive processing-Two of the same?

Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two different modulations of the auditory N1 event-related brain potential component are often discussed as an expression of auditory predictive processing. The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). Both phenomena were explained by action-related forward modelling, which takes place when the sensory input is predictable: prediction error signals are reduced when predicted sensory input is presented (N1 suppression) and elicited when predicted sensory input is omitted (omission N1). This common theoretical account is appealing but has not yet been directly tested. We manipulated the predictability of a sound in a self-generation paradigm in which, in two conditions, either 80% or 50% of the button presses did generate a sound, inducing a strong or a weak expectation for the occurrence of the sound. Consistent with the forward modelling account, an omission N1 was observed in the 80% but not in the 50% condition. However, N1 suppression was highly similar in both conditions. Thus, our results demonstrate a clear effect of predictability for the omission N1 but not for the N1 suppression. These results imply that the two phenomena rely (at least in part) on different mechanisms and challenge prediction related accounts of N1 suppression.

Somatosensory omissions reveal action-related predictive processing

The intricate relation between action and somatosensory perception has been studied extensively in the past decades. Generally, a forward model is thought to predict the somatosensory consequences of an action. These models propose that when an action is reliably coupled to a tactile stimulus, unexpected absence of the stimulus should elicit prediction error. Although such omission responses have been demonstrated in the auditory modality, it remains unknown whether this mechanism generalizes across modalities. This study therefore aimed to record action-induced somatosensory omission responses using EEG in humans. Self-paced button presses were coupled to somatosensory stimuli in 88% of trials, allowing a prediction, or in 50% of trials, not allowing a prediction. In the 88% condition, stimulus omission resulted in a neural response consisting of multiple components, as revealed by temporal principal component analysis. The oN1 response suggests similar sensory sources as stimulus-evoked activity, but an origin outside primary cortex. Subsequent oN2 and oP3 responses, as previously observed in the auditory domain, likely reflect modality-unspecific higher order processes. Together, findings straightforwardly demonstrate somatosensory predictions during action and provide evidence for a partially amodal mechanism of prediction error generation.

Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

Texture, a fundamental object attribute, is perceived through multisensory information including touch and auditory cues. Coherent perceptions may rely on shared texture representations across different senses in the brain. To test this hypothesis, we delivered haptic textures coupled with a sound synthesizer to generate real-time textural sounds. Participants completed roughness estimation tasks with haptic, auditory, or bimodal cues in an MRI scanner. Somatosensory, auditory, and visual cortices were all activated during haptic and auditory exploration, challenging the traditional view that primary sensory cortices are sense-specific. Furthermore, audio-tactile integration was found in secondary somatosensory (S2) and primary auditory cortices. Multivariate analyses revealed shared spatial activity patterns in primary motor and somatosensory cortices, for discriminating texture across both modalities. This study indicates that primary areas and S2 have a versatile representation of multisensory textures, which has significant implications for how the brain processes multisensory cues to interact more efficiently with our environment.

Auditory cortex ensembles jointly encode sound and locomotion speed to support sound perception during movement

The ability to process and act upon incoming sounds during locomotion is critical for survival and adaptive behavior. Despite the established role that the auditory cortex (AC) plays in behavior- and context-dependent sound processing, previous studies have found that auditory cortical activity is on average suppressed during locomotion as compared to immobility. While suppression of auditory cortical responses to self-generated sounds results from corollary discharge, which weakens responses to predictable sounds, the functional role of weaker responses to unpredictable external sounds during locomotion remains unclear. In particular, whether suppression of external sound-evoked responses during locomotion reflects reduced involvement of the AC in sound processing or whether it results from masking by an alternative neural computation in this state remains unresolved. Here, we tested the hypothesis that rather than simple inhibition, reduced sound-evoked responses during locomotion reflect a tradeoff with the emergence of explicit and reliable coding of locomotion velocity. To test this hypothesis, we first used neural inactivation in behaving mice and found that the AC plays a critical role in sound-guided behavior during locomotion. To investigate the nature of this processing, we used two-photon calcium imaging of local excitatory auditory cortical neural populations in awake mice. We found that locomotion had diverse influences on activity of different neurons, with a net suppression of baseline-subtracted sound-evoked responses and neural stimulus detection, consistent with previous studies. Importantly, we found that the net inhibitory effect of locomotion on baseline-subtracted sound-evoked responses was strongly shaped by elevated ongoing activity that compressed the response dynamic range, and that rather than reflecting enhanced "noise," this ongoing activity reliably encoded the animal's locomotion speed. Decoding analyses revealed that locomotion speed and sound are robustly co-encoded by auditory cortical ensemble activity. Finally, we found consistent patterns of joint coding of sound and locomotion speed in electrophysiologically recorded activity in freely moving rats. Together, our data suggest that rather than being suppressed by locomotion, auditory cortical ensembles explicitly encode it alongside sound information to support sound perception during locomotion.

Being active over one's own motion: Considering predictive mechanisms in self-motion perception

Self-motion perception is a key element guiding pilots' behavior. Its importance is mostly revealed when impaired, leading in most cases to spatial disorientation which is still today a major factor of accidents occurrence. Self-motion perception is known as mainly based on visuo-vestibular integration and can be modulated by the physical properties of the environment with which humans interact. For instance, several studies have shown that the respective weight of visual and vestibular information depends on their reliability. More recently, it has been suggested that the internal state of an operator can also modulate multisensory integration. Interestingly, the systems' automation can interfere with this internal state through the loss of the intentional nature of movements (i.e., loss of agency) and the modulation of associated predictive mechanisms. In this context, one of the new challenges is to better understand the relationship between automation and self-motion perception. The present review explains how linking the concepts of agency and self-motion is a first approach to address this issue.

Speaking in gestures: Left dorsal and ventral frontotemporal brain systems underlie communication in conducting

Conducting constitutes a well-structured system of signs anticipating information concerning the rhythm and dynamic of a musical piece. Conductors communicate the musical tempo to the orchestra, unifying the individual instrumental voices to form an expressive musical Gestalt. In a functional magnetic resonance imaging (fMRI) experiment, 12 professional conductors and 16 instrumentalists conducted real-time novel pieces with diverse complexity in orchestration and rhythm. For control, participants either listened to the stimuli or performed beat patterns, setting the time of a metronome or complex rhythms played by a drum. Activation of the left superior temporal gyrus (STG), supplementary and premotor cortex and Broca's pars opercularis (F3op) was shared in both musician groups and separated conducting from the other conditions. Compared to instrumentalists, conductors activated Broca's pars triangularis (F3tri) and the STG, which differentiated conducting from time beating and reflected the increase in complexity during conducting. In comparison to conductors, instrumentalists activated F3op and F3tri when distinguishing complex rhythm processing from simple rhythm processing. Fibre selection from a normative human connectome database, constructed using a global tractography approach, showed that the F3op and STG are connected via the arcuate fasciculus, whereas the F3tri and STG are connected via the extreme capsule. Like language, the anatomical framework characterising conducting gestures is located in the left dorsal system centred on F3op. This system reflected the sensorimotor mapping for structuring gestures to musical tempo. The ventral system centred on F3Tri may reflect the art of conductors to set this musical tempo to the individual orchestra's voices in a global, holistic way.

Active self-motion control and the role of agency under ambiguity

Purpose

Self-motion perception is a key factor in daily behaviours such as driving a car or piloting an aircraft. It is mainly based on visuo-vestibular integration, whose weighting mechanisms are modulated by the reliability properties of sensory inputs. Recently, it has been shown that the internal state of the operator can also modulate multisensory integration and may sharpen the representation of relevant inputs. In line with the concept of agency, it thus appears relevant to evaluate the impact of being in control of our own action on self-motion perception.

Methodology

Here, we tested two conditions of motion control (active/manual trigger versus passive/ observer condition), asking participants to discriminate between two consecutive longitudinal movements by identifying the larger displacement (displacement of higher intensity). We also tested motion discrimination under two levels of ambiguity by applying acceleration ratios that differed from our two "standard" displacements (i.e., 3 s; 0.012 m.s^-2 and 0.030 m.s^-2).

Results

We found an effect of control condition, but not of the level of ambiguity on the way participants perceived the standard displacement, i.e., perceptual bias (Point of Subjective Equality; PSE). Also, we found a significant effect of interaction between the active condition and the level of ambiguity on the ability to discriminate between displacements, i.e., sensitivity (Just Noticeable Difference; JND).

Originality

Being in control of our own motion through a manual intentional trigger of self-displacement maintains overall motion sensitivity when ambiguity increases.

Speech sound categorization: The contribution of non-auditory and auditory cortical regions

Which processes in the human brain lead to the categorical perception of speech sounds? Investigation of this question is hampered by the fact that categorical speech perception is normally confounded by acoustic differences in the stimulus. By using ambiguous sounds, however, it is possible to dissociate acoustic from perceptual stimulus representations. Twenty-seven normally hearing individuals took part in an fMRI study in which they were presented with an ambiguous syllable (intermediate between /da/ and /ga/) in one ear and with disambiguating acoustic feature (third formant, F3) in the other ear. Multi-voxel pattern searchlight analysis was used to identify brain areas that consistently differentiated between response patterns associated with different syllable reports. By comparing responses to different stimuli with identical syllable reports and identical stimuli with different syllable reports, we disambiguated whether these regions primarily differentiated the acoustics of the stimuli or the syllable report. We found that BOLD activity patterns in left perisylvian regions (STG, SMG), left inferior frontal regions (vMC, IFG, AI), left supplementary motor cortex (SMA/pre-SMA), and right motor and somatosensory regions (M1/S1) represent listeners' syllable report irrespective of stimulus acoustics. Most of these regions are outside of what is traditionally regarded as auditory or phonological processing areas. Our results indicate that the process of speech sound categorization implicates decision-making mechanisms and auditory-motor transformations.

The Neurobiology of Pathological Fatigue: New Models, New Questions

The last decade has seen the emergence of new theoretical frameworks to explain pathological fatigue, a much neglected, yet highly significant symptom across a wide range of diseases. While the new models of fatigue provide new hypotheses to test, they also raise a number of questions. The primary purpose of this essay is to examine the predictions of three recently proposed models of fatigue, the overlap and differences between them, and the evidence from diseases that may lend support to the models of fatigue. I also present expansions for the sensory attenuation model of fatigue. Further questions examined here are the following: What are the neural substrates of fatigue? How can sensory attenuation, which underpins agency also explain fatigue? Are fatigue and agency related?

Sound omission related brain responses in children

Action is an important way for children to learn about the world. Recent theories suggest that action is inherently accompanied by the sensory prediction of its effects. Such predictions can be revealed by rarely omitting the expected sensory consequence of the action, resulting in an omission response that is observable in the EEG. Although prediction errors play an important role in models of learning and development, little is known about omission-related brain responses in children. This study used a motor-auditory omission paradigm, testing a group of 6-8-year-old children and an adult group (N = 31 each). In an identity-specific condition, the sound coupled to the motor action was predictable, while in an identity unspecific condition the sound was unpredictable. Results of a temporal principal component analysis revealed that sound-related brain responses underlying the N1-complex differed considerably between age groups. Despite these developmental differences, omission responses (oN1) were similar between age groups. Two subcomponents of the oN1 were differently affected by specific and unspecific predictions. Results demonstrate that children, independent from the maturation of sound processing mechanisms, can implement specific and unspecific predictions as flexibly as adults. This supports theories that regard action and prediction error as important drivers of cognitive development.

Action-locked Neural Responses in Auditory Cortex to Self-generated Sounds

Sensory perception is a product of interactions between the internal state of an organism and the physical attributes of a stimulus. It has been shown across the animal kingdom that perception and sensory-evoked physiological responses are modulated depending on whether or not the stimulus is the consequence of voluntary actions. These phenomena are often attributed to motor signals sent to relevant sensory regions that convey information about upcoming sensory consequences. However, the neurophysiological signature of action-locked modulations in sensory cortex, and their relationship with perception, is still unclear. In the current study, we recorded neurophysiological (using Magnetoencephalography) and behavioral responses from 16 healthy subjects performing an auditory detection task of faint tones. Tones were either generated by subjects' voluntary button presses or occurred predictably following a visual cue. By introducing a constant temporal delay between button press/cue and tone delivery, and applying source-level analysis, we decoupled action-locked and auditory-locked activity in auditory cortex. We show action-locked evoked-responses in auditory cortex following sound-triggering actions and preceding sound onset. Such evoked-responses were not found for button-presses that were not coupled with sounds, or sounds delivered following a predictive visual cue. Our results provide evidence for efferent signals in human auditory cortex that are locked to voluntary actions coupled with future auditory consequences.

Motor Planning Modulates Neural Activity Patterns in Early Human Auditory Cortex

It is well established that movement planning recruits motor-related cortical brain areas in preparation for the forthcoming action. Given that an integral component to the control of action is the processing of sensory information throughout movement, we predicted that movement planning might also modulate early sensory cortical areas, readying them for sensory processing during the unfolding action. To test this hypothesis, we performed 2 human functional magnetic resonance imaging studies involving separate delayed movement tasks and focused on premovement neural activity in early auditory cortex, given the area's direct connections to the motor system and evidence that it is modulated by motor cortex during movement in rodents. We show that effector-specific information (i.e., movements of the left vs. right hand in Experiment 1 and movements of the hand vs. eye in Experiment 2) can be decoded, well before movement, from neural activity in early auditory cortex. We find that this motor-related information is encoded in a separate subregion of auditory cortex than sensory-related information and is present even when movements are cued visually instead of auditorily. These findings suggest that action planning, in addition to preparing the motor system for movement, involves selectively modulating primary sensory areas based on the intended action.

Auditory decisions in the supplementary motor area

In human speech and communication across various species, recognizing and categorizing sounds is fundamental for the selection of appropriate behaviors. However, how does the brain decide which action to perform based on sounds? We explored whether the supplementary motor area (SMA), responsible for linking sensory information to motor programs, also accounts for auditory-driven decision making. To this end, we trained two rhesus monkeys to discriminate between numerous naturalistic sounds and words learned as target (T) or non-target (nT) categories. We found that the SMA at single and population neuronal levels perform decision-related computations that transition from auditory to movement representations in this task. Moreover, we demonstrated that the neural population is organized orthogonally during the auditory and the movement periods, implying that the SMA performs different computations. In conclusion, our results suggest that the SMA integrates acoustic information in order to form categorical signals that drive behavior.

Illusion of control affects ERP amplitude reductions for auditory outcomes of self-generated actions

The reduction of neural responses to self-generated stimuli compared to external stimuli is thought to result from the matching of motor-based sensory predictions and sensory reafferences and to serve the identification of changes in the environment as caused by oneself. The amplitude of the auditory event-related potential (ERP) component N1 seems to closely reflect this matching process, while the later positive component (P2/ P3a) has been associated with judgments of agency, which are also sensitive to contextual top-down information. In this study, we examined the effect of perceived control over sound production on the processing of self-generated and external stimuli, as reflected in these components. We used a new version of a classic two-button choice task to induce different degrees of the illusion of control (IoC) and recorded ERPs for the processing of self-generated and external sounds in a subsequent task. N1 amplitudes were reduced for self-generated compared to external sounds, but not significantly affected by IoC. P2/3a amplitudes were affected by IoC: We found reduced P2/3a amplitudes after a high compared to a low IoC induction training, but only for self-generated, not for external sounds. These findings suggest that prior contextual belief information induced by an IoC affects later processing as reflected in the P2/P3a, possibly for the formation of agency judgments, while early processing reflecting motor-based predictions is not affected.

How Beat Perception Co-opts Motor Neurophysiology

Beat perception offers cognitive scientists an exciting opportunity to explore how cognition and action are intertwined in the brain even in the absence of movement. Many believe the motor system predicts the timing of beats, yet current models of beat perception do not specify how this is neurally implemented. Drawing on recent insights into the neurocomputational properties of the motor system, we propose that beat anticipation relies on action-like processes consisting of precisely patterned neural time-keeping activity in the supplementary motor area (SMA), orchestrated and sequenced by activity in the dorsal striatum. In addition to synthesizing recent advances in cognitive science and motor neuroscience, our framework provides testable predictions to guide future work.

Real and imagined sensory feedback have comparable effects on action anticipation

The forward model monitors the success of sensory feedback to an action and links it to an efference copy originating in the motor system. The Readiness Potential (RP) of the electroencephalogram has been denoted as a neural signature of the efference copy. An open question is whether imagined sensory feedback works similarly to real sensory feedback. We investigated the RP to audible and imagined sounds in a button-press paradigm and assessed the role of sound complexity (vocal vs. non-vocal sound). Sensory feedback (both audible and imagined) in response to a voluntary action modulated the RP amplitude time-locked to the button press. The RP amplitude increase was larger for actions with expected sensory feedback (audible and imagined) than those without sensory feedback, and associated with N1 suppression for audible sounds. Further, the early RP phase was increased when actions elicited an imagined vocal (self-voice) compared to non-vocal sound. Our results support the notion that sensory feedback is anticipated before voluntary actions. This is the case for both audible and imagined sensory feedback and confirms a role of overt and covert feedback in the forward model.

Voluntary Actions Modulate Perception and Neural Representation of Action-Consequences in a Hand-Dependent Manner

Evoked neural activity in sensory regions and perception of sensory stimuli are modulated when the stimuli are the consequence of voluntary movement, as opposed to an external source. It has been suggested that such modulations are due to motor commands that are sent to relevant sensory regions during voluntary movement. However, given the anatomical-functional laterality bias of the motor system, it is plausible that the pattern of such behavioral and neural modulations will also exhibit a similar bias, depending on the effector triggering the stimulus (e.g., right/left hand). Here, we examined this issue in the visual domain using behavioral and neural measures (fMRI). Healthy participants judged the relative brightness of identical visual stimuli that were either self-triggered (using right/left hand button presses), or triggered by the computer. Stimuli were presented either in the right or left visual field. Despite identical physical properties of the visual consequences, we found stronger perceptual modulations when the triggering hand was ipsi- (rather than contra-) lateral to the stimulated visual field. Additionally, fMRI responses in visual cortices differentiated between stimuli triggered by right/left hand. Our findings support a model in which voluntary actions induce sensory modulations that follow the anatomical-functional bias of the motor system.

Distinct Roles for the Cerebellum, Angular Gyrus, and Middle Temporal Gyrus in Action-Feedback Monitoring

Action-feedback monitoring is essential to ensure meaningful interactions with the external world. This process involves generating efference copy-based sensory predictions and comparing these with the actual action-feedback. As neural correlates of comparator processes, previous fMRI studies have provided heterogeneous results, including the cerebellum, angular and middle temporal gyrus. However, these studies usually comprised only self-generated actions. Therefore, they might have induced not only action-based prediction errors, but also general sensory mismatch errors. Here, we aimed to disentangle these processes using a custom-made fMRI-compatible movement device, generating active and passive hand movements with identical sensory feedback. Online visual feedback of the hand was presented with a variable delay. Participants had to judge whether the feedback was delayed. Activity in the right cerebellum correlated more positively with delay in active than in passive trials. Interestingly, we also observed activation in the angular and middle temporal gyri, but across both active and passive conditions. This suggests that the cerebellum is a comparator area specific to voluntary action, whereas angular and middle temporal gyri seem to detect more general intersensory conflict. Correlations with behavior and cerebellar activity nevertheless suggest involvement of these temporoparietal areas in processing and awareness of temporal discrepancies in action-feedback monitoring.

Incidental or Intentional? Different Brain Responses to One's Own Action Sounds in Hurdling vs. Tap Dancing

Most human actions produce concomitant sounds. Action sounds can be either part of the action goal (GAS, goal-related action sounds), as for instance in tap dancing, or a mere by-product of the action (BAS, by-product action sounds), as for instance in hurdling. It is currently unclear whether these two types of action sounds-incidental or intentional-differ in their neural representation and whether the impact on the performance evaluation of an action diverges between the two. We here examined whether during the observation of tap dancing compared to hurdling, auditory information is a more important factor for positive action quality ratings. Moreover, we tested whether observation of tap dancing vs. hurdling led to stronger attenuation in primary auditory cortex, and a stronger mismatch signal when sounds do not match our expectations. We recorded individual point-light videos of newly trained participants performing tap dancing and hurdling. In the subsequent functional magnetic resonance imaging (fMRI) session, participants were presented with the videos that displayed their own actions, including corresponding action sounds, and were asked to rate the quality of their performance. Videos were either in their original form or scrambled regarding the visual modality, the auditory modality, or both. As hypothesized, behavioral results showed significantly lower rating scores in the GAS condition compared to the BAS condition when the auditory modality was scrambled. Functional MRI contrasts between BAS and GAS actions revealed higher activation of primary auditory cortex in the BAS condition, speaking in favor of stronger attenuation in GAS, as well as stronger activation of posterior superior temporal gyri and the supplementary motor area in GAS. Results suggest that the processing of self-generated action sounds depends on whether we have the intention to produce a sound with our action or not, and action sounds may be more prone to be used as sensory feedback when they are part of the explicit action goal. Our findings contribute to a better understanding of the function of action sounds for learning and controlling sound-producing actions.

Predictive coding in a multisensory path integration task: An fMRI study

During self-motion through an environment, our sensory systems are confronted with a constant flow of information from different modalities. To successfully navigate, self-induced sensory signals have to be dissociated from externally induced sensory signals. Previous studies have suggested that the processing of self-induced sensory information is modulated by means of predictive coding mechanisms. However, the neural correlates of processing self-induced sensory information from different modalities during self-motion are largely unknown. Here, we asked if and how the processing of visually simulated self-motion and/or associated auditory stimuli is modulated by self-controlled action. Participants were asked to actively reproduce a previously observed simulated self-displacement (path integration). Blood oxygen level-dependent (BOLD) activation during this path integration was compared with BOLD activation during a condition in which we passively replayed the exact sensory stimulus that had been produced by the participants in previous trials. We found supramodal BOLD suppression in parietal and frontal regions. Remarkably, BOLD contrast in sensory areas was enhanced in a modality-specific manner. We conclude that the effect of action on sensory processing is strictly dependent on the respective behavioral task and its relevance.

Neuronal Development of Hearing and Language: Cochlear Implants and Critical Periods

The modern cochlear implant (CI) is the most successful neural prosthesis developed to date. CIs provide hearing to the profoundly hearing impaired and allow the acquisition of spoken language in children born deaf. Results from studies enabled by the CI have provided new insights into ( a) minimal representations at the periphery for speech reception, ( b) brain mechanisms for decoding speech presented in quiet and in acoustically adverse conditions, ( c) the developmental neuroscience of language and hearing, and ( d) the mechanisms and time courses of intramodal and cross-modal plasticity. Additionally, the results have underscored the interconnectedness of brain functions and the importance of top-down processes in perception and learning. The findings are described in this review with emphasis on the developing brain and the acquisition of hearing and spoken language.

Assessing Cerebral White Matter Microstructure in Children With Congenital Sensorineural Hearing Loss: A Tract-Based Spatial Statistics Study

Objectives

To assess the microstructural properties of cerebral white matter in children with congenital sensorineural hearing loss (CSNHL).

Methods

Children (>4 years of age) with profound CSNHL and healthy controls with normal hearing (the control group) were enrolled and underwent brain magnetic resonance imaging (MRI) scans with diffusion tensor imaging (DTI). DTI parameters including fractional anisotropy, mean diffusivity, axial diffusivity, and radial diffusivity were obtained from a whole-brain tract-based spatial statistics analysis and were compared between the two groups. In addition, a region of interest (ROI) approach focusing on auditory cortex, i.e., Heschl’s gyrus, using visual cortex, i.e., forceps major as an internal control, was performed. Correlations between mean DTI values and age were obtained with the ROI method.

Results

The study cohort consisted of 23 children with CSHNL (11 boys and 12 girls; mean age ± SD: 7.21 ± 2.67 years; range: 4.1–13.5 years) and 18 children in the control group (11 boys and 7 girls; mean age ± SD: 10.86 ± 3.56 years; range: 4.5–15.3 years). We found the axial diffusivity values being significantly greater in the left anterior thalamic radiation, right corticospinal tract, and corpus callosum in the CSHNL group than in the control group (p < 0.05). Significantly higher radial diffusivity values in the white matter tracts were noted in the CSHNL group as compared to the control group (p < 0.05). The fractional anisotropy values in the Heschl’s gyrus in the CSNHL group were lower compared to the control group (p = 0.0015). There was significant negative correlation between the mean fractional anisotropy values in Heschl’s gyrus and age in the CSNHL group < 7 years of age (r = −0.59, p = 0.004).

Conclusion

Our study showed higher axial and radial diffusivities in the children affected by CNHNL as compared to the hearing children. We also found lower fractional anisotropy values in the Heschl’s gyrus in the CSNHL group. Furthermore, we identified negative correlation between the fractional anisotropy values and age up to 7 years in the children born deaf. Our study findings suggest that myelination and axonal structure may be affected due to acoustic deprivation. This information may help to monitor hearing rehabilitation in the deaf children.

Orbitofrontal Cortex Neurons Respond to Sound and Activate Primary Auditory Cortex Neurons

Sensory environments change over a wide dynamic range and sensory processing can change rapidly to facilitate stable perception. While rapid changes may occur throughout the sensory processing pathway, cortical changes are believed to profoundly influence perception. Prior stimulation studies showed that orbitofrontal cortex (OFC) can modify receptive fields and sensory coding in A1, but the engagement of OFC during listening and the pathways mediating OFC influences on A1 are unknown. We show in mice that OFC neurons respond to sounds consistent with a role of OFC in audition. We then show in vitro that OFC axons are present in A1 and excite pyramidal and GABAergic cells in all layers of A1 via glutamatergic synapses. Optogenetic stimulation of OFC terminals in A1 in vivo evokes short-latency neural activity in A1 and pairing activation of OFC projections in A1 with sounds alters sound-evoked A1 responses. Together, our results identify a direct connection from OFC to A1 that can excite A1 neurons at the earliest stage of cortical processing, and thereby sculpt A1 receptive fields. These results are consistent with a role for OFC in adjusting to changing behavioral relevance of sensory inputs and modulating A1 receptive fields to enhance sound processing.

Gain control in the sensorimotor system

Coordinated movement depends on constant interaction between neural circuits that produce motor output and those that report sensory consequences. Fundamental to this process are mechanisms for controlling the influence that sensory signals have on motor pathways - for example, reducing feedback gains when they are disruptive and increasing gains when advantageous. Sensory gain control comes in many forms and serves diverse purposes - in some cases sensory input is attenuated to maintain movement stability and filter out irrelevant or self-generated signals, or enhanced to facilitate salient signals for improved movement execution and adaptation. The ubiquitous presence of sensory gain control across species at multiple levels of the nervous system reflects the importance of tuning the impact that feedback information has on behavioral output.

Predicted sensory consequences of voluntary actions modulate amplitude of preceding readiness potentials

Self-generated, voluntary actions, are preceded by a slow negativity in the scalp electroencephalography (EEG) signal recorded from frontal regions (termed 'readiness potential'; RP). This signal, and its lateralized subcomponent (LRP), is mainly regarded as preparatory motor activity associated with the forthcoming voluntary motor act. However, it is not clear whether this neural signature is associated with preparatory motor activity, expectation of its associated sensory consequences, or both. Here we recorded EEG data from 14 healthy subjects while they performed self-paced button presses with their right index and middle fingers. Button-presses with one finger triggered a sound (motor+sound condition), while button-presses with the other finger did not (motor-only condition). Additionally, subjects listened to externally-generated sounds delivered in expected timings (sound-only condition). We found that the RP amplitude (locked to time of button press) was significantly more negative in the motor+sound compared with motor-only conditions. Importantly, no signal negativity was observed prior to expected sound delivery in the sound-only condition. Thus, the differences in RP amplitude between motor+sound and motor-only conditions are beyond differences in mere expectation of a forthcoming auditory sound. Our results suggest that information regarding expected auditory consequences is represented in the RP preceding voluntary action execution.

How Movement Modulates Hearing

Hearing is often viewed as a passive process: Sound enters the ear, triggers a cascade of activity through the auditory system, and culminates in an auditory percept. In contrast to a passive process, motor-related signals strongly modulate the auditory system from the eardrum to the cortex. The motor modulation of auditory activity is most well documented during speech and other vocalizations but also can be detected during a wide variety of other sound-generating behaviors. An influential idea is that these motor-related signals suppress neural responses to predictable movement-generated sounds, thereby enhancing sensitivity to environmental sounds during movement while helping to detect errors in learned acoustic behaviors, including speech and musicianship. Findings in humans, monkeys, songbirds, and mice provide new insights into the circuits that convey motor-related signals to the auditory system, while lending support to the idea that these signals function predictively to facilitate hearing and vocal learning.

Preferential encoding of movement amplitude and speed in the primary motor cortex and cerebellum

Voluntary movements require control of multiple kinematic parameters, a task carried out by a distributed brain architecture. However, it remains unclear whether regions along the motor system encode single, or rather a mixture of, kinematic parameters during action execution. Here, rapid event-related functional magnetic resonance imaging was used to differentiate brain activity along the motor system during the encoding of movement amplitude, duration, and speed. We present cumulative evidence supporting preferential encoding of kinematic parameters along the motor system, based on blood-oxygenation-level dependent signal recorded in a well-controlled single-joint wrist-flexion task. Whereas activity in the left primary motor cortex (M1) showed preferential encoding of movement amplitude, the anterior lobe of the right cerebellum (primarily lobule V) showed preferential encoding of movement speed. Conversely, activity in the left supplementary motor area (SMA), basal ganglia (putamen), and anterior intraparietal sulcus was not preferentially modulated by any specific parameter. We found no preference in peak activation for duration encoding in any of the tested regions. Electromyographic data was mainly modulated by movement amplitude, restricting the distinction between amplitude and muscle force encoding. Together, these results suggest that during single-joint movements, distinct kinematic parameters are controlled by largely distinct brain-regions that work together to produce and control precise movements. Hum Brain Mapp 38:5970-5986, 2017. © 2017 Wiley Periodicals, Inc.

The spatiotemporal pattern of pure tone processing: A single-trial EEG-fMRI study

Although considerable research has been published on pure tone processing, its spatiotemporal pattern is not well understood. Specifically, the link between neural activity in the auditory pathway measured by functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) markers of pure tone processing in the P1, N1, P2, and N4 components is not well established. In this study, we used single-trial EEG-fMRI as a multi-modal fusion approach to integrate concurrently acquired EEG and fMRI data, in order to understand the spatial and temporal aspects of the pure tone processing pathway. Data were recorded from 33 subjects who were presented with stochastically alternating pure tone sequences with two different frequencies: 200 and 6400 Hz. Brain network correlated with trial-to-trial variability of the task-discriminating EEG amplitude was identified. We found that neural responses responding to pure tone perception are spatially along the auditory pathway and temporally divided into three stages: (1) the early stage (P1), wherein activation occurs in the midbrain, which constitutes a part of the low level auditory pathway; (2) the middle stage (N1, P2), wherein correlates were found in areas associated with the posterodorsal auditory pathway, including the primary auditory cortex and the motor cortex; (3) the late stage (N4), wherein correlation was found in the motor cortex. This indicates that trial-by-trial variation in neural activity in the P1, N1, P2, and N4 components reflects the sequential engagement of low- and high-level parts of the auditory pathway for pure tone processing. Our results demonstrate that during simple pure tone listening tasks, regions associated with the auditory pathway transiently correlate with trial-to-trial variability of the EEG amplitude, and they do so on a millisecond timescale with a distinct temporal ordering.

Motor origin of temporal predictions in auditory attention

In behavior, action and perception are inherently interdependent. However, the actual mechanistic contributions of the motor system to sensory processing are unknown. We present neurophysiological evidence that the motor system is involved in predictive timing, a brain function that aligns temporal fluctuations of attention with the timing of events in a task-relevant stream, thus facilitating sensory selection and optimizing behavior. In a magnetoencephalography experiment involving auditory temporal attention, participants had to disentangle two streams of sound on the unique basis of endogenous temporal cues. We show that temporal predictions are encoded by interdependent delta and beta neural oscillations originating from the left sensorimotor cortex, and directed toward auditory regions. We also found that overt rhythmic movements improved the quality of temporal predictions and sharpened the temporal selection of relevant auditory information. This latter behavioral and functional benefit was associated with increased signaling of temporal predictions in right-lateralized frontoparietal associative regions. In sum, this study points at a covert form of auditory active sensing. Our results emphasize the key role of motor brain areas in providing contextual temporal information to sensory regions, driving perceptual and behavioral selection.

Roles of Supplementary Motor Areas in Auditory Processing and Auditory Imagery

Although the supplementary and pre-supplementary motor areas have been intensely investigated in relation to their motor functions, they are also consistently reported in studies of auditory processing and auditory imagery. This involvement is commonly overlooked, in contrast to lateral premotor and inferior prefrontal areas. We argue here for the engagement of supplementary motor areas across a variety of sound categories, including speech, vocalizations, and music, and we discuss how our understanding of auditory processes in these regions relate to findings and hypotheses from the motor literature. We suggest that supplementary and pre-supplementary motor areas play a role in facilitating spontaneous motor responses to sound, and in supporting a flexible engagement of sensorimotor processes to enable imagery and to guide auditory perception.

Hearing and imagining sounds–including speech, vocalizations, and music–can recruit SMA and pre-SMA, which are normally discussed in relation to their motor functions.

Emerging research indicates that individual differences in the structure and function of SMA and pre-SMA can predict performance in auditory perception and auditory imagery tasks.

Responses during auditory processing primarily peak in pre-SMA and in the boundary area between pre-SMA and SMA. This boundary area is crucially involved in the control of speech and vocal production, suggesting that sounds engage this region in an effector-specific manner.

Activating sound-related motor representations in SMA and pre-SMA might facilitate behavioral responses to sounds. This might also support a flexible generation of sensory predictions based on previous experience to enable imagery and guide perception.

Perceived loudness of self-generated sounds is differentially modified by expected sound intensity

Performing actions with sensory consequences modifies physiological and behavioral responses relative to otherwise identical sensory input perceived in a passive manner. It is assumed that such modifications occur through an efference copy sent from motor cortex to sensory regions during performance of voluntary actions. In the auditory domain most behavioral studies report attenuated perceived loudness of self-generated auditory action-consequences. However, several recent behavioral and physiological studies report enhanced responses to such consequences. Here we manipulated the intensity of self-generated and externally-generated sounds and examined the type of perceptual modification (enhancement vs. attenuation) reported by healthy human subjects. We found that when the intensity of self-generated sounds was low, perceived loudness is enhanced. Conversely, when the intensity of self-generated sounds was high, perceived loudness is attenuated. These results might reconcile some of the apparent discrepancies in the reported literature and suggest that efference copies can adapt perception according to the differential sensory context of voluntary actions.