Responses to Pattern-Violating Visual Stimuli Evolve Differently Over Days in Somata and Distal Apical Dendrites

Scientists have long conjectured that the neocortex learns patterns in sensory data to generate top-down predictions of upcoming stimuli. In line with this conjecture, different responses to pattern-matching vs pattern-violating visual stimuli have been observed in both spiking and somatic calcium imaging data. However, it remains unknown whether these pattern-violation signals are different between the distal apical dendrites, which are heavily targeted by top-down signals, and the somata, where bottom-up information is primarily integrated. Furthermore, it is unknown how responses to pattern-violating stimuli evolve over time as an animal gains more experience with them. Here, we address these unanswered questions by analyzing responses of individual somata and dendritic branches of layer 2/3 and layer 5 pyramidal neurons tracked over multiple days in primary visual cortex of awake, behaving female and male mice. We use sequences of Gabor patches with patterns in their orientations to create pattern-matching and pattern-violating stimuli, and two-photon calcium imaging to record neuronal responses. Many neurons in both layers show large differences between their responses to pattern-matching and pattern-violating stimuli. Interestingly, these responses evolve in opposite directions in the somata and distal apical dendrites, with somata becoming less sensitive to pattern-violating stimuli and distal apical dendrites more sensitive. These differences between the somata and distal apical dendrites may be important for hierarchical computation of sensory predictions and learning, since these two compartments tend to receive bottom-up and top-down information, respectively.

Perceptual formant discrimination during speech movement planning

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

Effects of GABAergic and Glutamatergic Inputs on Temporal Prediction Signals in the Primate Cerebellar Nucleus

The cerebellum has been shown to be involved in temporal information processing. We recently demonstrated that neurons in the cerebellar dentate nucleus exhibited periodic activity predicting stimulus timing when monkeys attempted to detect a single omission of isochronous repetitive visual stimulus. In this study, we assessed the relative contribution of signals from Purkinje cells and mossy and climbing fibers to the periodic activity by comparing single neuronal firing before and during local infusion of GABA or glutamate receptor antagonists (gabazine or a mixture of 1,2,3,4-tetrahydro-6-nitro-2,3-dioxo-benzo[f]quinoxaline-7-sulfonamide hydrate (NBQX) and (±)-3-(2-carboxypiperazin-4-yl)-propyl-1-phosphonic acid (CPP)). Gabazine application reduced the magnitude of periodic activity and increased the baseline firing rate in most neurons. In contrast, during the blockade of glutamate receptors, both the magnitude of periodic firing modulation and the baseline activity remained unchanged in the population, while a minority of neurons significantly altered their activity. Furthermore, the amounts of changes in the baseline activity and the magnitude of periodic activity were inversely correlated in the gabazine experiments but not in the NBQX + CPP experiments. We also found that the variation of baseline activity decreased during gabazine application but sometimes increased during the blockade of glutamate receptors. These changes were not observed during prolonged recording without drug administration. These results suggest that the predictive neuronal activity in the dentate nucleus may mainly attribute to the inputs from the cerebellar cortex, while the signals from both mossy fibers and Purkinje cells may play a role in setting the level and variance of baseline activity during the task.

Illusion of control affects ERP amplitude reductions for auditory outcomes of self-generated actions

The reduction of neural responses to self-generated stimuli compared to external stimuli is thought to result from the matching of motor-based sensory predictions and sensory reafferences and to serve the identification of changes in the environment as caused by oneself. The amplitude of the auditory event-related potential (ERP) component N1 seems to closely reflect this matching process, while the later positive component (P2/ P3a) has been associated with judgments of agency, which are also sensitive to contextual top-down information. In this study, we examined the effect of perceived control over sound production on the processing of self-generated and external stimuli, as reflected in these components. We used a new version of a classic two-button choice task to induce different degrees of the illusion of control (IoC) and recorded ERPs for the processing of self-generated and external sounds in a subsequent task. N1 amplitudes were reduced for self-generated compared to external sounds, but not significantly affected by IoC. P2/3a amplitudes were affected by IoC: We found reduced P2/3a amplitudes after a high compared to a low IoC induction training, but only for self-generated, not for external sounds. These findings suggest that prior contextual belief information induced by an IoC affects later processing as reflected in the P2/P3a, possibly for the formation of agency judgments, while early processing reflecting motor-based predictions is not affected.

A Formal Account of Structuring Motor Actions With Sensory Prediction for a Naive Agent

For naive robots to become truly autonomous, they need a means of developing their perceptive capabilities instead of relying on hand crafted models. The sensorimotor contingency theory asserts that such a way resides in learning invariants of the sensorimotor flow. We propose a formal framework inspired by this theory for the description of sensorimotor experiences of a naive agent, extending previous related works. We then use said formalism to conduct a theoretical study where we isolate sufficient conditions for the determination of a sensory prediction function. Furthermore, we also show that algebraic structure found in this prediction can be taken as a proxy for structure on the motor displacements, allowing for the discovery of the combinatorial structure of said displacements. Both these claims are further illustrated in simulations where a toy naive agent determines the sensory predictions of its spatial displacements from its uninterpreted sensory flow, which it then uses to infer the combinatorics of said displacements.

Sensory Prediction of Limb Movement Is Critical for Automatic Online Control

Fast, online control of movement is an essential component of human motor skills, as it allows automatic correction of inaccurate planning. The present study explores the role of two types of concurrent signals in error correction: predicted visual reafferences coming from an internal representation of the hand, and actual visual feedback from the hand. While the role of sensory feedback in these corrections is well-established, much less is known about sensory prediction. The relative contributions of these two types of signals remain a subject of debate, as they are naturally interconnected. We address the issue in a study that compares online correction of an artificially induced, undetected planning error. Two conditions are tested, which only differ with respect to the accuracy of predicted visual reafferences. In the first, “Prism” experiment, a planning error is introduced by prisms that laterally displace the seen hand prior to hand movement onset. The prism-induced conflict between visual and proprioceptive inputs of the hand also generates an erroneous prediction of visual reafferences of the moving hand. In the second, “Jump” experiment, a planning error is introduced by a jump in the target position, during the orienting saccade, prior to hand movement onset. In the latter condition, predicted reafferences of the hand remained intact. In both experiments, after hand movement onset, the hand was either visible or hidden, which enabled us to manipulate the presence (or absence) of visual feedback during movement execution. The Prism experiment highlighted late and reduced correction of the planning error, even when natural visual feedback of the moving hand was available. In the Jump experiment, early and automatic corrections of the planning error were observed, even in the absence of visual feedback from the moving hand. Therefore, when predicted reafferences were accurate (the Jump experiment), visual feedback was processed rapidly and automatically. When they were erroneous (the Prism experiment), the same visual feedback was less efficient, and required voluntary, and late, control. Our study clearly demonstrates that in natural environments, reliable prediction is critical in the preprocessing of visual feedback, for fast and accurate movement.

Functional Connectivity between the Cerebellum and Somatosensory Areas Implements the Attenuation of Self-Generated Touch

Since the early 1970s, numerous behavioral studies have shown that self-generated touch feels less intense and less ticklish than the same touch applied externally. Computational motor control theories have suggested that cerebellar internal models predict the somatosensory consequences of our movements and that these predictions attenuate the perception of the actual touch. Despite this influential theoretical framework, little is known about the neural basis of this predictive attenuation. This is due to the limited number of neuroimaging studies, the presence of conflicting results about the role and the location of cerebellar activity, and the lack of behavioral measures accompanying the neural findings. Here, we combined psychophysics with fMRI to detect the neural processes underlying somatosensory attenuation in male and female healthy human participants. Activity in bilateral secondary somatosensory areas was attenuated when the touch was presented during a self-generated movement (self-generated touch) than in the absence of movement (external touch). An additional attenuation effect was observed in the cerebellum that is ipsilateral to the passive limb receiving the touch. Importantly, we further found that the degree of functional connectivity between the ipsilateral cerebellum and the contralateral primary and bilateral secondary somatosensory areas was linearly and positively related to the degree of behaviorally assessed attenuation; that is, the more participants perceptually attenuated their self-generated touches, the stronger this corticocerebellar coupling. Collectively, these results suggest that the ipsilateral cerebellum is fundamental in predicting self-generated touch and that this structure implements somatosensory attenuation via its functional connectivity with somatosensory areas.SIGNIFICANCE STATEMENT When we touch our hand with the other, the resulting sensation feels less intense than when another person or a machine touches our hand with the same intensity. Early computational motor control theories have proposed that the cerebellum predicts and cancels the sensory consequences of our movements; however, the neural correlates of this cancelation remain unknown. By means of fMRI, we show that the more participants attenuate the perception of their self-generated touch, the stronger the functional connectivity between the cerebellum and the somatosensory cortical areas. This provides conclusive evidence about the role of the cerebellum in predicting the sensory feedback of our movements and in attenuating the associated percepts via its connections to early somatosensory areas.

Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain

Temporal orienting improves sensory processing, akin to other top-down biases. However, it is unknown whether these improvements reflect increased neural gain to any stimuli presented at expected time points, or specific tuning to task-relevant stimulus aspects. Furthermore, while other top-down biases are selective, the extent of trade-offs across time is less well characterized. Here, we tested whether gain and/or tuning of auditory frequency processing in humans is modulated by rhythmic temporal expectations, and whether these modulations are specific to time points relevant for task performance. Healthy participants (N = 23) of either sex performed an auditory discrimination task while their brain activity was measured using magnetoencephalography/electroencephalography (M/EEG). Acoustic stimulation consisted of sequences of brief distractors interspersed with targets, presented in a rhythmic or jittered way. Target rhythmicity not only improved behavioral discrimination accuracy and M/EEG-based decoding of targets, but also of irrelevant distractors preceding these targets. To explain this finding in terms of increased sensitivity and/or sharpened tuning to auditory frequency, we estimated tuning curves based on M/EEG decoding results, with separate parameters describing gain and sharpness. The effect of rhythmic expectation on distractor decoding was linked to gain increase only, suggesting increased neural sensitivity to any stimuli presented at relevant time points.SIGNIFICANCE STATEMENT Being able to predict when an event may happen can improve perception and action related to this event, likely due to the alignment of neural activity to the temporal structure of stimulus streams. However, it is unclear whether rhythmic increases in neural sensitivity are specific to task-relevant targets, and whether they competitively impair stimulus processing at unexpected time points. By combining magnetoencephalography and encephalographic recordings, neural decoding of auditory stimulus features, and modeling, we found that rhythmic expectation improved neural decoding of both relevant targets and irrelevant distractors presented and expected time points, but did not competitively impair stimulus processing at unexpected time points. Using a quantitative model, these results were linked to nonspecific neural gain increases due to rhythmic expectation.

Learning to Expect: Predicting Sounds During Movement Is Related to Sensorimotor Association During Listening

Sensory experiences, such as sound, often result from our motor actions. Over time, repeated sound-producing performance can generate sensorimotor associations. However, it is not clear how sensory and motor information are associated. Here, we explore if sensory prediction is associated with the formation of sensorimotor associations during a learning task. We recorded event-related potentials (ERPs) while participants produced index and little finger-swipes on a bespoke device, generating novel sounds. ERPs were also obtained as participants heard those sounds played back. Peak suppression was compared to assess sensory prediction. Additionally, transcranial magnetic stimulation (TMS) was used during listening to generate finger-motor evoked potentials (MEPs). MEPs were recorded before and after training upon hearing these sounds, and then compared to reveal sensorimotor associations. Finally, we explored the relationship between these components. Results demonstrated that an increased positive-going peak (e.g., P2) and a suppressed negative-going peak (e.g., N2) were recorded during action, revealing some sensory prediction outcomes (P2: p = 0.050, ηp2 = 0.208; N2: p = 0.001, ηp2 = 0.474). Increased MEPs were also observed upon hearing congruent sounds compared with incongruent sounds (i.e., associated to a finger), demonstrating precise sensorimotor associations that were not present before learning (Index finger: p < 0.001, ηp2 = 0.614; Little finger: p < 0.001, ηp2 = 0.529). Consistent with our broad hypotheses, a negative association between the MEPs in one finger during listening and ERPs during performance of the other was observed (Index finger MEPs and Fz N1 action ERPs; r = −0.655, p = 0.003). Overall, data suggest that predictive mechanisms are associated with the fine-tuning of sensorimotor associations.

Attenuation of Responses to Self-Generated Sounds in Auditory Cortical Neurons

Many of the sounds that we perceive are caused by our own actions, for example when speaking or moving, and must be distinguished from sounds caused by external events. Studies using macroscopic measurements of brain activity in human subjects have consistently shown that responses to self-generated sounds are attenuated in amplitude. However, the underlying manifestation of this phenomenon at the cellular level is not well understood. To address this, we recorded the activity of neurons in the auditory cortex of mice in response to sounds generated by their own behavior. We found that the responses of auditory cortical neurons to these self-generated sounds were consistently attenuated, compared with the same sounds generated independently of the animals' behavior. This effect was observed in both putative pyramidal neurons and in interneurons and was stronger in lower layers of auditory cortex. Downstream of the auditory cortex, we found that responses of hippocampal neurons to self-generated sounds were almost entirely suppressed. Responses to self-generated optogenetic stimulation of auditory thalamocortical terminals were also attenuated, suggesting a cortical contribution to this effect. Further analyses revealed that the attenuation of self-generated sounds was not simply due to the nonspecific effects of movement or behavioral state on auditory responsiveness. However, the strength of attenuation depended on the degree to which self-generated sounds were expected to occur, in a cell-type-specific manner. Together, these results reveal the cellular basis underlying attenuated responses to self-generated sounds and suggest that predictive processes contribute to this effect.

SIGNIFICANCE STATEMENT

Distinguishing self-generated from externally generated sensory input poses a fundamental problem for behaving organisms. Our study in mice shows for the first time that responses of auditory cortical neurons are attenuated to sounds generated manually by the animals' own behavior. This effect is distinct from the nonspecific effect of behavioral activity on auditory responsiveness that has previously been reported and its magnitude is modulated by the probability with which self-generated sounds occur, suggesting an underlying predictive process. We also reveal how this effect varies across cell types and cortical layers. These findings lay a foundation for studying impairments in the processing of self-generated sounds, which are observed in psychiatric illness, in animal disease models.

The role of expectation in multisensory body representation - neural evidence

Sensory events contribute to body ownership, the feeling that the body belongs to me. However, the encoding of sensory events is not only reactive, but also proactive in that our brain generates prediction about forthcoming stimuli. In previous studies, we have shown that prediction of sensory events is a sufficient condition to induce the sense of body ownership. In this study, we investigated the underlying neural mechanisms. Participants were seated with their right arm resting upon a table just below another smaller table. Hence, the real hand was hidden from the participant's view and a life-sized rubber model of a right hand was placed on the small table in front of them. Participants observed a wooden plank while approaching - without touching - the rubber hand. We measured the phenomenology of the illusion by means of questionnaire. Neural activity was recorded by means of near-infrared spectroscopy (fNIRS). Results showed higher activation of multisensory parietal cortices in the rubber hand illusion induced by touch expectation. Furthermore, such activity was correlated with the subjective feeling of owning the rubber hand. Our results enrich current models of body ownership suggesting that our multisensory brain regions generate prediction on what could be my body and what could not. This finding might have interesting implications in all those cases in which body representation is altered, anorexia, bulimia nervosa and obesity, among others.

Unpredicted Pitch Modulates Beta Oscillatory Power during Rhythmic Entrainment to a Tone Sequence

Extracting temporal regularities in external stimuli in order to predict upcoming events is an essential aspect of perception. Fluctuations in induced power of beta band (15–25 Hz) oscillations in auditory cortex are involved in predictive timing during rhythmic entrainment, but whether such fluctuations are affected by prediction in the spectral (frequency/pitch) domain remains unclear. We tested whether unpredicted (i.e., unexpected) pitches in a rhythmic tone sequence modulate beta band activity by recording EEG while participants passively listened to isochronous auditory oddball sequences with occasional unpredicted deviant pitches at two different presentation rates. The results showed that the power in low-beta (15–20 Hz) was larger around 200–300 ms following deviant tones compared to standard tones, and this effect was larger when the deviant tones were less predicted. Our results suggest that the induced beta power activities in auditory cortex are consistent with a role in sensory prediction of both “when” (timing) upcoming sounds will occur as well as the prediction precision error of “what” (spectral content in this case). We suggest, further, that both timing and content predictions may co-modulate beta oscillations via attention. These findings extend earlier work on neural oscillations by investigating the functional significance of beta oscillations for sensory prediction. The findings help elucidate the functional significance of beta oscillations in perception.

Parallel processing streams for motor output and sensory prediction during action preparation

Sensory consequences of one's own actions are perceived as less intense than identical, externally generated stimuli. This is generally taken as evidence for sensory prediction of action consequences. Accordingly, recent theoretical models explain this attenuation by an anticipatory modulation of sensory processing prior to stimulus onset (Roussel et al. 2013) or even action execution (Brown et al. 2013). Experimentally, prestimulus changes that occur in anticipation of self-generated sensations are difficult to disentangle from more general effects of stimulus expectation, attention and task load (performing an action). Here, we show that an established manipulation of subjective agency over a stimulus leads to a predictive modulation in sensory cortex that is independent of these factors. We recorded magnetoencephalography while subjects performed a simple action with either hand and judged the loudness of a tone caused by the action. Effector selection was manipulated by subliminal motor priming. Compatible priming is known to enhance a subjective experience of agency over a consequent stimulus (Chambon and Haggard 2012). In line with this effect on subjective agency, we found stronger sensory attenuation when the action that caused the tone was compatibly primed. This perceptual effect was reflected in a transient phase-locked signal in auditory cortex before stimulus onset and motor execution. Interestingly, this sensory signal emerged at a time when the hemispheric lateralization of motor signals in M1 indicated ongoing effector selection. Our findings confirm theoretical predictions of a sensory modulation prior to self-generated sensations and support the idea that a sensory prediction is generated in parallel to motor output (Walsh and Haggard 2010), before an efference copy becomes available.

Alteration in forward model prediction of sensory outcome of motor action in focal hand dystonia

Focal hand dystonia in musicians is a movement disorder affecting highly trained movements. Rather than being a pure motor disorder related to movement execution only, movement planning, error prediction, and sensorimotor integration are also impaired. Internal models (IMs), of which two types, forward and inverse models have been described and most likely processed in the cerebellum, are known to be involved in these tasks. Recent results indicate that the cerebellum may be involved in the pathophysiology of focal dystonia (FD). Thus, the aim of our study was to investigate whether an IM deficit plays a role in FD. We focused on the forward model (FM), which predicts sensory consequences of motor commands and allows the discrimination between external sensory input and input deriving from motor action. We investigated 19 patients, aged 19-59 and 19 healthy musicians aged 19-36 as controls. Tactile stimuli were applied to fingers II-V of both hands by the experimenter or the patient. After each stimulus the participant rated the stimulus intensity on a scale between 0 (no sensation) and 1 (maximal intensity). The difference of perceived intensity between self- and externally applied (EA) stimuli was then calculated for each finger. For assessing differences between patients and controls we performed a cluster analysis of the affected hand and the corresponding hand of the controls using the fingers II-V as variables in a 4-dimensional hyperspace (chance level = 0.5). Using a cluster analysis, we found a correct classification of the affected finger in 78.9-94.7%. There was no difference between patients and healthy controls of the absolute value of the perceived stimulus intensity. Our results suggest an altered FM function in focal hand dystonia. It has the potential of suggesting a neural correlate within the cerebellum and of helping integrate findings with regard to altered sensorimotor processing and altered prediction in FD in a single framework.