Task relevance modulates the behavioural and neural effects of sensory predictions

Precision-dependent modulation of social attention

Social attention, guided by cues like gaze direction, is crucial for effective social interactions. However, how dynamic environmental context modulates this process remains unclear. Integrating a hierarchical Bayesian model with fMRI, this study investigated how individuals adjusted attention based on the predictions about cue validity (CV). Thirty-three participants performed a modified Posner location-cueing task with varying CV. Behaviorally, individuals' allocation of social attention was finely tuned to the precision (inverse variance) of CV predictions, with the predictions updated by precision-weighted prediction errors (PEs) about the occurrence of target locations. Neuroimaging results revealed that the interaction between allocation of social attention and CV influenced activity in regions involved in spatial attention and/or social perception. Precision-weighted PEs about target locations specifically modulated activity in the temporoparietal junction (TPJ), superior temporal sulcus (STS), and primary visual cortex (V1), underscoring their roles in refining attentional predictions. Dynamic causal modeling (DCM) further demonstrated that enhanced absolute precision-weighted PEs about target locations strengthened the effective connectivity from V1 and STS to TPJ, emphasizing their roles in conveying residual error signals upwards to high-level critical attention areas. These findings emphasized the pivotal role of precision in attentional modulation, enhancing our understanding of context-dependent social attention.

Bayesian prior uncertainty and surprisal elicit distinct neural patterns during sound localization in dynamic environments

Estimating the location of a stimulus is a key function in sensory processing, and widely considered to result from the integration of prior information and sensory input according to Bayesian principles. A deviation of sensory input from the prior elicits surprisal, depending on the uncertainty of the prior. While this mechanism is increasingly understood in the visual domain, much less is known about its implementation in audition, especially regarding spatial localization. Here, we combined human EEG with computational modeling to study auditory spatial inference in a noisy, volatile environment and analyzed behavioral and neural patterns associated with prior uncertainty and surprisal. First, our results demonstrate that participants indeed used prior information during periods of stable environmental statistics, but showed evidence of surprisal and discarded prior information following environmental changes. Second, we observed distinct EEG activity patterns associated with prior uncertainty and surprisal in both the time- and time-frequency domain, which are in line with previous studies using visual tasks. Third, these EEG activity patterns were predictive of our participants' sound localization error, response uncertainty, and prior bias on a trial-by-trial basis. In summary, our work provides novel behavioral and neural evidence for Bayesian inference during dynamic auditory localization.

Predictive coding: a more cognitive process than we thought?

In predictive coding (PC), higher-order brain areas generate predictions that are sent to lower-order sensory areas. Top-down predictions are compared with bottom-up sensory data, and mismatches evoke prediction errors. In PC, the prediction errors are encoded in layer 2/3 pyramidal neurons of sensory cortex that feed forward. The PC model has been tested with multiple recording modalities using the global-local oddball paradigm. Consistent with PC, neuroimaging studies reported prediction error responses in sensory and higher-order areas. However, recent studies of neuronal spiking suggest that genuine prediction errors emerge in prefrontal cortex (PFC). This implies that predictive processing is a more cognitive than sensory-based mechanism - an observation that challenges PC and better aligns with a framework we call predictive routing (PR).

Frequency-specific changes in prefrontal activity associated with maladaptive belief updating in volatile environments in euthymic bipolar disorder

Bipolar disorder (BD) involves altered reward processing and decision-making, with inconsistencies across studies. Here, we integrated hierarchical Bayesian modelling with magnetoencephalography (MEG) to characterise maladaptive belief updating in this condition. First, we determined if previously reported increased learning rates in BD stem from a heightened expectation of environmental changes. Additionally, we examined if this increased expectation speeds up belief updating in decision-making, associated with modulation of rhythmic neural activity within the prefrontal, orbitofrontal, and anterior cingulate cortex (PFC, OFC, ACC). Twenty-two euthymic BD and 27 healthy control (HC) participants completed a reward-based motor decision-making task in a volatile setting. Hierarchical Bayesian modelling revealed BD participants anticipated greater environmental volatility, resulting in a more stochastic mapping from beliefs to actions and paralleled by lower win rates and a reduced tendency to repeat rewarded actions than HC. Despite this, BD individuals adjusted their expectations of action-outcome contingencies more slowly, but both groups invigorated their actions similarly. On a neural level, while healthy individuals exhibited an alpha-beta suppression and gamma increase during belief updating, BD participants showed dampened effects, extending across the PFC, OFC, and ACC regions. This was accompanied by an abnormally increased beta-band directed information flow in BD. Overall, the results suggest euthymic BD individuals anticipate environmental change without adequately learning from it, contributing to maladaptive belief updating. Alterations in frequency-domain amplitude and functional connectivity within the PFC, OFC, and ACC during belief updating underlie the computational effects and could serve as potential indicators for predicting relapse in future research.

Model-Based Approaches to Investigating Mismatch Responses in Schizophrenia

Alterations of mismatch responses (ie, neural activity evoked by unexpected stimuli) are often considered a potential biomarker of schizophrenia. Going beyond establishing the type of observed alterations found in diagnosed patients and related cohorts, computational methods can yield valuable insights into the underlying disruptions of neural mechanisms and cognitive function. Here, we adopt a typology of model-based approaches from computational cognitive neuroscience, providing an overview of the study of mismatch responses and their alterations in schizophrenia from four complementary perspectives: (a) connectivity models, (b) decoding models, (c) neural network models, and (d) cognitive models. Connectivity models aim at inferring the effective connectivity patterns between brain regions that may underlie mismatch responses measured at the sensor level. Decoding models use multivariate spatiotemporal mismatch response patterns to infer the type of sensory violations or to classify participants based on their diagnosis. Neural network models such as deep convolutional neural networks can be used for improved classification performance as well as for a systematic study of various aspects of empirical data. Finally, cognitive models quantify mismatch responses in terms of signaling and updating perceptual predictions over time. In addition to describing the available methodology and reviewing the results of recent computational psychiatry studies, we offer suggestions for future work applying model-based techniques to advance the study of mismatch responses in schizophrenia.

The brain's duck test in phantom percepts: Multisensory congruence in neuropathic pain and tinnitus

Chronic neuropathic pain and chronic tinnitus have been likened to phantom percepts, in which a complete or partial sensory deafferentation results in a filling in of the missing information derived from memory. 150 participants, 50 with tinnitus, 50 with chronic pain and 50 healthy controls underwent a resting state EEG. Source localized current density is recorded from all the sensory cortices (olfactory, gustatory, somatosensory, auditory, vestibular, visual) as well as the parahippocampal area. Functional connectivity by means of lagged phase synchronization is also computed between these regions of interest. Pain and tinnitus are associated with gamma band activity, reflecting prediction errors, in all sensory cortices except the olfactory and gustatory cortex. Functional connectivity identifies theta frequency connectivity between each of the sensory cortices except the chemical senses to the parahippocampus, but not between the individual sensory cortices. When one sensory domain is deprived, the other senses may provide the parahippocampal 'contextual' area with the most likely sound or somatosensory sensation to fill in the gap, applying an abductive 'duck test' approach, i.e., based on stored multisensory congruence. This novel concept paves the way to develop novel treatments for pain and tinnitus, using multisensory (i.e. visual, vestibular, somatosensory, auditory) modulation with or without associated parahippocampal targeting.

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Acetylcholine modulates the precision of prediction error in the auditory cortex

A fundamental property of sensory systems is their ability to detect novel stimuli in the ambient environment. The auditory brain contains neurons that decrease their response to repetitive sounds but increase their firing rate to novel or deviant stimuli; the difference between both responses is known as stimulus-specific adaptation or neuronal mismatch (nMM). Here, we tested the effect of microiontophoretic applications of ACh on the neuronal responses in the auditory cortex (AC) of anesthetized rats during an auditory oddball paradigm, including cascade controls. Results indicate that ACh modulates the nMM, affecting prediction error responses but not repetition suppression, and this effect is manifested predominantly in infragranular cortical layers. The differential effect of ACh on responses to standards, relative to deviants (in terms of averages and variances), was consistent with the representational sharpening that accompanies an increase in the precision of prediction errors. These findings suggest that ACh plays an important role in modulating prediction error signaling in the AC and gating the access of these signals to higher cognitive levels.

A parahippocampal-sensory Bayesian vicious circle generates pain or tinnitus: a source-localized EEG study

Pain and tinnitus share common pathophysiological mechanisms, clinical features, and treatment approaches. A source-localized resting-state EEG study was conducted in 150 participants: 50 healthy controls, 50 pain, and 50 tinnitus patients. Resting-state activity as well as functional and effective connectivity was computed in source space. Pain and tinnitus were characterized by increased theta activity in the pregenual anterior cingulate cortex, extending to the lateral prefrontal cortex and medial anterior temporal lobe. Gamma-band activity was increased in both auditory and somatosensory cortex, irrespective of the pathology, and extended to the dorsal anterior cingulate cortex and parahippocampus. Functional and effective connectivity were largely similar in pain and tinnitus, except for a parahippocampal-sensory loop that distinguished pain from tinnitus. In tinnitus, the effective connectivity between parahippocampus and auditory cortex is bidirectional, whereas the effective connectivity between parahippocampus and somatosensory cortex is unidirectional. In pain, the parahippocampal-somatosensory cortex is bidirectional, but parahippocampal auditory cortex unidirectional. These modality-specific loops exhibited theta-gamma nesting. Applying a Bayesian brain model of brain functioning, these findings suggest that the phenomenological difference between auditory and somatosensory phantom percepts result from a vicious circle of belief updating in the context of missing sensory information. This finding may further our understanding of multisensory integration and speaks to a universal treatment for pain and tinnitus-by selectively disrupting parahippocampal-somatosensory and parahippocampal-auditory theta-gamma activity and connectivity.

Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans

Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.

Top-down specific preparatory activations for selective attention and perceptual expectations

Proactive cognition brain models are mainstream nowadays. Within these, preparation is understood as an endogenous, top-down function that takes place prior to the actual perception of a stimulus and improves subsequent behavior. Neuroimaging has shown the existence of such preparatory activity separately in different cognitive domains, however no research to date has sought to uncover their potential similarities and differences. Two of these, often confounded in the literature, are Selective Attention (information relevance) and Perceptual Expectation (information probability). We used EEG to characterize the mechanisms that pre-activate specific contents in Attention and Expectation. In different blocks, participants were cued to the relevance or to the probability of target categories, faces vs. names, in a gender discrimination task. Multivariate Pattern (MVPA) and Representational Similarity Analyses (RSA) during the preparation window showed that both manipulations led to a significant, ramping-up prediction of the relevant or expected target category. However, classifiers trained with data from one condition did not generalize to the other, indicating the existence of unique anticipatory neural patterns. In addition, a Canonical Template Tracking procedure showed that there was stronger anticipatory perceptual reinstatement for relevance than for expectation blocks. Overall, the results indicate that preparation during attention and expectation acts through distinguishable neural mechanisms. These findings have important implications for current models of brain functioning, as they are a first step towards characterizing and dissociating the neural mechanisms involved in top-down anticipatory processing.

Reaction Time "Mismatch Costs" Change with the Likelihood of Stimulus-Response Compatibility

Dyadic interactions require dynamic correspondence between one's own movements and those of the other agent. This mapping is largely viewed as imitative, with the behavioural hallmark being a reaction-time cost for mismatched actions. Yet the complex motor patterns humans enact together extend beyond direct-matching, varying adaptively between imitation, complementary movements, and counter-imitation. Optimal behaviour requires an agent to predict not only what is likely to be observed but also how that observed action will relate to their own motor planning. In 28 healthy adults, we examined imitation and counter-imitation in a task that varied the likelihood of stimulus-response congruence from highly predictable, to moderately predictable, to unpredictable. To gain mechanistic insights into the statistical learning of stimulus-response compatibility, we compared two computational models of behaviour: (1) a classic fixed learning-rate model (Rescorla-Wagner reinforcement [RW]) and (2) a hierarchical model of perceptual-behavioural processes in which the learning rate adapts to the inferred environmental volatility (hierarchical Gaussian filter [HGF]). Though more complex and hence penalized by model selection, the HGF provided a more likely model of the participants' behaviour. Matching motor responses were only primed (faster) in the most experimentally volatile context. This bias was reversed so that mismatched actions were primed when beliefs about volatility were lower. Inferential statistics indicated that matching responses were only primed in unpredictable contexts when stimuli-response congruence was at 50:50 chance. Outside of these unpredictable blocks the classic stimulus-response compatibility effect was reversed: Incongruent responses were faster than congruent ones. We show that hierarchical Bayesian learning of environmental statistics may underlie response priming during dyadic interactions.

Hierarchical unimodal processing within the primary somatosensory cortex during a bimodal detection task

Do sensory cortices process more than one sensory modality? To answer these questions, scientists have generated a wide variety of studies at distinct space-time scales in different animal models, and often shown contradictory conclusions. Some conclude that this process occurs in early sensory cortices, but others that this occurs in areas central to sensory cortices. Here, we sought to determine whether sensory neurons process and encode physical stimulus properties of different modalities (tactile and acoustic). For this, we designed a bimodal detection task where the senses of touch and hearing compete from trial to trial. Two Rhesus monkeys performed this novel task, while neural activity was recorded in areas 3b and 1 of the primary somatosensory cortex (S1). We analyzed neurons' coding properties and variability, organizing them by their receptive field's position relative to the stimulation zone. Our results indicate that neurons of areas 3b and 1 are unimodal, encoding only the tactile modality in both the firing rate and variability. Moreover, we found that neurons in area 3b carried more information about the periodic stimulus structure than those in area 1, possessed lower response and coding latencies, and had a lower intrinsic time scale. In sum, these differences reveal a hidden processing-based hierarchy. Finally, using a powerful nonlinear dimensionality reduction algorithm, we show that the activity from areas 3b and 1 can be separated, establishing a clear division in the functionality of these two subareas of S1.

Expectations boost the reconstruction of auditory features from electrophysiological responses to noisy speech

Online speech processing imposes significant computational demands on the listening brain, the underlying mechanisms of which remain poorly understood. Here, we exploit the perceptual "pop-out" phenomenon (i.e. the dramatic improvement of speech intelligibility after receiving information about speech content) to investigate the neurophysiological effects of prior expectations on degraded speech comprehension. We recorded electroencephalography (EEG) and pupillometry from 21 adults while they rated the clarity of noise-vocoded and sine-wave synthesized sentences. Pop-out was reliably elicited following visual presentation of the corresponding written sentence, but not following incongruent or neutral text. Pop-out was associated with improved reconstruction of the acoustic stimulus envelope from low-frequency EEG activity, implying that improvements in perceptual clarity were mediated via top-down signals that enhanced the quality of cortical speech representations. Spectral analysis further revealed that pop-out was accompanied by a reduction in theta-band power, consistent with predictive coding accounts of acoustic filling-in and incremental sentence processing. Moreover, delta-band power, alpha-band power, and pupil diameter were all increased following the provision of any written sentence information, irrespective of content. Together, these findings reveal distinctive profiles of neurophysiological activity that differentiate the content-specific processes associated with degraded speech comprehension from the context-specific processes invoked under adverse listening conditions.

State anxiety alters the neural oscillatory correlates of predictions and prediction errors during reward-based learning

Anxiety influences how the brain estimates and responds to uncertainty. The consequences of these processes on behaviour have been described in theoretical and empirical studies, yet the associated neural correlates remain unclear. Rhythm-based accounts of Bayesian predictive coding propose that predictions in generative models of perception are represented in alpha (8-12 Hz) and beta oscillations (13-30 Hz). Updates to predictions are driven by prediction errors weighted by precision (inverse variance) encoded in gamma oscillations (>30 Hz) and associated with the suppression of beta activity. We tested whether state anxiety alters the neural oscillatory activity associated with predictions and precision-weighted prediction errors (pwPE) during learning. Healthy human participants performed a probabilistic reward-based learning task in a volatile environment. In our previous work, we described learning behaviour in this task using a hierarchical Bayesian model, revealing more precise (biased) beliefs about the tendency of the reward contingency in state anxiety, consistent with reduced learning in this group. The model provided trajectories of predictions and pwPEs for the current study, allowing us to assess their parametric effects on the time-frequency representations of EEG data. Using convolution modelling for oscillatory responses, we found that, relative to a control group, state anxiety increased beta activity in frontal and sensorimotor regions during processing of pwPE, and in fronto-parietal regions during encoding of predictions. No effects of state anxiety on gamma modulation were found. Our findings expand prior evidence on the oscillatory representations of predictions and pwPEs into the reward-based learning domain. The results suggest that state anxiety modulates beta-band oscillatory correlates of pwPE and predictions in generative models, providing insights into the neural processes associated with biased belief updating and poorer learning.

Precision control for a flexible body representation

Adaptive body representation requires the continuous integration of multisensory inputs within a flexible 'body model' in the brain. The present review evaluates the idea that this flexibility is augmented by the contextual modulation of sensory processing 'top-down'; which can be described as precision control within predictive coding formulations of Bayesian inference. Specifically, I focus on the proposal that an attenuation of proprioception may facilitate the integration of conflicting visual and proprioceptive bodily cues. Firstly, I review empirical work suggesting that the processing of visual vs proprioceptive body position information can be contextualised 'top-down'; for instance, by adopting specific attentional task sets. Building up on this, I review research showing a similar contextualisation of visual vs proprioceptive information processing in the rubber hand illusion and in visuomotor adaptation. Together, the reviewed literature suggests that proprioception, despite its indisputable importance for body perception and action control, can be attenuated top-down (through precision control) to facilitate the contextual adaptation of the brain's body model to novel visual feedback.

Rhythmic Modulation of Visual Perception by Continuous Rhythmic Auditory Stimulation

At any given moment our sensory systems receive multiple, often rhythmic, inputs from the environment. Processing of temporally structured events in one sensory modality can guide both behavioral and neural processing of events in other sensory modalities, but whether this occurs remains unclear. Here, we used human electroencephalography (EEG) to test the cross-modal influences of a continuous auditory frequency-modulated (FM) sound on visual perception and visual cortical activity. We report systematic fluctuations in perceptual discrimination of brief visual stimuli in line with the phase of the FM-sound. We further show that this rhythmic modulation in visual perception is related to an accompanying rhythmic modulation of neural activity recorded over visual areas. Importantly, in our task, perceptual and neural visual modulations occurred without any abrupt and salient onsets in the energy of the auditory stimulation and without any rhythmic structure in the visual stimulus. As such, the results provide a critical validation for the existence and functional role of cross-modal entrainment and demonstrates its utility for organizing the perception of multisensory stimulation in the natural environment.SIGNIFICANCE STATEMENT Our sensory environment is filled with rhythmic structures that are often multi-sensory in nature. Here, we show that the alignment of neural activity to the phase of an auditory frequency-modulated (FM) sound has cross-modal consequences for vision: yielding systematic fluctuations in perceptual discrimination of brief visual stimuli that are mediated by accompanying rhythmic modulation of neural activity recorded over visual areas. These cross-modal effects on visual neural activity and perception occurred without any abrupt and salient onsets in the energy of the auditory stimulation and without any rhythmic structure in the visual stimulus. The current work shows that continuous auditory fluctuations in the natural environment can provide a pacing signal for neural activity and perception across the senses.

Fear conditioning prompts sparser representations of conditioned threat in primary visual cortex

Repeated exposure to threatening stimuli alters sensory responses. We investigated the underlying neural mechanism by re-analyzing previously published simultaneous electroencephalogram-functional magnetic resonance imaging (EEG-fMRI) data from humans viewing oriented gratings during Pavlovian fear conditioning. In acquisition, one grating (CS+) was paired with a noxious noise, the unconditioned stimulus (US). The other grating (CS-) was never paired with the US. In habituation, which preceded acquisition, and in extinction, the same two gratings were presented without US. Using fMRI multivoxel patterns in primary visual cortex during habituation as reference, we found that during acquisition, aversive learning selectively prompted systematic changes in multivoxel patterns evoked by CS+. Specifically, CS+ evoked voxel patterns in V1 became sparser as aversive learning progressed, and the sparsified pattern appeared to be preserved in extinction. Concomitant with the voxel pattern changes, occipital alpha oscillations were increasingly more desynchronized during CS+ (but not CS-) trials. Across acquisition trials, the rate of change in CS+-related alpha desynchronization was correlated with the rate of change in multivoxel pattern representations of CS+. Furthermore, alpha oscillations co-varied with blood-oxygen-level-dependent (BOLD) data in the ventral attention network, but not with BOLD in the amygdala. Thus, fear conditioning prompts persistent sparsification of voxel patterns evoked by threat, likely mediated by attention-related mechanisms

Neural Correlates of Auditory Pattern Learning in the Auditory Cortex

Learning of new auditory stimuli often requires repetitive exposure to the stimulus. Fast and implicit learning of sounds presented at random times enables efficient auditory perception. However, it is unclear how such sensory encoding is processed on a neural level. We investigated neural responses that are developed from a passive, repetitive exposure to a specific sound in the auditory cortex of anesthetized rats, using electrocorticography. We presented a series of random sequences that are generated afresh each time, except for a specific reference sequence that remains constant and re-appears at random times across trials. We compared induced activity amplitudes between reference and fresh sequences. Neural responses from both primary and non-primary auditory cortical regions showed significantly decreased induced activity amplitudes for reference sequences compared to fresh sequences, especially in the beta band. This is the first study showing that neural correlates of auditory pattern learning can be evoked even in anesthetized, passive listening animal models.

Cortical beta oscillations reflect the contextual gating of visual action feedback

•

We decouple seen and felt hand postures during action via virtual reality.

•

Vision of the hand is either task-relevant or a distractor.

•

Task-relevance of vision is reflected by in- or decreases of occipital beta power.

•

DCM suggests underlying changes in cortical (visual) excitability.

•

Occipital beta may indicate the contextual gating of visual action feedback.

In sensorimotor integration, the brain needs to decide how its predictions should accommodate novel evidence by ‘gating’ sensory data depending on the current context. Here, we examined the oscillatory correlates of this process by recording magnetoencephalography (MEG) data during a new task requiring action under intersensory conflict. We used virtual reality to decouple visual (virtual) and proprioceptive (real) hand postures during a task in which the phase of grasping movements tracked a target (in either modality). Thus, we rendered visual information either task-relevant or a (to-be-ignored) distractor. Under visuo-proprioceptive incongruence, occipital beta power decreased (relative to congruence) when vision was task-relevant but increased when it had to be ignored. Dynamic causal modeling (DCM) revealed that this interaction was best explained by diametrical, task-dependent changes in visual gain. These results suggest a crucial role for beta oscillations in the contextual gating (i.e., gain or precision control) of visual vs proprioceptive action feedback, depending on current behavioral demands.

Ketamine Affects Prediction Errors about Statistical Regularities: A Computational Single-Trial Analysis of the Mismatch Negativity

The auditory mismatch negativity (MMN) is significantly reduced in schizophrenia. Notably, a similar MMN reduction can be achieved with NMDA receptor (NMDAR) antagonists. Both phenomena have been interpreted as reflecting an impairment of predictive coding or, more generally, the "Bayesian brain" notion that the brain continuously updates a hierarchical model to infer the causes of its sensory inputs. Specifically, neurobiological interpretations of predictive coding view perceptual inference as an NMDAR-dependent process of minimizing hierarchical precision-weighted prediction errors (PEs), and disturbances of this putative process play a key role in hierarchical Bayesian theories of schizophrenia. Here, we provide empirical evidence for this theory, demonstrating the existence of multiple, hierarchically related PEs in a "roving MMN" paradigm. We applied a hierarchical Bayesian model to single-trial EEG data from healthy human volunteers of either sex who received the NMDAR antagonist S-ketamine in a placebo-controlled, double-blind, within-subject fashion. Using an unrestricted analysis of the entire time-sensor space, our trial-by-trial analysis indicated that low-level PEs (about stimulus transitions) are expressed early (102-207 ms poststimulus), while high-level PEs (about transition probability) are reflected by later components (152-199 and 215-277 ms) of single-trial responses. Furthermore, we find that ketamine significantly diminished the expression of high-level PE responses, implying that NMDAR antagonism disrupts the inference on abstract statistical regularities. Our findings suggest that NMDAR dysfunction impairs hierarchical Bayesian inference about the world's statistical structure. Beyond the relevance of this finding for schizophrenia, our results illustrate the potential of computational single-trial analyses for assessing potential pathophysiological mechanisms.

No Evidence that Predictions and Attention Modulate the First Feedforward Sweep of Cortical Information Processing

Predictive coding models propose that predictions (stimulus likelihood) reduce sensory signals as early as primary visual cortex (V1), and that attention (stimulus relevance) can modulate these effects. Indeed, both prediction and attention have been shown to modulate V1 activity, albeit with fMRI, which has low temporal resolution. This leaves it unclear whether these effects reflect a modulation of the first feedforward sweep of visual information processing and/or later, feedback-related activity. In two experiments, we used electroencephalography and orthogonally manipulated spatial predictions and attention to address this issue. Although clear top-down biases were found, as reflected in pre-stimulus alpha-band activity, we found no evidence for top-down effects on the earliest visual cortical processing stage (<80 ms post-stimulus), as indexed by the amplitude of the C1 event-related potential component and multivariate pattern analyses. These findings indicate that initial visual afferent activity may be impenetrable to top-down influences by spatial prediction and attention.

Synchronisation of Neural Oscillations and Cross-modal Influences

At any given moment, we receive multiple signals from our different senses. Prior research has shown that signals in one sensory modality can influence neural activity and behavioural performance associated with another sensory modality. Recent human and nonhuman primate studies suggest that such cross-modal influences in sensory cortices are mediated by the synchronisation of ongoing neural oscillations. In this review, we consider two mechanisms proposed to facilitate cross-modal influences on sensory processing, namely cross-modal phase resetting and neural entrainment. We consider how top-down processes may further influence cross-modal processing in a flexible manner, and we highlight fruitful directions for further research.

Multi-spectral oscillatory dynamics serving directed and divided attention

Attention-related amplification of neural representations of external stimuli has been well documented in the visual domain, however, research concerning the oscillatory dynamics of such directed attention is relatively sparse in humans. Specifically, it is unknown which spectrally-specific neural responses are mainly impacted by the direction and division of attention, as well as whether the effects of attention on these oscillations are spatially disparate. In this study, we use magnetoencephalography and a visual-somatosensory oddball task to investigate the whole-brain oscillatory dynamics of directed (Experiment 1; N = 26) and divided (Experiment 2; N = 34) visual attention. Sensor-level data were transformed into the time-frequency domain and significant responses from baseline were imaged using a frequency-resolved beamformer. We found that multi-spectral cortical oscillations were stronger when attention was sustained in the visual space and that these effects exhibited informative spatial distributions that differed by frequency. More specifically, we found stronger frontal theta (4–8 Hz), frontal and occipital alpha (8–14 Hz), occipital beta (16–22 Hz), and frontal gamma (74–84 Hz) responses when visual attention was sustained than when it was directed away from the visual domain. Similarly, in the divided attention condition, we observed stronger fronto-parietal theta activity and temporo-parietal alpha and beta oscillations when visual attention was sustained toward the visual stimuli than divided between the visual and somatosensory domains. Investigating how attentional gain is implemented in the human brain is essential for better understanding how this process is degraded in disease, and may provide useful targets for future therapies.

Spatial Attention and Temporal Expectation Exert Differential Effects on Visual and Auditory Discrimination

Anticipation of an impending stimulus shapes the state of the sensory systems, optimizing neural and behavioral responses. Here, we studied the role of brain oscillations in mediating spatial and temporal anticipations. Because spatial attention and temporal expectation are often associated with visual and auditory processing, respectively, we directly contrasted the visual and auditory modalities and asked whether these anticipatory mechanisms are similar in both domains. We recorded the magnetoencephalogram in healthy human participants performing an auditory and visual target discrimination task, in which cross-modal cues provided both temporal and spatial information with regard to upcoming stimulus presentation. Motivated by prior findings, we were specifically interested in delta (1-3 Hz) and alpha (8-13 Hz) band oscillatory state in anticipation of target presentation and their impact on task performance. Our findings support the view that spatial attention has a stronger effect in the visual domain, whereas temporal expectation effects are more prominent in the auditory domain. For the spatial attention manipulation, we found a typical pattern of alpha lateralization in the visual system, which correlated with response speed. Providing a rhythmic temporal cue led to increased postcue synchronization of low-frequency rhythms, although this effect was more broadband in nature, suggesting a general phase reset rather than frequency-specific neural entrainment. In addition, we observed delta-band synchronization with a frontal topography, which correlated with performance, especially in the auditory task. Combined, these findings suggest that spatial and temporal anticipations operate via a top-down modulation of the power and phase of low-frequency oscillations, respectively.

Dissociable neural effects of temporal expectations due to passage of time and contextual probability

The human brain is equipped with complex mechanisms to track the changing probability of events in time. While the passage of time itself usually leads to a mounting expectation, context can provide additional information about when events are likely to happen. In this study we dissociate these two sources of temporal expectation in terms of their neural correlates and underlying brain connectivity patterns. We analysed magnetoencephalographic (MEG) data acquired from N = 24 healthy participants listening to auditory stimuli. These stimuli could be presented at different temporal intervals but occurred most often at intermediate intervals, forming a contextual probability distribution. Evoked MEG response amplitude was sensitive to both passage of time (time elapsed since the cue) and contextual probability, albeit at different latencies: the effects of passage of time were observed earlier than the effects of context. The underlying sources of MEG activity were also different across the two types of temporal prediction: the effects of passage of time were localised to early auditory regions and superior temporal gyri, while context was additionally linked to activity in inferior parietal cortices. Finally, these differences were modelled using biophysical (dynamic causal) modelling: passage of time was explained in terms of widespread gain modulation and decreased prediction error signalling at lower levels of the hierarchy, while contextual expectation led to more localised gain modulation and decreased prediction error signalling at higher levels of the hierarchy. These results present a comprehensive account of how independent sources of temporal prediction may be differentially expressed in cortical circuits.

Content-based Dissociation of Hippocampal Involvement in Prediction

Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top-down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli.

Feature-specific prediction errors for visual mismatch

Predictive coding (PC) theory posits that our brain employs a predictive model of the environment to infer the causes of its sensory inputs. A fundamental but untested prediction of this theory is that the same stimulus should elicit distinct precision weighted prediction errors (pwPEs) when different (feature-specific) predictions are violated, even in the absence of attention. Here, we tested this hypothesis using functional magnetic resonance imaging (fMRI) and a multi-feature roving visual mismatch paradigm where rare changes in either color (red, green), or emotional expression (happy, fearful) of faces elicited pwPE responses in human participants. Using a computational model of learning and inference, we simulated pwPE and prediction trajectories of a Bayes-optimal observer and used these to analyze changes in blood oxygen level dependent (BOLD) responses to changes in color and emotional expression of faces while participants engaged in a distractor task. Controlling for visual attention by eye-tracking, we found pwPE responses to unexpected color changes in the fusiform gyrus. Conversely, unexpected changes of facial emotions elicited pwPE responses in cortico-thalamo-cerebellar structures associated with emotion and theory of mind processing. Predictions pertaining to emotions activated fusiform, occipital and temporal areas. Our results are consistent with a general role of PC across perception, from low-level to complex and socially relevant object features, and suggest that monitoring of the social environment occurs continuously and automatically, even in the absence of attention.

Predicting change: Approximate inference under explicit representation of temporal structure in changing environments

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants' beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants' behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior.

Asymmetrical Relationship between Prediction and Control during Visuomotor Adaptation

Current theories suggest that the ability to control the body and to predict its associated sensory consequences is key for skilled motor behavior. It is also suggested that these abilities need to be updated when the mapping between motor commands and sensory consequences is altered. Here we challenge this view by investigating the transfer of adaptation to rotated visual feedback between one task in which human participants had to control a cursor with their hand in order to track a moving target, and another in which they had to predict with their eyes the visual consequences of their hand movement on the cursor. Hand and eye tracking performances were evaluated respectively through cursor–target and eye–cursor distance. Results reveal a striking dissociation: although prior adaptation of hand tracking greatly facilitates eye tracking, the adaptation of eye tracking does not transfer to hand tracking. We conclude that although the update of control is associated with the update of prediction, prediction can be updated independently of control. To account for this pattern of results, we propose that task demands mediate the update of prediction and control. Although a joint update of prediction and control seemed mandatory for success in our hand tracking task, the update of control was only facultative for success in our eye tracking task. More generally, those results promote the view that prediction and control are mediated by separate neural processes and suggest that people can learn to predict movement consequences without necessarily promoting their ability to control these movements.

Neural Oscillations Orchestrate Multisensory Processing

At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.

Training proprioception with sound: effects of real-time auditory feedback on intermodal learning

Our study analyzed the effects of real-time auditory feedback on intermodal learning during a bilateral knee repositioning task. Thirty healthy participants were randomly allocated to control and experimental groups. Participants performed an active knee joint repositioning task for the four target angles (20°, 40°, 60°, and 80°) bilaterally, with or without additional real-time auditory feedback. Here, the frequency of auditory feedback was mapped to the knee's angle range (0-90°). Retention measurements were performed on the same four angles, without auditory feedback, after 15 min and 24 hours. A generalized knee proprioception test was performed after the 24-h retention measurement on three untrained knee angles (15°, 35°, and 55°). Statistical analysis revealed a significant enhancement of knee proprioception, shown as a lower knee repositioning error with auditory feedback. This enhancement of proprioception also persisted in tests performed between the 5th and 6th auditory-motor training blocks (without auditory feedback). Enhancement in proprioception also remained stable during retention measurements (after 15 min and 24 h). Similarly, enhancement in the generalized proprioception on untrained knee angles was evident in the experimental group. This study extends our previous findings and demonstrates the beneficial effects of real-time auditory feedback to facilitate intermodal learning by enhancing knee proprioception in a persisting and generalized manner.

Not All Predictions Are Equal: "What" and "When" Predictions Modulate Activity in Auditory Cortex through Different Mechanisms

Using predictions based on environmental regularities is fundamental for adaptive behavior. While it is widely accepted that predictions across different stimulus attributes (e.g., time and content) facilitate sensory processing, it is unknown whether predictions across these attributes rely on the same neural mechanism. Here, to elucidate the neural mechanisms of predictions, we combine invasive electrophysiological recordings (human electrocorticography in 4 females and 2 males) with computational modeling while manipulating predictions about content ("what") and time ("when"). We found that "when" predictions increased evoked activity over motor and prefrontal regions both at early (∼180 ms) and late (430-450 ms) latencies. "What" predictability, however, increased evoked activity only over prefrontal areas late in time (420-460 ms). Beyond these dissociable influences, we found that "what" and "when" predictability interactively modulated the amplitude of early (165 ms) evoked responses in the superior temporal gyrus. We modeled the observed neural responses using biophysically realistic neural mass models, to better understand whether "what" and "when" predictions tap into similar or different neurophysiological mechanisms. Our modeling results suggest that "what" and "when" predictability rely on complementary neural processes: "what" predictions increased short-term plasticity in auditory areas, whereas "when" predictability increased synaptic gain in motor areas. Thus, content and temporal predictions engage complementary neural mechanisms in different regions, suggesting domain-specific prediction signaling along the cortical hierarchy. Encoding predictions through different mechanisms may endow the brain with the flexibility to efficiently signal different sources of predictions, weight them by their reliability, and allow for their encoding without mutual interference.SIGNIFICANCE STATEMENT Predictions of different stimulus features facilitate sensory processing. However, it is unclear whether predictions of different attributes rely on similar or different neural mechanisms. By combining invasive electrophysiological recordings of cortical activity with experimental manipulations of participants' predictions about content and time of acoustic events, we found that the two types of predictions had dissociable influences on cortical activity, both in terms of the regions involved and the timing of the observed effects. Further, our biophysical modeling analysis suggests that predictability of content and time rely on complementary neural processes: short-term plasticity in auditory areas and synaptic gain in motor areas, respectively. This suggests that predictions of different features are encoded with complementary neural mechanisms in different brain regions.

Neural mechanisms for selectively tuning in to the target speaker in a naturalistic noisy situation

The neural mechanism for selectively tuning in to a target speaker while tuning out the others in a multi-speaker situation (i.e., the cocktail-party effect) remains elusive. Here we addressed this issue by measuring brain activity simultaneously from a listener and from multiple speakers while they were involved in naturalistic conversations. Results consistently show selectively enhanced interpersonal neural synchronization (INS) between the listener and the attended speaker at left temporal–parietal junction, compared with that between the listener and the unattended speaker across different multi-speaker situations. Moreover, INS increases significantly prior to the occurrence of verbal responses, and even when the listener’s brain activity precedes that of the speaker. The INS increase is independent of brain-to-speech synchronization in both the anatomical location and frequency range. These findings suggest that INS underlies the selective process in a multi-speaker situation through neural predictions at the content level but not the sensory level of speech.

When many people are speaking, e.g. at a party, we can selectively attend to just one speaker. Here, using ‘hyperscanning’, the authors show that interpersonal neural synchronization is selectively increased between a listener and the attended speaker, compared to between the listener and an unattended speaker.