Not All Predictions Are Equal: "What" and "When" Predictions Modulate Activity in Auditory Cortex through Different Mechanisms

"What" and "When" Predictions Jointly Modulate Speech Processing

Adaptive behavior rests on predictions based on statistical regularities in the environment. Such regularities pertain to stimulus contents ("what") and timing ("when"), and both interactively modulate sensory processing. In speech streams, predictions can be formed at multiple hierarchical levels of contents (e.g., syllables vs words) and timing (faster vs slower time scales). Whether and how these hierarchies map onto each other remains unknown. Under one hypothesis, neural hierarchies may link "what" and "when" predictions within sensory processing areas: with lower versus higher cortical regions mediating interactions for smaller versus larger units (syllables vs words). Alternatively, interactions between "what" and "when" regularities might rest on a generic, sensory-independent mechanism. To address these questions, we manipulated "what" and "when" regularities at two levels-single syllables and disyllabic pseudowords-while recording neural activity using magnetoencephalography (MEG) in healthy volunteers (N = 22). We studied how neural responses to syllable and/or pseudoword deviants are modulated by "when" regularity. "When" regularity modulated "what" mismatch responses with hierarchical specificity, such that responses to deviant pseudowords (vs syllables) were amplified by temporal regularity at slower (vs faster) time scales. However, both these interactive effects were source-localized to the same regions, including frontal and parietal cortices. Effective connectivity analysis showed that the integration of "what" and "when" regularity selectively modulated connectivity within regions, consistent with gain effects. This suggests that the brain integrates "what" and "when" predictions that are congruent with respect to their hierarchical level, but this integration is mediated by a shared and distributed cortical network.

Predictive acoustical processing in human cortical layers

In our dynamic environments, predictive processing is vital for auditory perception and its associated behaviors. Predictive coding formalizes inferential processes by implementing them as information exchange across cortical layers and areas. With laminar-specific blood oxygenation level dependent we measured responses to a cascading oddball paradigm, to ground predictive auditory processes on the mesoscopic human cortical architecture. We show that the violation of predictions are potentially hierarchically organized and associated with responses in superficial layers of the planum polare and middle layers of the lateral temporal cortex. Moreover, we relate the updating of the brain's internal model to changes in deep layers. Using a modeling approach, we derive putative changes in neural dynamics while accounting for draining effects. Our results support the role of temporal cortical architecture in the implementation of predictive coding and highlight the ability of laminar fMRI to investigate mesoscopic processes in a large extent of temporal areas.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Statistical Learning of Incidental Perceptual Regularities Induces Sensory Conditioned Cortical Responses

Statistical learning of sensory patterns can lead to predictive neural processes enhancing stimulus perception and enabling fast deviancy detection. Predictive processes have been extensively demonstrated when environmental statistical regularities are relevant to task execution. Preliminary evidence indicates that statistical learning can even occur independently of task relevance and top-down attention, although the temporal profile and neural mechanisms underlying sensory predictions and error signals induced by statistical learning of incidental sensory regularities remain unclear. In our study, we adopted an implicit sensory conditioning paradigm that elicited the generation of specific perceptual priors in relation to task-irrelevant audio-visual associations, while recording Electroencephalography (EEG). Our results showed that learning task-irrelevant associations between audio-visual stimuli resulted in anticipatory neural responses to predictive auditory stimuli conveying anticipatory signals of expected visual stimulus presence or absence. Moreover, we observed specific modulation of cortical responses to probabilistic visual stimulus presentation or omission. Pattern similarity analysis indicated that predictive auditory stimuli tended to resemble the response to expected visual stimulus presence or absence. Remarkably, Hierarchical Gaussian filter modeling estimating dynamic changes of prediction error signals in relation to differential probabilistic occurrences of audio-visual stimuli further demonstrated instantiation of predictive neural signals by showing distinct neural processing of prediction error in relation to violation of expected visual stimulus presence or absence. Overall, our findings indicated that statistical learning of non-salient and task-irrelevant perceptual regularities could induce the generation of neural priors at the time of predictive stimulus presentation, possibly conveying sensory-specific information about the predicted consecutive stimulus.

Syllabic rhythm and prior linguistic knowledge interact with individual differences to modulate phonological statistical learning

Phonological statistical learning - our ability to extract meaningful regularities from spoken language - is considered critical in the early stages of language acquisition, in particular for helping to identify discrete words in continuous speech. Most phonological statistical learning studies use an experimental task introduced by Saffran et al. (1996), in which the syllables forming the words to be learned are presented continuously and isochronously. This raises the question of the extent to which this purportedly powerful learning mechanism is robust to the kinds of rhythmic variability that characterize natural speech. Here, we tested participants with arhythmic, semi-rhythmic, and isochronous speech during learning. In addition, we investigated how input rhythmicity interacts with two other factors previously shown to modulate learning: prior knowledge (syllable order plausibility with respect to participants' first language) and learners' speech auditory-motor synchronization ability. We show that words are extracted by all learners even when the speech input is completely arhythmic. Interestingly, high auditory-motor synchronization ability increases statistical learning when the speech input is temporally more predictable but only when prior knowledge can also be used. This suggests an additional mechanism for learning based on predictions not only about when but also about what upcoming speech will be.

Predictable and unpredictable deviance detection in the human hippocampus and amygdala

Our brains extract structure from the environment and form predictions given past experience. Predictive circuits have been identified in wide-spread cortical regions. However, the contribution of medial temporal structures in predictions remains under-explored. The hippocampus underlies sequence detection and is sensitive to novel stimuli, sufficient to gain access to memory, while the amygdala to novelty. Yet, their electrophysiological profiles in detecting predictable and unpredictable deviant auditory events remain unknown. Here, we hypothesized that the hippocampus would be sensitive to predictability, while the amygdala to unexpected deviance. We presented epileptic patients undergoing presurgical monitoring with standard and deviant sounds, in predictable or unpredictable contexts. Onsets of auditory responses and unpredictable deviance effects were detected earlier in the temporal cortex compared with the amygdala and hippocampus. Deviance effects in 1-20 Hz local field potentials were detected in the lateral temporal cortex, irrespective of predictability. The amygdala showed stronger deviance in the unpredictable context. Low-frequency deviance responses in the hippocampus (1-8 Hz) were observed in the predictable but not in the unpredictable context. Our results reveal a distributed network underlying the generation of auditory predictions and suggest that the neural basis of sensory predictions and prediction error signals needs to be extended.

Acetylcholine modulates the precision of prediction error in the auditory cortex

A fundamental property of sensory systems is their ability to detect novel stimuli in the ambient environment. The auditory brain contains neurons that decrease their response to repetitive sounds but increase their firing rate to novel or deviant stimuli; the difference between both responses is known as stimulus-specific adaptation or neuronal mismatch (nMM). Here, we tested the effect of microiontophoretic applications of ACh on the neuronal responses in the auditory cortex (AC) of anesthetized rats during an auditory oddball paradigm, including cascade controls. Results indicate that ACh modulates the nMM, affecting prediction error responses but not repetition suppression, and this effect is manifested predominantly in infragranular cortical layers. The differential effect of ACh on responses to standards, relative to deviants (in terms of averages and variances), was consistent with the representational sharpening that accompanies an increase in the precision of prediction errors. These findings suggest that ACh plays an important role in modulating prediction error signaling in the AC and gating the access of these signals to higher cognitive levels.

Interdependence of "What" and "When" in the Brain

From a brain's-eye-view, when a stimulus occurs and what it is are interrelated aspects of interpreting the perceptual world. Yet in practice, the putative perceptual inferences about sensory content and timing are often dichotomized and not investigated as an integrated process. We here argue that neural temporal dynamics can influence what is perceived, and in turn, stimulus content can influence the time at which perception is achieved. This computational principle results from the highly interdependent relationship of what and when in the environment. Both brain processes and perceptual events display strong temporal variability that is not always modeled; we argue that understanding-and, minimally, modeling-this temporal variability is key for theories of how the brain generates unified and consistent neural representations and that we ignore temporal variability in our analysis practice at the peril of both data interpretation and theory-building. Here, we review what and when interactions in the brain, demonstrate via simulations how temporal variability can result in misguided interpretations and conclusions, and outline how to integrate and synthesize what and when in theories and models of brain computation.

Null effects of temporal prediction on recognition memory but evidence for differential neural activity at encoding. A registered report

Previous research has demonstrated that rhythmic presentation of stimuli during encoding boosts subsequent recognition and is associated with distinct neural activity compared with when stimuli are presented in an arrhythmic manner. However, it is unclear whether the effect is driven by automatic entrainment to rhythm or non-rhythmic temporal prediction. This registered report presents an Electroencephalographic (EEG) study aimed at establishing the cognitive and neural mechanisms of the effect of temporal prediction on recognition. In a blocked design, stimulus onset during encoding was systematically manipulated in four conditions prior to recognition testing: rhythmic fixed (RF), rhythmic variable (RV), arrhythmic fixed (AF), and arrhythmic variable (AV). By orthogonally varying rhythm and temporal position we were able to assess their independent contributions to recognition enhancement. Our behavioural results did not replicate previous findings that show a difference in recognition memory based on temporal predictability at encoding. However, event-related potential (ERP) component analysis did show an early (N1) interaction effect of temporal position and rhythm, and later (N2 and Dm) effects driven by temporal position only. Taken together, we observed effects of temporal prediction at encoding, but these differences did not translate to later effects of memory, suggesting that effects of temporal prediction on recognition are less robust than previously thought.

"What" and "when" predictions modulate auditory processing in a mutually congruent manner

Introduction

Extracting regularities from ongoing stimulus streams to form predictions is crucial for adaptive behavior. Such regularities exist in terms of the content of the stimuli and their timing, both of which are known to interactively modulate sensory processing. In real-world stimulus streams such as music, regularities can occur at multiple levels, both in terms of contents (e.g., predictions relating to individual notes vs. their more complex groups) and timing (e.g., pertaining to timing between intervals vs. the overall beat of a musical phrase). However, it is unknown whether the brain integrates predictions in a manner that is mutually congruent (e.g., if "beat" timing predictions selectively interact with "what" predictions falling on pulses which define the beat), and whether integrating predictions in different timing conditions relies on dissociable neural correlates.

Methods

To address these questions, our study manipulated "what" and "when" predictions at different levels - (local) interval-defining and (global) beat-defining - within the same stimulus stream, while neural activity was recorded using electroencephalogram (EEG) in participants (N = 20) performing a repetition detection task.

Results

Our results reveal that temporal predictions based on beat or interval timing modulated mismatch responses to violations of "what" predictions happening at the predicted time points, and that these modulations were shared between types of temporal predictions in terms of the spatiotemporal distribution of EEG signals. Effective connectivity analysis using dynamic causal modeling showed that the integration of "what" and "when" predictions selectively increased connectivity at relatively late cortical processing stages, between the superior temporal gyrus and the fronto-parietal network.

Discussion

Taken together, these results suggest that the brain integrates different predictions with a high degree of mutual congruence, but in a shared and distributed cortical network. This finding contrasts with recent studies indicating separable mechanisms for beat-based and memory-based predictive processing.

Cortical speech tracking is related to individual prediction tendencies

Listening can be conceptualized as a process of active inference, in which the brain forms internal models to integrate auditory information in a complex interaction of bottom-up and top-down processes. We propose that individuals vary in their "prediction tendency" and that this variation contributes to experiential differences in everyday listening situations and shapes the cortical processing of acoustic input such as speech. Here, we presented tone sequences of varying entropy level, to independently quantify auditory prediction tendency (as the tendency to anticipate low-level acoustic features) for each individual. This measure was then used to predict cortical speech tracking in a multi speaker listening task, where participants listened to audiobooks narrated by a target speaker in isolation or interfered by 1 or 2 distractors. Furthermore, semantic violations were introduced into the story, to also examine effects of word surprisal during speech processing. Our results show that cortical speech tracking is related to prediction tendency. In addition, we find interactions between prediction tendency and background noise as well as word surprisal in disparate brain regions. Our findings suggest that individual prediction tendencies are generalizable across different listening situations and may serve as a valuable element to explain interindividual differences in natural listening situations.

Top-down specific preparatory activations for selective attention and perceptual expectations

Proactive cognition brain models are mainstream nowadays. Within these, preparation is understood as an endogenous, top-down function that takes place prior to the actual perception of a stimulus and improves subsequent behavior. Neuroimaging has shown the existence of such preparatory activity separately in different cognitive domains, however no research to date has sought to uncover their potential similarities and differences. Two of these, often confounded in the literature, are Selective Attention (information relevance) and Perceptual Expectation (information probability). We used EEG to characterize the mechanisms that pre-activate specific contents in Attention and Expectation. In different blocks, participants were cued to the relevance or to the probability of target categories, faces vs. names, in a gender discrimination task. Multivariate Pattern (MVPA) and Representational Similarity Analyses (RSA) during the preparation window showed that both manipulations led to a significant, ramping-up prediction of the relevant or expected target category. However, classifiers trained with data from one condition did not generalize to the other, indicating the existence of unique anticipatory neural patterns. In addition, a Canonical Template Tracking procedure showed that there was stronger anticipatory perceptual reinstatement for relevance than for expectation blocks. Overall, the results indicate that preparation during attention and expectation acts through distinguishable neural mechanisms. These findings have important implications for current models of brain functioning, as they are a first step towards characterizing and dissociating the neural mechanisms involved in top-down anticipatory processing.

Preterm neonates distinguish rhythm violation through a hierarchy of cortical processing

Rhythm is a fundamental component of the auditory world, present even during the prenatal life. While there is evidence that some auditory capacities are already present before birth, whether and how the premature neural networks process auditory rhythm is yet not known. We investigated the neural response of premature neonates at 30-34 weeks gestational age to violations from rhythmic regularities in an auditory sequence using high-resolution electroencephalography and event-related potentials. Unpredicted rhythm violations elicited a fronto-central mismatch response, indicating that the premature neonates detected the rhythmic regularities. Next, we examined the cortical effective connectivity underlying the elicited mismatch response using dynamic causal modeling. We examined the connectivity between cortical sources using a set of 16 generative models that embedded alternate hypotheses about the role of the frontal cortex as well as backward fronto-temporal connection. Our results demonstrated that the processing of rhythm violations was not limited to the primary auditory areas, and as in the case of adults, encompassed a hierarchy of temporo-frontal cortical structures. The result also emphasized the importance of top-down (backward) projections from the frontal cortex in explaining the mismatch response. Our findings demonstrate a sophisticated cortical structure underlying predictive rhythm processing at the onset of the thalamocortical and cortico-cortical circuits, two months before term.

How childhood maltreatment alters perception and cognition - the predictive processing account of borderline personality disorder

Borderline personality disorder (BPD) is a severe mental disorder, comprised of heterogeneous psychological and neurobiological pathologies. Here, we propose a predictive processing (PP) account of BPD to integrate these seemingly unrelated pathologies. In particular, we argue that the experience of childhood maltreatment, which is highly prevalent in BPD, leaves a developmental legacy with two facets: first, a coarse-grained, alexithymic model of self and others - leading to a rigidity and inflexibility concerning beliefs about self and others. Second, this developmental legacy leads to a loss of confidence or precision afforded beliefs about the consequences of social behavior. This results in an over reliance on sensory evidence and social feedback, with concomitant lability, impulsivity and hypersensitivity. In terms of PP, people with BPD show a distorted belief updating in response to new information with two opposing manifestations: rapid changes in beliefs and a lack of belief updating despite disconfirmatory evidence. This account of distorted information processing has the potential to explain both the instability (of affect, self-image, and interpersonal relationships) and the rigidity (of beliefs about self and others) which is typical of BPD. At the neurobiological level, we propose that enhanced levels of dopamine are associated with the increased integration of negative social feedback, and we also discuss the hypothesis of an impaired inhibitory control of the prefrontal cortex in the processing of negative social information. Our account may provide a new understanding not only of the clinical aspects of BPD, but also a unifying theory of the corresponding neurobiological pathologies. We conclude by outlining some directions for future research on the behavioral, neurobiological, and computational underpinnings of this model, and point to some clinical implications of it.

Dynamics of Oddball Sound Processing: Trial-by-Trial Modeling of ECoG Signals

Recent computational models of perception conceptualize auditory oddball responses as signatures of a (Bayesian) learning process, in line with the influential view of the mismatch negativity (MMN) as a prediction error signal. Novel MMN experimental paradigms have put an emphasis on neurophysiological effects of manipulating regularity and predictability in sound sequences. This raises the question of the contextual adaptation of the learning process itself, which on the computational side speaks to the mechanisms of gain-modulated (or precision-weighted) prediction error. In this study using electrocorticographic (ECoG) signals, we manipulated the predictability of oddball sound sequences with two objectives: (i) Uncovering the computational process underlying trial-by-trial variations of the cortical responses. The fluctuations between trials, generally ignored by approaches based on averaged evoked responses, should reflect the learning involved. We used a general linear model (GLM) and Bayesian Model Reduction (BMR) to assess the respective contributions of experimental manipulations and learning mechanisms under probabilistic assumptions. (ii) To validate and expand on previous findings regarding the effect of changes in predictability using simultaneous EEG-MEG recordings. Our trial-by-trial analysis revealed only a few stimulus-responsive sensors but the measured effects appear to be consistent over subjects in both time and space. In time, they occur at the typical latency of the MMN (between 100 and 250 ms post-stimulus). In space, we found a dissociation between time-independent effects in more anterior temporal locations and time-dependent (learning) effects in more posterior locations. However, we could not observe any clear and reliable effect of our manipulation of predictability modulation onto the above learning process. Overall, these findings clearly demonstrate the potential of trial-to-trial modeling to unravel perceptual learning processes and their neurophysiological counterparts.

Unpredictability of the “when” influences prediction error processing of the “what” and “where”

The capability to establish accurate predictions is an integral part of learning. Whether predictions about different dimensions of a stimulus interact with each other, and whether such an interaction affects learning, has remained elusive. We conducted a statistical learning study with EEG (electroencephalography), where a stream of consecutive sound triplets was presented with deviants that were either: (a) statistical, depending on the triplet ending probability, (b) physical, due to a change in sound location or (c) double deviants, i.e. a combination of the two. We manipulated the predictability of stimulus-onset by using random stimulus-onset asynchronies. Temporal unpredictability due to random onsets reduced the neurophysiological responses to statistical and location deviants, as indexed by the statistical mismatch negativity (sMMN) and the location MMN. Our results demonstrate that the predictability of one stimulus attribute influences the processing of prediction error signals of other stimulus attributes, and thus also learning of those attributes.

Predictive Processing in Poetic Language: Event-Related Potentials Data on Rhythmic Omissions in Metered Speech

Predictions during language comprehension are currently discussed from many points of view. One area where predictive processing may play a particular role concerns poetic language that is regularized by meter and rhyme, thus allowing strong predictions regarding the timing and stress of individual syllables. While there is growing evidence that these prosodic regularities influence language processing, less is known about the potential influence of prosodic preferences (binary, strong-weak patterns) on neurophysiological processes. To this end, the present electroencephalogram (EEG) study examined whether the predictability of strong and weak syllables within metered speech would differ as a function of meter (trochee vs. iamb). Strong, i.e., accented positions within a foot should be more predictable than weak, i.e., unaccented positions. Our focus was on disyllabic pseudowords that solely differed between trochaic and iambic structure, with trochees providing the preferred foot in German. Methodologically, we focused on the omission Mismatch Negativity (oMMN) that is elicited when an anticipated auditory stimulus is omitted. The resulting electrophysiological brain response is particularly interesting because its elicitation does not depend on a physical stimulus. Omissions in deviant position of a passive oddball paradigm occurred at either first- or second-syllable position of the aforementioned pseudowords, resulting in a 2-by-2 design with the factors foot type and omission position. Analyses focused on the mean oMMN amplitude and latency differences across the four conditions. The result pattern was characterized by an interaction of the effects of foot type and omission position for both amplitudes and latencies. In first position, omissions resulted in larger and earlier oMMNs for trochees than for iambs. In second position, omissions resulted in larger oMMNs for iambs than for trochees, but the oMMN latency did not differ. The results suggest that omissions, particularly in initial position, are modulated by a trochaic preference in German. The preferred strong-weak pattern may have strengthened the prosodic prediction, especially for matching, trochaic stimuli, such that the violation of this prediction led to an earlier and stronger prediction error. Altogether, predictive processing seems to play a particular role in metered speech, especially if the meter is based on the preferred foot type.

Ongoing neural oscillations influence behavior and sensory representations by suppressing neuronal excitability

The ability to process and respond to external input is critical for adaptive behavior. Why, then, do neural and behavioral responses vary across repeated presentations of the same sensory input? Ongoing fluctuations of neuronal excitability are currently hypothesized to underlie the trial-by-trial variability in sensory processing. To test this, we capitalized on intracranial electrophysiology in neurosurgical patients performing an auditory discrimination task with visual cues: specifically, we examined the interaction between prestimulus alpha oscillations, excitability, task performance, and decoded neural stimulus representations. We found that strong prestimulus oscillations in the alpha+ band (i.e., alpha and neighboring frequencies), rather than the aperiodic signal, correlated with a low excitability state, indexed by reduced broadband high-frequency activity. This state was related to slower reaction times and reduced neural stimulus encoding strength. We propose that the alpha+ rhythm modulates excitability, thereby resulting in variability in behavior and sensory representations despite identical input.

Musicianship and melodic predictability enhance neural gain in auditory cortex during pitch deviance detection

When listening to music, pitch deviations are more salient and elicit stronger prediction error responses when the melodic context is predictable and when the listener is a musician. Yet, the neuronal dynamics and changes in connectivity underlying such effects remain unclear. Here, we employed dynamic causal modeling (DCM) to investigate whether the magnetic mismatch negativity response (MMNm)-and its modulation by context predictability and musical expertise-are associated with enhanced neural gain of auditory areas, as a plausible mechanism for encoding precision-weighted prediction errors. Using Bayesian model comparison, we asked whether models with intrinsic connections within primary auditory cortex (A1) and superior temporal gyrus (STG)-typically related to gain control-or extrinsic connections between A1 and STG-typically related to propagation of prediction and error signals-better explained magnetoencephalography responses. We found that, compared to regular sounds, out-of-tune pitch deviations were associated with lower intrinsic (inhibitory) connectivity in A1 and STG, and lower backward (inhibitory) connectivity from STG to A1, consistent with disinhibition and enhanced neural gain in these auditory areas. More predictable melodies were associated with disinhibition in right A1, while musicianship was associated with disinhibition in left A1 and reduced connectivity from STG to left A1. These results indicate that musicianship and melodic predictability, as well as pitch deviations themselves, enhance neural gain in auditory cortex during deviance detection. Our findings are consistent with predictive processing theories suggesting that precise and informative error signals are selected by the brain for subsequent hierarchical processing.

Active inference, selective attention, and the cocktail party problem

In this paper, we introduce a new generative model for an active inference account of preparatory and selective attention, in the context of a classic 'cocktail party' paradigm. In this setup, pairs of words are presented simultaneously to the left and right ears and an instructive spatial cue directs attention to the left or right. We use this generative model to test competing hypotheses about the way that human listeners direct preparatory and selective attention. We show that assigning low precision to words at attended-relative to unattended-locations can explain why a listener reports words from a competing sentence. Under this model, temporal changes in sensory precision were not needed to account for faster reaction times with longer cue-target intervals, but were necessary to explain ramping effects on event-related potentials (ERPs)-resembling the contingent negative variation (CNV)-during the preparatory interval. These simulations reveal that different processes are likely to underlie the improvement in reaction times and the ramping of ERPs that are associated with spatial cueing.

Preparatory delta phase response is correlated with naturalistic speech comprehension performance

While human speech comprehension is thought to be an active process that involves top-down predictions, it remains unclear how predictive information is used to prepare for the processing of upcoming speech information. We aimed to identify the neural signatures of the preparatory processing of upcoming speech. Participants selectively attended to one of two competing naturalistic, narrative speech streams, and a temporal response function (TRF) method was applied to derive event-related-like neural responses from electroencephalographic data. The phase responses to the attended speech at the delta band (1-4 Hz) were correlated with the comprehension performance of individual participants, with a latency of - 200-0 ms relative to the onset of speech amplitude envelope fluctuations over the fronto-central and left-lateralized parietal electrodes. The phase responses to the attended speech at the alpha band also correlated with comprehension performance but with a latency of 650-980 ms post-onset over the fronto-central electrodes. Distinct neural signatures were found for the attentional modulation, taking the form of TRF-based amplitude responses at a latency of 240-320 ms post-onset over the left-lateralized fronto-central and occipital electrodes. Our findings reveal how the brain gets prepared to process an upcoming speech in a continuous, naturalistic speech context.

Violation of rhythmic expectancies can elicit late frontal gamma activity nested in theta oscillations

Rhythm processing involves building expectations according to the hierarchical temporal structure of auditory events. Although rhythm processing has been addressed in the context of predictive coding, the properties of the oscillatory response in different cortical areas are still not clear. We explored the oscillatory properties of the neural response to rhythmic incongruence and the cross-frequency coupling between multiple frequencies to further investigate the mechanisms underlying rhythm perception. We designed an experiment to investigate the neural response to rhythmic deviations in which the tone either arrived earlier than expected or the tone in the same metrical position was omitted. These two manipulations modulate the rhythmic structure differently, with the former creating a larger violation of the general structure of the musical stimulus than the latter. Both deviations resulted in an MMN response, whereas only the rhythmic deviant resulted in a subsequent P3a. Rhythmic deviants due to the early occurrence of a tone, but not omission deviants, seemed to elicit a late high gamma response (60-80 Hz) at the end of the P3a over the left frontal region, which, interestingly, correlated with the P3a amplitude over the same region and was also nested in theta oscillations. The timing of the elicited high-frequency gamma oscillations related to rhythmic deviation suggests that it might be related to the update of the predictive neural model, corresponding to the temporal structure of the events in higher-level cortical areas.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Sensorimotor contributions to working memory differ between the discrimination of Same and Different syllable pairs

Sensorimotor activity during speech perception is both pervasive and highly variable, changing as a function of the cognitive demands imposed by the task. The purpose of the current study was to evaluate whether the discrimination of Same (matched) and Different (unmatched) syllable pairs elicit different patterns of sensorimotor activity as stimuli are processed in working memory. Raw EEG data recorded from 42 participants were decomposed with independent component analysis to identify bilateral sensorimotor mu rhythms from 36 subjects. Time frequency decomposition of mu rhythms revealed concurrent event related desynchronization (ERD) in alpha and beta frequency bands across the peri- and post-stimulus time periods, which were interpreted as evidence of sensorimotor contributions to working memory encoding and maintenance. Left hemisphere alpha/beta ERD was stronger in Different trials than Same trials during the post-stimulus period, while right hemisphere alpha/beta ERD was stronger in Same trials than Different trials. A between-hemispheres contrast revealed no differences during Same trials, while post-stimulus alpha/beta ERD was stronger in the left hemisphere than the right during Different trials. Results were interpreted to suggest that predictive coding mechanisms lead to repetition suppression effects in Same trials. Mismatches arising from predictive coding mechanisms in Different trials shift subsequent working memory processing to the speech-dominant left hemisphere. Findings clarify how sensorimotor activity differentially supports working memory encoding and maintenance stages during speech discrimination tasks and have potential to inform sensorimotor models of speech perception and working memory.

Temporal uncertainty enhances suppression of neural responses to predictable visual stimuli

•

Does stimulus timing impact the processing of predicted visual features?.

•

We evaluated if expectation suppression effects are modulated by temporal predictability.

•

Expectation suppression was robust in both visual ERFs and feature decoding accuracy.

•

Visual responses to predictable stimuli are greater for predictable vs. unpredictable timing.

•

Sensory evidence is given less weight when timing is uncertain.

Contextual information triggers predictions about the content (“what”) of environmental stimuli to update an internal generative model of the surrounding world. However, visual information dynamically changes across time, and temporal predictability (“when”) may influence the impact of internal predictions on visual processing. In this magnetoencephalography (MEG) study, we investigated how processing feature specific information (“what”) is affected by temporal predictability (“when”). Participants (N = 16) were presented with four consecutive Gabor patches (entrainers) with constant spatial frequency but with variable orientation and temporal onset. A fifth target Gabor was presented after a longer delay and with higher or lower spatial frequency that participants had to judge. We compared the neural responses to entrainers where the Gabor orientation could, or could not be temporally predicted along the entrainer sequence, and with inter-entrainer timing that was constant (predictable), or variable (unpredictable). We observed suppression of evoked neural responses in the visual cortex for predictable stimuli. Interestingly, we found that temporal uncertainty increased expectation suppression. This suggests that in temporally uncertain scenarios the neurocognitive system invests less resources in integrating bottom-up information. Multivariate pattern analysis showed that predictable visual features could be decoded from neural responses. Temporal uncertainty did not affect decoding accuracy for early visual responses, with the feature specificity of early visual neural activity preserved across conditions. However, decoding accuracy was less sustained over time for temporally jittered than for isochronous predictable visual stimuli. These findings converge to suggest that the cognitive system processes visual features of temporally predictable stimuli in higher detail, while processing temporally uncertain stimuli may rely more heavily on abstract internal expectations.

Heartbeat and somatosensory perception

Our perception of the external world is influenced by internal bodily signals. For example, we recently showed that timing of stimulation along the cardiac cycle and spontaneous fluctuations of heartbeat-evoked potential (HEP) amplitudes influence somatosensory perception and the associated neural processing (Al et al., 2020). While cardiac phase affected detection sensitivity and late components of the somatosensory-evoked potentials (SEPs), HEP amplitudes affected detection criterion and both early and late SEP components. In a new EEG study, we investigate whether these results are replicable in a modified paradigm, which includes two succeeding temporal intervals. In one of the intervals, subjects received a weak electrical finger stimulation and reported first whether they detected any stimulation and then allocated the stimulus to one of the two intervals. Our results confirm the previously reported cardiac cycle and prestimulus HEP effects on somatosensory perception and evoked potentials. In addition, we obtained two new findings. Source analyses in this and our original study show that the increased likelihood of conscious perception goes along with HEP fluctuations in parietal and posterior cingulate regions, known to play important roles in interoceptive processes. Furthermore, HEP amplitudes were shown to decrease when subjects engaged in the somatosensory task compared to a resting state condition. Our findings are consistent with the view that HEP amplitudes are a marker of interoceptive (versus exteroceptive) attention and provide a neural underpinning for this view.

Do Auditory Mismatch Responses Differ Between Acoustic Features?

Mismatch negativity (MMN) is the electroencephalographic (EEG) waveform obtained by subtracting event-related potential (ERP) responses evoked by unexpected deviant stimuli from responses evoked by expected standard stimuli. While the MMN is thought to reflect an unexpected change in an ongoing, predictable stimulus, it is unknown whether MMN responses evoked by changes in different stimulus features have different magnitudes, latencies, and topographies. The present study aimed to investigate whether MMN responses differ depending on whether sudden stimulus change occur in pitch, duration, location or vowel identity, respectively. To calculate ERPs to standard and deviant stimuli, EEG signals were recorded in normal-hearing participants (N = 20; 13 males, 7 females) who listened to roving oddball sequences of artificial syllables. In the roving paradigm, any given stimulus is repeated several times to form a standard, and then suddenly replaced with a deviant stimulus which differs from the standard. Here, deviants differed from preceding standards along one of four features (pitch, duration, vowel or interaural level difference). The feature levels were individually chosen to match behavioral discrimination performance. We identified neural activity evoked by unexpected violations along all four acoustic dimensions. Evoked responses to deviant stimuli increased in amplitude relative to the responses to standard stimuli. A univariate (channel-by-channel) analysis yielded no significant differences between MMN responses following violations of different features. However, in a multivariate analysis (pooling information from multiple EEG channels), acoustic features could be decoded from the topography of mismatch responses, although at later latencies than those typical for MMN. These results support the notion that deviant feature detection may be subserved by a different process than general mismatch detection.

Both contextual regularity and selective attention affect the reduction of precision-weighted prediction errors but in distinct manners

Predictive coding model of perception postulates that the primary objective of the brain is to infer the causes of sensory inputs by reducing prediction errors (i.e., the discrepancy between expected and actual information). Moreover, prediction errors are weighted by their precision (i.e., inverse variance), which quantifies the degree of certainty about the variables. There is accumulating evidence that the reduction of precision-weighted prediction errors can be affected by contextual regularity (as an external factor) and selective attention (as an internal factor). However, it is unclear whether the two factors function together or separately. Here we used electroencephalography (EEG) to examine the putative interaction of contextual regularity and selective attention on this reduction process. Participants were presented with pairs of regular and irregular quartets in attended and unattended conditions. We found that contextual regularity and selective attention independently modulated the N1/MMN where the repetition effect was absent. On the P2, the two factors respectively interacted with the repetition effect without interacting with each other. The results showed that contextual regularity and selective attention likely affect the reduction of precision-weighted prediction errors in distinct manners. While contextual regularity finetunes our efficiency at reducing precision-weighted prediction errors, selective attention seems to modulate the reduction process following the Matthew effect of accumulated advantage.

Learning to predict: Neuronal signatures of auditory expectancy in human event-related potentials

Learning to anticipate future states of the world based on statistical regularities in the environment is a key component of perception and is vital for the survival of many organisms. Such statistical learning and prediction are crucial for acquiring language and music appreciation. Importantly, learned expectations can be implicitly derived from exposure to sensory input, without requiring explicit information regarding contingencies in the environment. Whereas many previous studies of statistical learning have demonstrated larger neuronal responses to unexpected versus expected stimuli, the neuronal bases of the expectations themselves remain poorly understood. Here we examined behavioral and neuronal signatures of learned expectancy via human scalp-recorded event-related brain potentials (ERPs). Participants were instructed to listen to a series of sounds and press a response button as quickly as possible upon hearing a target noise burst, which was either reliably or unreliably preceded by one of three pure tones in low-, mid-, and high-frequency ranges. Participants were not informed about the statistical contingencies between the preceding tone ‘cues’ and the target. Over the course of a stimulus block, participants responded more rapidly to reliably cued targets. This behavioral index of learned expectancy was paralleled by a negative ERP deflection, designated as a neuronal contingency response (CR), which occurred immediately prior to the onset of the target. The amplitude and latency of the CR were systematically modulated by the strength of the predictive relationship between the cue and the target. Re-averaging ERPs with respect to the latency of behavioral responses revealed no consistent relationship between the CR and the motor response, suggesting that the CR represents a neuronal signature of learned expectancy or anticipatory attention. Our results demonstrate that statistical regularities in an auditory input stream can be implicitly learned and exploited to influence behavior. Furthermore, we uncover a potential ‘prediction signal’ that reflects this fundamental learning process.

Insights into human cognition from intracranial EEG: A review of audition, memory, internal cognition, and causality

By recording neural activity directly from the human brain, researchers gain unprecedented insight into how neurocognitive processes unfold in real time. We first briefly discuss how intracranial electroencephalography (iEEG) recordings, performed for clinical practice, are used to study human cognition with the spatiotemporal and single-trial precision traditionally limited to non-human animal research. We then delineate how studies using iEEG have informed our understanding of issues fundamental to human cognition: auditory prediction, working and episodic memory, and internal cognition. We also discuss the potential of iEEG to infer causality through the manipulation or 'engineering' of neurocognitive processes via spatiotemporally precise electrical stimulation. We close by highlighting limitations of iEEG, potential of burgeoning techniques to further increase spatiotemporal precision, and implications for future research using intracranial approaches to understand, restore, and enhance human cognition.

Purpose-Dependent Consequences of Temporal Expectations Serving Perception and Action

Temporal expectations enable anticipatory brain states that prepare us for upcoming perception and action. We investigated the purpose-dependent nature and consequences of cued temporal expectations on brain and behavior in male and female human volunteers, using two matched visual-motor tasks that stressed either response speed or visual accuracy. We show that the consequences of temporal expectations are fundamentally purpose dependent. Temporal expectations predominantly affected response times when visual demands were low and speed was more important, but perceptual accuracy when visual demands were more challenging. Using magnetoencephalography, we further show how temporal expectations latch onto anticipatory neural states associated with concurrent spatial expectations—modulating task-specific anticipatory neural lateralization of oscillatory brain activity in a modality- and frequency-specific manner. By relating these brain states to behavior, we finally reveal how the behavioral relevance of such anticipatory brain states is similarly purpose dependent.

SIGNIFICANCE STATEMENT Knowing when events may occur helps to prepare neural activity for upcoming perception and action. It is becoming increasingly clear that distinct sources of temporal expectations may facilitate performance via distinct mechanisms. Another relevant dimension to consider regards the distinct purposes that temporal expectations may serve. Here, we demonstrate that the consequences of temporal expectations on neurophysiological brain activity and behavior are fundamentally purpose dependent, and show how temporal expectations interact with task-relevant neural states in a modality- and frequency-specific manner. This brings the important insight that the ways in which temporal expectations influence brain and behavior, and how brain activity is related to behavior, are not fixed properties but rather depend on the task at hand.

Prediction and memory: A predictive coding account

The hippocampus is crucial for episodic memory, but it is also involved in online prediction. Evidence suggests that a unitary hippocampal code underlies both episodic memory and predictive processing, yet within a predictive coding framework the hippocampal-neocortical interactions that accompany these two phenomena are distinct and opposing. Namely, during episodic recall, the hippocampus is thought to exert an excitatory influence on the neocortex, to reinstate activity patterns across cortical circuits. This contrasts with empirical and theoretical work on predictive processing, where descending predictions suppress prediction errors to 'explain away' ascending inputs via cortical inhibition. In this hypothesis piece, we attempt to dissolve this previously overlooked dialectic. We consider how the hippocampus may facilitate both prediction and memory, respectively, by inhibiting neocortical prediction errors or increasing their gain. We propose that these distinct processing modes depend upon the neuromodulatory gain (or precision) ascribed to prediction error units. Within this framework, memory recall is cast as arising from fictive prediction errors that furnish training signals to optimise generative models of the world, in the absence of sensory data.

Dopamine modulates subcortical responses to surprising sounds

Dopamine guides behavior and learning through pleasure, according to classic understanding. Dopaminergic neurons are traditionally thought to signal positive or negative prediction errors (PEs) when reward expectations are, respectively, exceeded or not matched. These signed PEs are quite different from the unsigned PEs, which report surprise during sensory processing. But mounting theoretical accounts from the predictive processing framework postulate that dopamine, as a neuromodulator, could potentially regulate the postsynaptic gain of sensory neurons, thereby scaling unsigned PEs according to their expected precision or confidence. Despite ample modeling work, the physiological effects of dopamine on the processing of surprising sensory information are yet to be addressed experimentally. In this study, we tested how dopamine modulates midbrain processing of unexpected tones. We recorded extracellular responses from the rat inferior colliculus to oddball and cascade sequences, before, during, and after the microiontophoretic application of dopamine or eticlopride (a D₂-like receptor antagonist). Results demonstrate that dopamine reduces the net neuronal responsiveness exclusively to unexpected sensory input without significantly altering the processing of expected input. We conclude that dopaminergic projections from the thalamic subparafascicular nucleus to the inferior colliculus could encode the expected precision of unsigned PEs, attenuating via D₂-like receptors the postsynaptic gain of sensory inputs forwarded by the auditory midbrain neurons. This direct dopaminergic modulation of sensory PE signaling has profound implications for both the predictive coding framework and the understanding of dopamine function.

Information about unexpected stimuli is encoded in the form of prediction error signals. The earliest prediction error signals identified in the auditory brain emerge subcortically in the inferior colliculus. This study reveals the essential role of dopamine in encoding the precision of prediction errors at the auditory midbrain.

A challenge for predictive coding: Representational or experiential diversity?

To become a unifying theory of brain function, predictive processing (PP) must accommodate its rich representational diversity. Gilead et al. claim such diversity requires a multi-process theory, and thus is out of reach for PP, which postulates a universal canonical computation. We contend this argument and instead propose that PP fails to account for the experiential level of representations.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Natural rhythms of periodic temporal attention

That attention is a fundamentally rhythmic process has recently received abundant empirical evidence. The essence of temporal attention, however, is to flexibly focus in time. Whether this function is constrained by an underlying rhythmic neural mechanism is unknown. In six interrelated experiments, we behaviourally quantify the sampling capacities of periodic temporal attention during auditory or visual perception. We reveal the presence of limited attentional capacities, with an optimal sampling rate of ~1.4 Hz in audition and ~0.7 Hz in vision. Investigating the motor contribution to temporal attention, we show that it scales with motor rhythmic precision, maximal at ~1.7 Hz. Critically, motor modulation is beneficial to auditory but detrimental to visual temporal attention. These results are captured by a computational model of coupled oscillators, that reveals the underlying structural constraints governing the temporal alignment between motor and attention fluctuations.

Cortical mapping of mismatch responses to independent acoustic features

Predictive coding is an influential theory of neural processing underlying perceptual inference. However, it is unknown to what extent prediction violations of different sensory features are mediated in different regions in auditory cortex, with different dynamics, and by different mechanisms. This study investigates the neural responses to synthesized acoustic syllables, which could be expected or unexpected, along several features. By using electrocorticography (ECoG) in rat auditory cortex (subjects: adult female Wistar rats with normal hearing), we aimed at mapping regional differences in mismatch responses to different stimulus features. Continuous streams of morphed syllables formed roving oddball sequences in which each stimulus was repeated several times (thereby forming a standard) and subsequently replaced with a deviant stimulus which differed from the standard along one of several acoustic features: duration, pitch, interaural level differences (ILD), or consonant identity. Each of these features could assume one of several different levels, and the resulting change from standard to deviant could be larger or smaller. The deviant stimuli were then repeated to form new standards. We analyzed responses to the first repetition of a new stimulus (deviant) and its last repetition in a stimulus train (standard). For the ECoG recording, we implanted urethane-anaesthetized rats with 8 × 8 surface electrode arrays covering a 3 × 3 mm cortical patch encompassing primary and higher-order auditory cortex. We identified the response topographies and latencies of population activity evoked by acoustic stimuli in the rat auditory regions, and mapped their sensitivity to expectation violations along different acoustic features. For all features, the responses to deviant stimuli increased in amplitude relative to responses to standard stimuli. Deviance magnitude did not further modulate these mismatch responses. Mismatch responses to different feature violations showed a heterogeneous distribution across cortical areas, with no evidence for systematic topographic gradients for any of the tested features. However, within rats, the spatial distribution of mismatch responses varied more between features than the spatial distribution of tone-evoked responses. This result supports the notion that prediction error signaling along different stimulus features is subserved by different cortical populations, albeit with substantial heterogeneity across individuals.

Beat-based and Memory-based Temporal Expectations in Rhythm: Similar Perceptual Effects, Different Underlying Mechanisms

Predicting the timing of incoming information allows the brain to optimize information processing in dynamic environments. Behaviorally, temporal expectations have been shown to facilitate processing of events at expected time points, such as sounds that coincide with the beat in musical rhythm. Yet, temporal expectations can develop based on different forms of structure in the environment, not just the regularity afforded by a musical beat. Little is still known about how different types of temporal expectations are neurally implemented and affect performance. Here, we orthogonally manipulated the periodicity and predictability of rhythmic sequences to examine the mechanisms underlying beat-based and memory-based temporal expectations, respectively. Behaviorally and using EEG, we looked at the effects of beat-based and memory-based expectations on auditory processing when rhythms were task-relevant or task-irrelevant. At expected time points, both beat-based and memory-based expectations facilitated target detection and led to attenuation of P1 and N1 responses, even when expectations were task-irrelevant (unattended). For beat-based expectations, we additionally found reduced target detection and enhanced N1 responses for events at unexpected time points (e.g., off-beat), regardless of the presence of memory-based expectations or task relevance. This latter finding supports the notion that periodicity selectively induces rhythmic fluctuations in neural excitability and furthermore indicates that, although beat-based and memory-based expectations may similarly affect auditory processing of expected events, their underlying neural mechanisms may be different.

Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain

Temporal orienting improves sensory processing, akin to other top-down biases. However, it is unknown whether these improvements reflect increased neural gain to any stimuli presented at expected time points, or specific tuning to task-relevant stimulus aspects. Furthermore, while other top-down biases are selective, the extent of trade-offs across time is less well characterized. Here, we tested whether gain and/or tuning of auditory frequency processing in humans is modulated by rhythmic temporal expectations, and whether these modulations are specific to time points relevant for task performance. Healthy participants (N = 23) of either sex performed an auditory discrimination task while their brain activity was measured using magnetoencephalography/electroencephalography (M/EEG). Acoustic stimulation consisted of sequences of brief distractors interspersed with targets, presented in a rhythmic or jittered way. Target rhythmicity not only improved behavioral discrimination accuracy and M/EEG-based decoding of targets, but also of irrelevant distractors preceding these targets. To explain this finding in terms of increased sensitivity and/or sharpened tuning to auditory frequency, we estimated tuning curves based on M/EEG decoding results, with separate parameters describing gain and sharpness. The effect of rhythmic expectation on distractor decoding was linked to gain increase only, suggesting increased neural sensitivity to any stimuli presented at relevant time points.SIGNIFICANCE STATEMENT Being able to predict when an event may happen can improve perception and action related to this event, likely due to the alignment of neural activity to the temporal structure of stimulus streams. However, it is unclear whether rhythmic increases in neural sensitivity are specific to task-relevant targets, and whether they competitively impair stimulus processing at unexpected time points. By combining magnetoencephalography and encephalographic recordings, neural decoding of auditory stimulus features, and modeling, we found that rhythmic expectation improved neural decoding of both relevant targets and irrelevant distractors presented and expected time points, but did not competitively impair stimulus processing at unexpected time points. Using a quantitative model, these results were linked to nonspecific neural gain increases due to rhythmic expectation.

Does hallucination perceptual modality impact psychosis risk?

OBJECTIVE

Subthreshold perceptual abnormalities are commonly used to identify individuals at clinical high risk (CHR) for developing a psychotic disorder. Predictive validity for modality-specific perceptual abnormality severity on psychosis risk is unknown.

METHODS

We examined prospectively collected data from 164 individuals age 12-35 meeting criteria for CHR followed for 6-24 months or until conversion to psychosis. Using intake interview notes, baseline perceptual abnormality scores were split into auditory, visual, somatic/tactile, and olfactory/gustatory components, and auditory scores were further split into those for verbal vs non-verbal content. Relationships between perceptual abnormality characteristics and conversion were assessed with Cox proportional hazards regression and logistic regression.

RESULTS

Unusual thought content and paranoia were predictive of conversion, but no modality-specific perceptual abnormality score predicted conversion status or days to conversion. However, when auditory perceptual abnormalities were further categorized as verbal vs non-verbal, the severity of verbal experiences was predictive of conversion to psychosis (P = 0.007).

CONCLUSIONS

Perceptual abnormality scores failed to meaningfully predict conversion to psychosis in either direction in this CHR sample. However, verbal auditory experiences may identify a group of CHR individuals at elevated risk of conversion. Further exploration of the relationship between phenomenological aspects of perceptual abnormalities and conversion risk is warranted.

The boundaries of the self: The sense of agency across different sensorimotor aspects

The sense of agency (SoA) is the sensation of control over our actions. SoA is thought to rely mainly upon the comparison of predictions regarding the sensory outcomes of one's actions and the actual sensory outcomes. Previous studies have shown that when a discrepancy is introduced between one's actions and the sensory feedback, the reported SoA is reduced. Experimental manipulations of SoA are typically induced by introducing a discrepancy between a motor action and visual feedback of a specific sensorimotor aspect. For example, introducing a delay or a spatial deviation between the action and its sensory feedback reduces SoA. However, it is yet unclear whether the sensorimotor prediction processes underlying SoA are related between different aspects. Here in one exploratory and one preregistered experiment we tested the sense of agency across temporal, spatial, and anatomical aspects in a within-subject design. Using a novel virtual-reality task allowing the manipulation of the visual feedback of a motor action across different aspects, we show that the sensitivity of agency is different across aspects, agency judgments are correlated across aspects within subjects and bias toward attributing the viewed action to the self or to an external source is correlated as well. Our results suggest that sensorimotor prediction mechanisms underlying SoA are related between different aspects and that people have a predisposition for the directionality of agency judgments. These findings reveal the psychophysical attributes of SoA across sensorimotor aspects. Data and preregistration are available at https://goo.gl/SkbGrb.

Feature-specific prediction errors for visual mismatch

Predictive coding (PC) theory posits that our brain employs a predictive model of the environment to infer the causes of its sensory inputs. A fundamental but untested prediction of this theory is that the same stimulus should elicit distinct precision weighted prediction errors (pwPEs) when different (feature-specific) predictions are violated, even in the absence of attention. Here, we tested this hypothesis using functional magnetic resonance imaging (fMRI) and a multi-feature roving visual mismatch paradigm where rare changes in either color (red, green), or emotional expression (happy, fearful) of faces elicited pwPE responses in human participants. Using a computational model of learning and inference, we simulated pwPE and prediction trajectories of a Bayes-optimal observer and used these to analyze changes in blood oxygen level dependent (BOLD) responses to changes in color and emotional expression of faces while participants engaged in a distractor task. Controlling for visual attention by eye-tracking, we found pwPE responses to unexpected color changes in the fusiform gyrus. Conversely, unexpected changes of facial emotions elicited pwPE responses in cortico-thalamo-cerebellar structures associated with emotion and theory of mind processing. Predictions pertaining to emotions activated fusiform, occipital and temporal areas. Our results are consistent with a general role of PC across perception, from low-level to complex and socially relevant object features, and suggest that monitoring of the social environment occurs continuously and automatically, even in the absence of attention.

A division of labor between power and phase coherence in encoding attention to stimulus streams

Both time-based (when) and feature-based (what) aspects of attention facilitate behavior, so it is natural to hypothesize additive effects. We tested this conjecture by recording response behavior and electroencephalographic (EEG) data to auditory pitch changes, embedded at different time lags in a continuous sound stream. Participants reacted more rapidly to larger rather than smaller feature change magnitudes (deviancy), as well as to changes appearing after longer rather than shorter waiting times (hazard rate of response times). However, the feature and time dimensions of attention separately contributed to response speed, with no significant interaction. Notably, phase coherence at low frequencies (delta and theta bands, 1-7 Hz) predominantly reflected attention capture by feature changes, while oscillatory power at higher frequency bands, alpha (8-12 Hz) and beta (13-25 Hz) reflected the orienting of attention in time. Power and phase coherence predicted different portions of response speed variance, suggesting a division of labor in encoding sensory attention in complex auditory scenes.