Prediction error and repetition suppression have distinct effects on neural representations of visual information

Hierarchical deviant processing in auditory cortex of awake mice

Detecting patterns, and noticing unexpected pattern changes, in the environment is a vital aspect of sensory processing. Adaptation and prediction error responses are two components of neural processing related to these tasks, and previous studies in the auditory system in rodents show that these two components are partially dissociable in terms of the topography and latency of neural responses to sensory deviants. However, many previous studies have focused on repetitions of single stimuli, such as pure tones, which have limited ecological validity. In this study, we tested whether the auditory cortical activity shows adaptation to repetition of more complex sound patterns (disyllabic pairs). Specifically, we compared neural responses to violations of sequences based on single stimulus probability only, against responses to more complex violations based on stimulus order. We employed an auditory oddball paradigm and monitored the auditory cortex (AC) activity of awake mice (N = 8) using wide-field calcium imaging. We found that cortical responses were sensitive both to single stimulus probabilities and to more global stimulus patterns, as mismatch signals were elicited following both substitution deviants and transposition deviants. Notably, higher order AC area elicited larger mismatch signaling to those deviants than primary AC, which suggests a hierarchical gradient of prediction error signaling in the auditory cortex. Such a hierarchical gradient was observed for late but not early peaks of calcium transients to deviants, suggesting that the late part of the deviant response may reflect prediction error signaling in response to more complex sensory pattern violations. SIGNIFICANCE STATEMENT: Detecting the unexpected change of patterns from the dynamic environment is vital for sensory processing, as it is essential to survival for humans and animals. Using wide-field calcium imaging, we investigated whether the auditory cortex of awake mice exhibits a hierarchical gradient of prediction error signaling and its sensitivity to violations of sequences based on stimulus features and stimulus order. We discovered the high-order auditory cortex elicited more significant mismatch signaling to those deviants than primary auditory cortex in substitution and transposition deviants. Calcium transients to deviants showed a hierarchical gradient for late but not for early peaks, indicating that the late part of the deviant response may reflect prediction error signaling in response to more complex sensory pattern violations.

Limited Evidence for Probabilistic Cueing Effects on Grating-Evoked Event-Related Potentials and Orientation Decoding Performance

We can rapidly learn recurring patterns that occur within our sensory environments. This knowledge allows us to form expectations about future sensory events. Several influential predictive coding models posit that, when a stimulus matches our expectations, the activity of feature-selective neurons in the visual cortex will be suppressed relative to when that stimulus is unexpected. However, after accounting for known critical confounds, there is currently scant evidence for these hypothesized effects from studies recording electrophysiological neural activity. To provide a strong test for expectation effects on stimulus-evoked responses in the visual cortex, we performed a probabilistic cueing experiment while recording electroencephalographic (EEG) data. Participants (n = 48) learned associations between visual cues and subsequently presented gratings. A given cue predicted the appearance of a certain grating orientation with 10%, 25%, 50%, 75%, or 90% validity. We did not observe any stimulus expectancy effects on grating-evoked event-related potentials. Multivariate classifiers trained to discriminate between grating orientations performed better when classifying 10% compared to 90% probability gratings. However, classification performance did not substantively differ across any other stimulus expectancy conditions. Our findings provide very limited evidence for modulations of prediction error signaling by probabilistic expectations as specified in contemporary predictive coding models.

Violated predictions enhance the representational fidelity of visual features in perception

Predictive coding theories argue that recent experience establishes expectations that generate prediction errors when violated. In humans, brain imaging studies have revealed unique signatures of violated predictions in sensory cortex, but the perceptual consequences of these effects remain unknown. We had observers perform a dual-report task on the orientation of a briefly presented target grating within predictable or random sequences, while we recorded pupil size as an index of surprise. Observers first made a speeded response to categorize the orientation of the target grating (clockwise or counterclockwise from vertical), then reproduced its orientation without time pressure by rotating a bar. This allowed us to separately assess response speed and precision for the same stimuli. Critically, on half the trials, the target orientation deviated from the spatiotemporal structure established by the preceding gratings. Observers responded faster and more accurately to unexpected gratings, and pupillometry provided physiological evidence of observers' surprise in response to these events. In a second experiment, we cued the spatial location and timing of the grating and found the same pattern of results, demonstrating that unexpected orientation information is sufficient to produce faster and more precise responses, even when the location and timing of the relevant stimuli are fully expected. These findings indicate that unexpected events are prioritized by the visual system both in terms of processing speed and representational fidelity.

Integrated Perceptual Decisions Rely on Parallel Evidence Accumulation

The ability to make accurate and timely decisions, such as judging when it is safe to cross the road, is the foundation of adaptive behavior. While the computational and neural processes supporting simple decisions on isolated stimuli have been well characterized, decision-making in the real world often requires integration of discrete sensory events over time and space. Most previous experimental work on perceptual decision-making has focused on tasks that involve only a single, task-relevant source of sensory input. It remains unclear, therefore, how such integrative decisions are regulated computationally. Here we used psychophysics, electroencephalography, and computational modeling to understand how the human brain combines visual motion signals across space in the service of a single, integrated decision. To that purpose, we presented two random-dot kinematograms in the left and the right visual hemifields. Coherent motion signals were shown briefly and concurrently in each location, and healthy adult human participants of both sexes reported the average of the two motion signals. We directly tested competing predictions arising from influential serial and parallel accounts of visual processing. Using a biologically plausible model of motion filtering, we found evidence in favor of parallel integration as the fundamental computational mechanism regulating integrated perceptual decisions.

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Expectation Modulates Repetition Suppression at Late But Not Early Stages during Visual Word Recognition: Evidence from Event-related Potentials

Visual word recognition is commonly rapid and efficient, incorporating top-down predictive processing mechanisms. Neuroimaging studies with face stimuli suggest that repetition suppression (RS) reflects predictive processing at the neural level, as this effect is larger when repetitions are more frequent, that is, more expected. It remains unclear, however, at the temporal level whether and how RS and its modulation by expectation occur in visual word recognition. To address this gap, the present study aimed to investigate the presence and time course of these effects during visual word recognition using EEG. Thirty-six native Cantonese speakers were presented with pairs of Chinese written words and performed a nonlinguistic oddball task. The second word of a pair was either a repetition of the first or a different word (alternation). In repetition blocks, 75% of trials were repetitions and 25% were alternations, whereas the reverse was true in alternation blocks. Topographic analysis of variance of EEG at each time point showed robust RS effects in three time windows (141-227 msec, 242-445 msec, and 467-513 msec) reflecting facilitation of visual word recognition. Importantly, the modulation of RS by expectation was observed at the late rather than early intervals (334-387 msec, 465-550 msec, and 559-632 msec) and more than 100 msec after the first RS effects. In the predictive coding view of RS, only late repetition effects are modulated by expectation, whereas early RS effects may be mediated by lower-level predictions. Taken together, our findings provide the first EEG evidence revealing distinct temporal dynamics of RS effects and repetition probability on RS effects in visual processing of Chinese words.

Concurrent Encoding of Sequence Predictability and Event-Evoked Prediction Error in Unfolding Auditory Patterns

Human listeners possess an innate capacity to discern patterns within rapidly unfolding sensory input. Core questions, guiding ongoing research, focus on the mechanisms through which these representations are acquired and whether the brain prioritizes or suppresses predictable sensory signals. Previous work, using fast auditory sequences (tone-pips presented at a rate of 20 Hz), revealed sustained response effects that appear to track the dynamic predictability of the sequence. Here, we extend the investigation to slower sequences (4 Hz), permitting the isolation of responses to individual tones. Stimuli were 50 ms tone-pips, ordered into random (RND) and regular (REG; a repeating pattern of 10 frequencies) sequences; Two timing profiles were created: in "fast" sequences, tone-pips were presented in direct succession (20 Hz); in "slow" sequences, tone-pips were separated by a 200 ms silent gap (4 Hz). Naive participants (N = 22; both sexes) passively listened to these sequences, while brain responses were recorded using magnetoencephalography (MEG). Results unveiled a heightened magnitude of sustained brain responses in REG when compared to RND patterns. This manifested from three tones after the onset of the pattern repetition, even in the context of slower sequences characterized by extended pattern durations (2,500 ms). This observation underscores the remarkable implicit sensitivity of the auditory brain to acoustic regularities. Importantly, brain responses evoked by single tones exhibited the opposite pattern-stronger responses to tones in RND than REG sequences. The demonstration of simultaneous but opposing sustained and evoked response effects reveals concurrent processes that shape the representation of unfolding auditory patterns.

Prior probability cues bias sensory encoding with increasing task exposure

When observers have prior knowledge about the likely outcome of their perceptual decisions, they exhibit robust behavioural biases in reaction time and choice accuracy. Computational modelling typically attributes these effects to strategic adjustments in the criterion amount of evidence required to commit to a choice alternative - usually implemented by a starting point shift - but recent work suggests that expectations may also fundamentally bias the encoding of the sensory evidence itself. Here, we recorded neural activity with EEG while participants performed a contrast discrimination task with valid, invalid, or neutral probabilistic cues across multiple testing sessions. We measured sensory evidence encoding via contrast-dependent steady-state visual-evoked potentials (SSVEP), while a read-out of criterion adjustments was provided by effector-selective mu-beta band activity over motor cortex. In keeping with prior modelling and neural recording studies, cues evoked substantial biases in motor preparation consistent with criterion adjustments, but we additionally found that the cues produced a significant modulation of the SSVEP during evidence presentation. While motor preparation adjustments were observed in the earliest trials, the sensory-level effects only emerged with extended task exposure. Our results suggest that, in addition to strategic adjustments to the decision process, probabilistic information can also induce subtle biases in the encoding of the evidence itself.

Enhanced electrophysiological responses to explicitly predicted and pre-imagined inputs, with confirmation from online decoding with neuro-feedback

Neural responses to sensory inputs can scale with the likelihood of encountering the input. This is consistent with the predictive coding framework, in that the human brain is expected to be less responsive to predicted inputs. Typically, however, prediction is not explicitly measured. It is inferred from the probability of encountering an event. When an input is explicitly predicted, responses to predicted inputs can be enhanced. Here, we ask if this effect can be ascribed to a generic priming effect, from pre-cogitating about one of two possible inputs. Consistent with this, we find that P300s (a relatively late event-related potential measured with electroencephalography) are greater for explicitly predicted audio and visual inputs, and that this effect cannot be distinguished from a priming effect from pre-imagining audio or visual presentations. Evidence indicates that participants engaged in pre-imagining presentations, as we were able to decode online what type of presentation (audio or visual) they were imagining with a high success rate (approx. 73%), and we encouraged compliance with neuro-feedback regarding this success rate. Our data confirm that human cortex can be more responsive to inputs that have been subject to pre-cogitation-including explicit predictions. This highlights that while anticipatory processes can reduce responding to likely inputs, they can also enhance responding to explicitly predicted inputs.

Event Probabilities Have a Different Impact on Early and Late Electroencephalographic Measures Regarded as Metrics of Prediction

The oddball protocol has been used to study the neural and perceptual consequences of implicit predictions in the human brain. The protocol involves presenting a sequence of identical repeated events that are eventually broken by a novel "oddball" presentation. Oddball presentations have been linked to increased neural responding and to an exaggeration of perceived duration relative to repeated events. Because the number of repeated events in such protocols is circumscribed, as more repeats are encountered, the conditional probability of a further repeat decreases-whereas the conditional probability of an oddball increases. These facts have not been appreciated in many analyses of oddballs; repeats and oddballs have rather been treated as binary event categories. Here, we show that the human brain is sensitive to conditional event probabilities in an active, visual oddball paradigm. P300 responses (a relatively late component of visually evoked potentials measured with EEG) tended to be greater for less likely oddballs and repeats. By contrast, P1 responses (an earlier component) increased for repeats as a goal-relevant target presentation neared, but this effect occurred even when repeat probabilities were held constant, and oddball P1 responses were invariant. We also found that later, more likely oddballs seemed to last longer, and this effect was largely independent of the number of preceding repeats. These findings speak against a repetition suppression account of the temporal oddball effect. Overall, our data highlight an impact of event probability on later, rather than earlier, electroencephalographic measures previously related to predictive processes-and the importance of considering conditional probabilities in sequential presentation paradigms.

Stimulus expectations do not modulate visual event-related potentials in probabilistic cueing designs

Humans and other animals can learn and exploit repeating patterns that occur within their environments. These learned patterns can be used to form expectations about future sensory events. Several influential predictive coding models have been proposed to explain how learned expectations influence the activity of stimulus-selective neurons in the visual system. These models specify reductions in neural response measures when expectations are fulfilled (termed expectation suppression) and increases following surprising sensory events. However, there is currently scant evidence for expectation suppression in the visual system when confounding factors are taken into account. Effects of surprise have been observed in blood oxygen level dependent (BOLD) signals, but not when using electrophysiological measures. To provide a strong test for expectation suppression and surprise effects we performed a predictive cueing experiment while recording electroencephalographic (EEG) data. Participants (n=48) learned cue-face associations during a training session and were then exposed to these cue-face pairs in a subsequent experiment. Using univariate analyses of face-evoked event-related potentials (ERPs) we did not observe any differences across expected (90% probability), neutral (50%) and surprising (10%) face conditions. Across these comparisons, Bayes factors consistently favoured the null hypothesis throughout the time-course of the stimulus-evoked response. When using multivariate pattern analysis we did not observe above-chance classification of expected and surprising face-evoked ERPs. By contrast, we found robust within- and across-trial stimulus repetition effects. Our findings do not support predictive coding-based accounts that specify reduced prediction error signalling when perceptual expectations are fulfilled. They instead highlight the utility of other types of predictive processing models that describe expectation-related phenomena in the visual system without recourse to prediction error signalling.

Expectation violations enhance neuronal encoding of sensory information in mouse primary visual cortex

The response of cortical neurons to sensory stimuli is shaped both by past events (adaptation) and the expectation of future events (prediction). Here we employed a visual stimulus paradigm with different levels of predictability to characterise how expectation influences orientation selectivity in the primary visual cortex (V1) of male mice. We recorded neuronal activity using two-photon calcium imaging (GCaMP6f) while animals viewed sequences of grating stimuli which either varied randomly in their orientations or rotated predictably with occasional transitions to an unexpected orientation. For single neurons and the population, there was significant enhancement in the gain of orientation-selective responses to unexpected gratings. This gain-enhancement for unexpected stimuli was prominent in both awake and anaesthetised mice. We implemented a computational model to demonstrate how trial-to-trial variability in neuronal responses were best characterised when adaptation and expectation effects were combined.

Distinct early and late neural mechanisms regulate feature-specific sensory adaptation in the human visual system

A canonical feature of sensory systems is that they adapt to prolonged or repeated inputs, suggesting the brain encodes the temporal context in which stimuli are embedded. Sensory adaptation has been observed in the central nervous systems of many animal species, using techniques sensitive to a broad range of spatiotemporal scales of neural activity. Two competing models have been proposed to account for the phenomenon. One assumes that adaptation reflects reduced neuronal sensitivity to sensory inputs over time (the "fatigue" account); the other posits that adaptation arises due to increased neuronal selectivity (the "sharpening" account). To adjudicate between these accounts, we exploited the well-known "tilt aftereffect", which reflects adaptation to orientation information in visual stimuli. We recorded whole-brain activity with millisecond precision from human observers as they viewed oriented gratings before and after adaptation, and used inverted encoding modeling to characterize feature-specific neural responses. We found that both fatigue and sharpening mechanisms contribute to the tilt aftereffect, but that they operate at different points in the sensory processing cascade to produce qualitatively distinct outcomes. Specifically, fatigue operates during the initial stages of processing, consistent with tonic inhibition of feedforward responses, whereas sharpening occurs ~200 ms later, consistent with feedback or local recurrent activity. Our findings reconcile two major accounts of sensory adaptation, and reveal how this canonical process optimizes the detection of change in sensory inputs through efficient neural coding.

Hierarchical Deviant Processing in Auditory Cortex of Awake Mice

Detecting patterns, and noticing unexpected pattern changes, in the environment is a vital aspect of sensory processing. Adaptation and prediction error responses are two components of neural processing related to these tasks, and previous studies in the auditory system in rodents show that these two components are partially dissociable in terms of the topography and latency of neural responses to sensory deviants. However, many previous studies have focused on repetitions of single stimuli, such as pure tones, which have limited ecological validity. In this study, we tested whether the auditory cortical activity shows adaptation to repetition of more complex sound patterns (bisyllabic pairs). Specifically, we compared neural responses to violations of sequences based on single stimulus probability only, against responses to more complex violations based on stimulus order. We employed an auditory oddball paradigm and monitored the auditory cortex (ACtx) activity of awake mice (N=8) using wide-field calcium imaging. We found that cortical responses were sensitive both to single stimulus probabilities and to more global stimulus patterns, as mismatch signals were elicited following both substitution deviants and transposition deviants. Notably, A2 area elicited larger mismatch signaling to those deviants than primary ACtx (A1), which suggests a hierarchical gradient of prediction error signaling in the auditory cortex. Such a hierarchical gradient was observed for late but not early peaks of calcium transients to deviants, suggesting that the late part of the deviant response may reflect prediction error signaling in response to more complex sensory pattern violations.

Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions

While it is universally accepted that the brain makes predictions, there is little agreement about how this is accomplished and under which conditions. Accurate prediction requires neural circuits to learn and store spatiotemporal patterns observed in the natural environment, but it is not obvious how such information should be stored, or encoded. Information theory provides a mathematical formalism that can be used to measure the efficiency and utility of different coding schemes for data transfer and storage. This theory shows that codes become efficient when they remove predictable, redundant spatial and temporal information. Efficient coding has been used to understand retinal computations and may also be relevant to understanding more complicated temporal processing in visual cortex. However, the literature on efficient coding in cortex is varied and can be confusing since the same terms are used to mean different things in different experimental and theoretical contexts. In this work, we attempt to provide a clear summary of the theoretical relationship between efficient coding and temporal prediction, and review evidence that efficient coding principles explain computations in the retina. We then apply the same framework to computations occurring in early visuocortical areas, arguing that data from rodents is largely consistent with the predictions of this model. Finally, we review and respond to criticisms of efficient coding and suggest ways that this theory might be used to design future experiments, with particular focus on understanding the extent to which neural circuits make predictions from efficient representations of environmental statistics.

Neural Circuits for Dynamics-Based Segmentation of Time Series

The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on data sets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series.

Intention-based and sensory-based predictions

We inhabit a continuously changing world, where the ability to anticipate future states of the environment is critical for adaptation. Anticipation can be achieved by learning about the causal or temporal relationship between sensory events, as well as by learning to act on the environment to produce an intended effect. Together, sensory-based and intention-based predictions provide the flexibility needed to successfully adapt. Yet it is currently unknown whether the two sources of information are processed independently to form separate predictions, or are combined into a common prediction. To investigate this, we ran an experiment in which the final tone of two possible four-tone sequences could be predicted from the preceding tones in the sequence and/or from the participants' intention to trigger that final tone. This tone could be congruent with both sensory-based and intention-based predictions, incongruent with both, or congruent with one while incongruent with the other. Trials where predictions were incongruent with each other yielded similar prediction error responses irrespectively of the violated prediction, indicating that both predictions were formulated and coexisted simultaneously. The violation of intention-based predictions yielded late additional error responses, suggesting that those violations underwent further differential processing which the violations of sensory-based predictions did not receive.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Neurophysiology of Human Perceptual Decision-Making

The discovery of neural signals that reflect the dynamics of perceptual decision formation has had a considerable impact. Not only do such signals enable detailed investigations of the neural implementation of the decision-making process but they also can expose key elements of the brain's decision algorithms. For a long time, such signals were only accessible through direct animal brain recordings, and progress in human neuroscience was hampered by the limitations of noninvasive recording techniques. However, recent methodological advances are increasingly enabling the study of human brain signals that finely trace the dynamics of the unfolding decision process. In this review, we highlight how human neurophysiological data are now being leveraged to furnish new insights into the multiple processing levels involved in forming decisions, to inform the construction and evaluation of mathematical models that can explain intra- and interindividual differences, and to examine how key ancillary processes interact with core decision circuits.

Enhanced Sensory Coding in Mouse Vibrissal and Visual Cortex through TRPA1

Transient receptor potential ankyrin 1 (TRPA1) is a non-selective cation channel, broadly expressed throughout the body. Despite its expression in the mammalian brain, little is known about the contribution of TRPA1 to cortical function. Here, we characterize how TRPA1 affects sensory information processing in two cortical areas in mice: the primary vibrissal (whisker) somatosensory cortex (vS1) and the primary visual cortex (V1). In vS1, local activation of TRPA1 by allyl isothiocyanate (AITC) increases the ongoing activity of neurons and their evoked response to vibrissal stimulation, producing a positive gain modulation. The gain modulation is reversed by TRPA1 inhibitor HC-030031 and is absent in TRPA1 knockout mice. Similarly, in V1, TRPA1 activation increases the gain of direction and orientation selectivity. Linear decoding of V1 population activity confirms faster and more reliable encoding of visual signals under TRPA1 activation. Overall, our findings reveal a physiological role for TRPA1 in enhancing sensory signals in the mammalian cortex.

Evaluating the evidence for expectation suppression in the visual system

Reports of expectation suppression have shaped the development of influential predictive coding-based theories of visual perception. However recent work has highlighted confounding factors that may mimic or inflate expectation suppression effects. In this review, we describe four confounds that are prevalent across experiments that tested for expectation suppression: effects of surprise, attention, stimulus repetition and adaptation, and stimulus novelty. With these confounds in mind we then critically review the evidence for expectation suppression across probabilistic cueing, statistical learning, oddball, action-outcome learning and apparent motion designs. We found evidence for expectation suppression within a specific subset of statistical learning designs that involved weeks of sequence learning prior to neural activity measurement. Across other experimental contexts, whereby stimulus appearance probabilities were learned within one or two testing sessions, there was inconsistent evidence for genuine expectation suppression. We discuss how an absence of expectation suppression could inform models of predictive processing, repetition suppression and perceptual decision-making. We also provide suggestions for designing experiments that may better test for expectation suppression in future work.

The time-course of prediction formation and revision in human visual motion processing

Establishing the real-time position of a moving object poses a challenge to the visual system due to neural processing delays. While sensory information is travelling through the visual hierarchy, the object continues moving and information about its position becomes outdated. By extrapolating the position of a moving object along its trajectory, predictive mechanisms might effectively decrease the processing time associated with these objects. Here, we use time-resolved decoding of electroencephalographic (EEG) data from an apparent motion paradigm to demonstrate the interaction of two separate predictive mechanisms. First, we reveal predictive latency advantages for position representations as soon as the second object in an apparent motion sequence - even before the stimulus contains any physical motion energy. This is consistent with the existence of omni-directional, within-layer waves of sub-threshold activity that bring neurons coding for adjacent positions closer to their firing threshold, thereby reducing the processing time of the second stimulus in one of those positions. Second, we show that an additional direction-specific latency advantage emerges from the third sequence position onward, once the direction of the apparent motion stimulus is uniquely determined. Because the receptive fields of early visual areas are too small to encompass sequential apparent motion positions (as evidenced by the lack of latency modulation for the second stimulus position), this latency advantage most likely arises from descending predictions from higher to lower visual areas through feedback connections. Finally, we reveal that the same predictive activation that facilitates the processing of the object in its expected position needs to be overcome when the object's trajectory unexpectedly reverses, causing an additional latency disadvantage for stimuli that violate predictions. Altogether, our results suggest that two complementary mechanisms interact to form and revise predictions in visual motion processing, modulating the latencies of neural position representations at different levels of visual processing.

Can expectation suppression be explained by reduced attention to predictable stimuli?

The expectation-suppression effect - reduced stimulus-evoked responses to expected stimuli - is widely considered to be an empirical hallmark of reduced prediction errors in the framework of predictive coding. Here we challenge this notion by proposing that that expectation suppression could be explained by a reduced attention effect. Specifically, we argue that reduced responses to predictable stimuli can also be explained by a reduced saliency-driven allocation of attention. We base our discussion mainly on findings in the visual cortex and propose that resolving this controversy requires the assessment of qualitative differences between the ways in which attention and surprise enhance brain responses.

Neural mechanism of priming in visual search

Selective attention affords scrutinizing items in our environment. However, attentional selection changes over time and across space. Empirically, repetition of visual search conditions changes attentional processing. Priming of pop-out is a vivid example. Repeatedly searching for the same pop-out search feature is accomplished with faster response times and fewer errors. We review the psychophysical background of priming of pop-out, focusing on the hypothesis that it arises through changes in visual selective attention. We also describe research done with macaque monkeys to understand the neural mechanisms supporting visual selective attention and priming of pop-out, and survey research on priming of pop-out using noninvasive brain measures with humans. We conclude by hypothesizing three alternative neural mechanisms and highlighting open questions.

Diverse Temporal Dynamics of Repetition Suppression Revealed by Intracranial Recordings in the Human Ventral Temporal Cortex

Repeated stimulus presentations commonly produce decreased neural responses-a phenomenon known as repetition suppression (RS) or adaptation-in ventral temporal cortex (VTC) of humans and nonhuman primates. However, the temporal features of RS in human VTC are not well understood. To fill this gap in knowledge, we utilized the precise spatial localization and high temporal resolution of electrocorticography (ECoG) from nine human subjects implanted with intracranial electrodes in the VTC. The subjects viewed nonrepeated and repeated images of faces with long-lagged intervals and many intervening stimuli between repeats. We report three main findings: 1) robust RS occurs in VTC for activity in high-frequency broadband (HFB), but not lower-frequency bands; 2) RS of the HFB signal is associated with lower peak magnitude (PM), lower total responses, and earlier peak responses; and 3) RS effects occur early within initial stages of stimulus processing and persist for the entire stimulus duration. We discuss these findings in the context of early and late components of visual perception, as well as theoretical models of repetition suppression.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Neural dynamics of the attentional blink revealed by encoding orientation selectivity during rapid visual presentation

The human brain is inherently limited in the information it can make consciously accessible. When people monitor a rapid stream of visual items for two targets, they typically fail to see the second target if it occurs within 200–500 ms of the first, a phenomenon called the attentional blink (AB). The neural basis for the AB is poorly understood, partly because conventional neuroimaging techniques cannot resolve visual events displayed close together in time. Here we introduce an approach that characterises the precise effect of the AB on behaviour and neural activity. We employ multivariate encoding analyses to extract feature-selective information carried by randomly-oriented gratings. We show that feature selectivity is enhanced for correctly reported targets and suppressed when the same items are missed, whereas irrelevant distractor items are unaffected. The findings suggest that the AB involves both short- and long-range neural interactions between visual representations competing for access to consciousness.

People often fail to perceive the second of two brief visual targets, a phenomenon known as the attentional blink (AB). Here the authors modelled behaviour and brain activity to show that the AB arises from short- and long-range interactions between representations of elementary visual features.

Learning What Is Irrelevant or Relevant: Expectations Facilitate Distractor Inhibition and Target Facilitation through Distinct Neural Mechanisms

It is well known that attention can facilitate performance by top-down biasing processing of task-relevant information in advance. Recent findings from behavioral studies suggest that distractor inhibition is not under similar direct control but strongly dependent on expectations derived from previous experience. Yet, how expectations about distracting information influence distractor inhibition at the neural level remains unclear. The current study addressed this outstanding question in three experiments in which search displays with repeating distractor or target locations across trials allowed human observers (male and female) to learn which location to selectively suppress or boost. Behavioral findings demonstrated that both distractor and target location learning resulted in more efficient search, as indexed by faster response times. Crucially, distractor learning benefits were observed without target location foreknowledge, unaffected by the number of possible target locations, and could not be explained by priming alone. To determine how distractor location expectations facilitated performance, we applied a spatial encoding model to EEG data to reconstruct activity in neural populations tuned to distractor or target locations. Target location learning increased neural tuning to target locations in advance, indicative of preparatory biasing. This sensitivity increased after target presentation. By contrast, distractor expectations did not change preparatory spatial tuning. Instead, distractor expectations reduced distractor-specific processing, as reflected in the disappearance of the Pd event-related potential component, a neural marker of distractor inhibition, and decreased decoding accuracy. These findings suggest that the brain may no longer process expected distractors as distractors, once it has learned they can safely be ignored.SIGNIFICANCE STATEMENT We constantly try hard to ignore conspicuous events that distract us from our current goals. Surprisingly, and in contrast to dominant attention theories, ignoring distracting, but irrelevant, events does not seem to be as flexible as is focusing our attention on those same aspects. Instead, distractor suppression appears to strongly rely on learned, context-dependent expectations. Here, we investigated how learning about upcoming distractors changes distractor processing and directly contrasted the underlying neural dynamics to target learning. We show that, while target learning enhanced anticipatory sensory tuning, distractor learning only modulated reactive suppressive processing. These results suggest that expected distractors may no longer be considered distractors by the brain once it has learned that they can safely be ignored.

Attention promotes the neural encoding of prediction errors

The encoding of sensory information in the human brain is thought to be optimised by two principal processes: 'prediction' uses stored information to guide the interpretation of forthcoming sensory events, and 'attention' prioritizes these events according to their behavioural relevance. Despite the ubiquitous contributions of attention and prediction to various aspects of perception and cognition, it remains unknown how they interact to modulate information processing in the brain. A recent extension of predictive coding theory suggests that attention optimises the expected precision of predictions by modulating the synaptic gain of prediction error units. Because prediction errors code for the difference between predictions and sensory signals, this model would suggest that attention increases the selectivity for mismatch information in the neural response to a surprising stimulus. Alternative predictive coding models propose that attention increases the activity of prediction (or 'representation') neurons and would therefore suggest that attention and prediction synergistically modulate selectivity for 'feature information' in the brain. Here, we applied forward encoding models to neural activity recorded via electroencephalography (EEG) as human observers performed a simple visual task to test for the effect of attention on both mismatch and feature information in the neural response to surprising stimuli. Participants attended or ignored a periodic stream of gratings, the orientations of which could be either predictable, surprising, or unpredictable. We found that surprising stimuli evoked neural responses that were encoded according to the difference between predicted and observed stimulus features, and that attention facilitated the encoding of this type of information in the brain. These findings advance our understanding of how attention and prediction modulate information processing in the brain, as well as support the theory that attention optimises precision expectations during hierarchical inference by increasing the gain of prediction errors.

Brain signatures of a multiscale process of sequence learning in humans

Extracting the temporal structure of sequences of events is crucial for perception, decision-making, and language processing. Here, we investigate the mechanisms by which the brain acquires knowledge of sequences and the possibility that successive brain responses reflect the progressive extraction of sequence statistics at different timescales. We measured brain activity using magnetoencephalography in humans exposed to auditory sequences with various statistical regularities, and we modeled this activity as theoretical surprise levels using several learning models. Successive brain waves related to different types of statistical inferences. Early post-stimulus brain waves denoted a sensitivity to a simple statistic, the frequency of items estimated over a long timescale (habituation). Mid-latency and late brain waves conformed qualitatively and quantitatively to the computational properties of a more complex inference: the learning of recent transition probabilities. Our findings thus support the existence of multiple computational systems for sequence processing involving statistical inferences at multiple scales.

Prediction error and repetition suppression have distinct effects on neural representations of visual information

Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.