Object-related regularities are processed automatically: evidence from the visual mismatch negativity

Predicting the unpredicted … brain response: A systematic review of the feature-related visual mismatch negativity (vMMN) and the experimental parameters that affect it

In this systematic review and meta-analysis, I consider aspects of experimental design that affect the visual mismatch negativity (vMMN)-an electrophysiological (neural) correlate of prediction error in vision that is typically largest between 150 ms and 300 ms in the event-related potential (ERP) at occipito-parietal regions on the scalp. I compiled data from 145 published studies investigating changes in a single property or feature of visual input. This review provides a concise summary of the vMMN literature on unexpected changes in features of visual input, outlining the most used (according to review) and optimal (following discussion on theoretical and practical implications) parameters of experiments investigating feature deviance for posterity as well as contemporary research. The data compiled was analysed to reveal meaningful relationships between aspects of experimental design and vMMN mean amplitude and peak latency. Results suggest that whether a control for adaptation is used, whether attention is towards vs. away from the stimulus of interest, and stimulus presentation time determines mean amplitude. Whether attention is towards vs. away from the stimulus of interest, the time between the stimulus of interest, deviant probability, and the number of standards separating deviants determines peak latency. There is also some indication that magnitude of deviance affects mean amplitude in studies exploring orientation deviance. This review and its findings elucidate potentially fruitful areas of future research.

Distinguishing expectation and attention effects in processing temporal patterns of visual input

The current study investigated how the brain sets up expectations from stimulus regularities by evaluating the neural responses to expectations driven implicitly (by the stimuli themselves) and explicitly (by task demands). How the brain uses prior information to create expectations and what role attention plays in forming or holding predictions to efficiently respond to incoming sensory information is still debated. We presented temporal patterns of visual input while recording EEG under two different task conditions. When the patterns were task-relevant and pattern recognition was required to perform the button press task, three different event-related brain potentials (ERPs) were elicited, each reflecting a different aspect of pattern expectation. In contrast, when the patterns were task-irrelevant, none of the neural indicators of pattern recognition or pattern violation detection were observed to the same temporally structured sequences. Thus, results revealed a clear distinction between expectation and attention that was prompted by task requirements. These results provide complementary pieces of evidence that implicit exposure to a stimulus pattern may not be sufficient to drive neural effects of expectations that lead to predictive error responses. Task-driven attentional control can dissociate from stimulus-driven expectations, to effectively minimize distracting information and maximize attentional regulation.

Visual mismatch negativity is more than the sum of microsequences

Visual mismatch negativity (vMMN), the difference between the event-related potentials (ERPs) to repeated (standard) events and changing (deviant) events, can be caused either by diminished activity to the repeated ones (stimulus-specific adaptation, SSA), increased activity to the new ones, or both effects. To determine which of these effects contribute to the emergence of vMMN, we investigated the effect of repetition on visual ERPs. To this end, we measured electrical brain activity to task-irrelevant stimuli both in case of stimulus onset (continuously present objects, ON-events) and stimulus offset (frequently or infrequently disappearing parts of the objects, OFF-events). We examined ERPs to changing events ("deviants"), first and second repetitions ("standards") and to changing events preceded by one, two or three different events in non-oddball (50 % deviant frequency) sequences along with ERPs to similar deviant and standard events in oddball (12,5 % deviant frequency) sequences. The absence of exogenous N1 adaptation (i.e., adaptation of the negativity in the range of putative deviant minus standard difference) in non-oddball sequences coupled with the emergence of vMMN in oddball sequences proves that vMMN can be considered more than pure stimulus-specific adaptation effect.

Prediction errors in surface segmentation are reflected in the visual mismatch negativity, independently of task and surface features

The visual system quickly registers perceptual regularities in the environment and responds to violations in these patterns. Errors of perceptual prediction are associated with electrocortical modulation, including the visual mismatch negativity (vMMN) and P2 event-related potential. One relatively unexplored question is whether these prediction error signals can encode higher-level properties such as surface segmentation, or whether they are limited to lower-level perceptual features. Using a roving standard paradigm, a triangle surface appeared either behind (featuring amodal contours) or in front of (featuring real contours) a second surface with hole-like windows. A surface layout appeared for two to five repetitions before switching to the other "deviant" layout; lighting and orientation of stimuli varied across presentations while remaining isoluminant. Observers responded when they detected a rare "pinched" triangle, which occasionally appeared. Cortical activity-reflected in mismatch responses affecting the P2-N2 and P300 amplitudes-was sensitive to a change in stimulus layout, when surfaces shifted position in depth, following several repetitions. Specifically, layout deviants led to a more negative P2-N2 complex at posterior electrodes, and greater P300 positivity at central sites. Independently of these signals of a deviant surface layout, further modulations of the P2 encoded differences between layouts and detection of the rare target stimulus. Comparison of the effect of preceding layout repetitions on this prediction error signal suggests that it is all or none and not graded with respect to the number of previous repetitions. We show that within the visual domain, unnoticed and task-irrelevant changes in visual surface segmentation leads to observable electrophysiological signals of prediction error that are dissociable from stimulus-specific encoding and lower-level perceptual processing.

Feature-specific prediction errors for visual mismatch

Predictive coding (PC) theory posits that our brain employs a predictive model of the environment to infer the causes of its sensory inputs. A fundamental but untested prediction of this theory is that the same stimulus should elicit distinct precision weighted prediction errors (pwPEs) when different (feature-specific) predictions are violated, even in the absence of attention. Here, we tested this hypothesis using functional magnetic resonance imaging (fMRI) and a multi-feature roving visual mismatch paradigm where rare changes in either color (red, green), or emotional expression (happy, fearful) of faces elicited pwPE responses in human participants. Using a computational model of learning and inference, we simulated pwPE and prediction trajectories of a Bayes-optimal observer and used these to analyze changes in blood oxygen level dependent (BOLD) responses to changes in color and emotional expression of faces while participants engaged in a distractor task. Controlling for visual attention by eye-tracking, we found pwPE responses to unexpected color changes in the fusiform gyrus. Conversely, unexpected changes of facial emotions elicited pwPE responses in cortico-thalamo-cerebellar structures associated with emotion and theory of mind processing. Predictions pertaining to emotions activated fusiform, occipital and temporal areas. Our results are consistent with a general role of PC across perception, from low-level to complex and socially relevant object features, and suggest that monitoring of the social environment occurs continuously and automatically, even in the absence of attention.

Preattentive and Predictive Processing of Visual Motion

Interaction with the environment requires fast and reliable sensory processing. The visual system is confronted with a continuous flow of high-dimensional input (e.g. orientation, color, motion). From a theoretical point of view, it would be advantageous if critical information was processed independent of attentional load, i.e. preattentively. Here, we hypothesized that visual motion is such a critical signal and aimed for a neural signature of its preattentive encoding. Furthermore, we were interested in the neural correlates of predictability of linear motion trajectories based on the presence or absence of preceding motion. We presented a visual oddball paradigm and studied event-related potentials (ERPs). Stimuli were linearly moving Gabor patches that disappeared behind an occluder. The difference between deviant and standard trials was a trajectory change which happened behind the occluder in deviant trials only, inducing a prediction error. As hypothesized, we found a visual mismatch negativity-component over parietal and occipital electrodes. In a further condition, trials without preceding motion were presented in which the patch just appeared from behind the occluder and, hence, was not predictable. We found larger ERP-components for unpredictable stimuli. In summary, our results provide evidence for a preattentive and predictive processing of linear trajectories of visual motion.

Oscillatory Encoding of Visual Stimulus Familiarity

Familiarity of the environment changes the way we perceive and encode incoming information. However, the neural substrates underlying this phenomenon are poorly understood. Here we describe a new form of experience-dependent low-frequency oscillations in the primary visual cortex (V1) of awake adult male mice. The oscillations emerged in visually evoked potentials and single-unit activity following repeated visual stimulation. The oscillations were sensitive to the spatial frequency content of a visual stimulus and required the mAChRs for their induction and expression. Finally, ongoing visually evoked θ (4-8 Hz) oscillations boost the visually evoked potential amplitude of incoming visual stimuli if the stimuli are presented at the high excitability phase of the oscillations. Our results demonstrate that an oscillatory code can be used to encode familiarity and serves as a gate for oncoming sensory inputs.SIGNIFICANCE STATEMENT Previous experience can influence the processing of incoming sensory information by the brain and alter perception. However, the mechanistic understanding of how this process takes place is lacking. We have discovered that persistent low-frequency oscillations in the primary visual cortex encode information about familiarity and the spatial frequency of the stimulus. These familiarity evoked oscillations influence neuronal responses to the oncoming stimuli in a way that depends on the oscillation phase. Our work demonstrates a new mechanism of visual stimulus feature detection and learning.

Do early neural correlates of visual consciousness show the oblique effect? A binocular rivalry and event-related potential study

When dissimilar images are presented one to each eye, we do not see both images; rather, we see one at a time, alternating unpredictably. This is called binocular rivalry, and it has recently been used to study brain processes that correlate with visual consciousness, because perception changes without any change in the sensory input. Such studies have used various types of images, but the most popular have been gratings: sets of bright and dark lines of orthogonal orientations presented one to each eye. We studied whether using cardinal rival gratings (vertical, 0°, and horizontal, 90°) versus oblique rival gratings (left-oblique, -45°, and right-oblique, 45°) influences early neural correlates of visual consciousness, because of the oblique effect: the tendency for visual performance to be greater for cardinal gratings than for oblique gratings. Participants viewed rival gratings and pressed keys indicating which of the two gratings they perceived, was dominant. Next, we changed one of the gratings to match the grating shown to the other eye, yielding binocular fusion. Participants perceived the rivalry-to-fusion change to the dominant grating and not to the other, suppressed grating. Using event-related potentials (ERPs), we found neural correlates of visual consciousness at the P1 for both sets of gratings, as well as at the P1-N1 for oblique gratings, and we found a neural correlate of the oblique effect at the N1, but only for perceived changes. These results show that the P1 is the earliest neural activity associated with visual consciousness and that visual consciousness might be necessary to elicit the oblique effect.

Visual mismatch negativity to vanishing parts of objects in younger and older adults

We investigated visual mismatch negativity (vMMN) to vanishing parts of continuously present objects by comparing the event-related potentials (ERPs) to infrequently (deviant) and frequently (standard) disappearing parts of the objects. This paradigm both excludes low-level stimulus-specific adaptation differences between the responses to deviants and standards, and increases the ecological validity of the stimuli. In comparison to frequently disappearing parts of the stimulus objects, infrequently vanishing parts elicited posterior negative event-related brain activity (vMMN). However, no vMMN emerged to the reappearance of the same parts of the objects. We compared the ERPs of an older and a younger sample of participants. In the 120-180 ms time period vMMN was similar in the two age groups, but in the 180-220 ms time period vMMN emerged only in the younger participants. We consider this difference as an index of more elaborate automatic processing of infrequent stimulus changes in younger adults.

When Elderly Outperform Young Adults-Integration in Vision Revealed by the Visual Mismatch Negativity

We studied the possibility of age-related differences of visual integration at an automatic and at a task-related level. Data of 15 young (21.9 ± 1.8 years) and 15 older (66.6 ± 3.5 years) women were analyzed in our experiment. Automatic processing was investigated in a passive oddball paradigm, and the visual mismatch negativity (vMMN) of event-related brain potentials was measured. Letters and pseudo-letters were presented either as single characters, or the characters were presented successively in two fragments. In case of simultaneous presentation of the two fragments (whole character) vMMN emerged in both age groups. However, in successive presentation vMMN was elicited only by the deviant pseudo-letters, and only in the older group. The longest stimulus onset asynchrony (SOA) in this group was 50 ms, indicating longer information persistence in elderly. In a psychophysical experiment, the task was to indicate, which member of a character pair was a legal letter. Again, the letters and pseudo-letters were presented as fragments. We obtained successful integration at 30 ms (0 ms interstimulus interval), but not at longer SOAs in both age groups, showing that in case of task-relevant stimulation level there was no detectable age-related performance difference. We interpreted the results as the efficiency of local inhibitory circuits is compromised in elderly, leading to longer stimulus persistence, and hence better visual perception in this particular case.

Visual mismatch negativity: a predictive coding view

An increasing number of studies investigate the visual mismatch negativity (vMMN) or use the vMMN as a tool to probe various aspects of human cognition. This paper reviews the theoretical underpinnings of vMMN in the light of methodological considerations and provides recommendations for measuring and interpreting the vMMN. The following key issues are discussed from the experimentalist's point of view in a predictive coding framework: (1) experimental protocols and procedures to control "refractoriness" effects; (2) methods to control attention; (3) vMMN and veridical perception.