A spatio-temporal interaction on the apparent motion trace

Special treatment of prediction errors in autism spectrum disorder

For autistic individuals, sensory stimulation can be experienced as overwhelming. Models of predictive coding postulate that cortical mechanisms disamplify predictable information and amplify prediction errors that surpass a defined precision level. In autism, the neuronal processing is putting an inflexibly high precision on prediction errors according to the HIPPEA theory (High, Inflexible Precision of Prediction Errors in Autism). We used an apparent motion paradigm to test this prediction. In apparent motion paradigms, the illusory motion of an object creates a prediction about where and when an internally generated token would be moving along the apparent motion trace. This illusion facilitates the perception of a flashing stimulus (target) appearing in-time with the apparent motion token and is perceived as a predictable event (predictable target). In contrast, a flashing stimulus appearing out-of-time with the apparent motion illusion is an unpredictable target that is less often detected even though it produces a prediction error signal. If a prediction error does not surpass a given precision threshold the stimulation event is discounted and therefore less often detected than predictable tokens. In autism, the precision threshold is lower and the same prediction errors (unpredictable target) triggers a detection similar to that of a predictable flash stimulus. To test this hypothesis, we recruited 11 autistic males and 9 neurotypical matched controls. The participants were tasked to detect flashing stimuli placed on an apparent motion trace either in-time or out-of-time with the apparent motion illusion. Descriptively, 66% (6/9) of neurotypical and 64% (7/11) of autistic participants were better at detecting predictable targets. The prediction established by illusory motion appears to assist autistic and neurotypical individuals equally in the detection of predictable over unpredictable targets. Importantly, 55% (6/11) of autistic participants had faster responses for unpredictable targets, whereas only 22% (2/9) of neurotypicals had faster responses to unpredictable compared to predictable targets. Hence, these tentative results suggest that for autistic participants, unpredictable targets produce an above threshold prediction error, which leads to faster response. This difference in unpredictable target detection can be encapsulated under the HIPPEA theory, suggesting that precision setting could be aberrant in autistic individuals with respect to prediction errors. These tentative results should be considered in light of the small sample. For this reason, we provide the full set of materials necessary to replicate and extend the results.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

Can expectation suppression be explained by reduced attention to predictable stimuli?

The expectation-suppression effect - reduced stimulus-evoked responses to expected stimuli - is widely considered to be an empirical hallmark of reduced prediction errors in the framework of predictive coding. Here we challenge this notion by proposing that that expectation suppression could be explained by a reduced attention effect. Specifically, we argue that reduced responses to predictable stimuli can also be explained by a reduced saliency-driven allocation of attention. We base our discussion mainly on findings in the visual cortex and propose that resolving this controversy requires the assessment of qualitative differences between the ways in which attention and surprise enhance brain responses.

Tactile interactions in the path of tactile apparent motion

Perceptual completion is a fundamental perceptual function serving to maintain robust perception against noise. For example, we can perceive a vivid experience of motion even for the discrete inputs across time and space (apparent motion: AM). In vision, stimuli irrelevant to AM perception are suppressed to maintain smooth AM perception along the AM trajectory where no physical inputs are applied. We investigated whether such perceptual masking induced by perceptual completion of dynamic inputs is general across sensory modalities by focusing on touch. Participants tried to detect a vibro-tactile target stimulus presented along the trajectory of AM induced by two other tactile stimuli on the forearm. In a control condition, the inducing stimuli were applied simultaneously, resulting in no motion percept. Tactile target detection was impaired with tactile AM. Our findings support the notion that the perceptual masking induced by perceptual completion mechanism of AM is a general function rather than a sensory specific effect.

Apparent Motion Induces Activity Suppression in Early Visual Cortex and Impairs Visual Detection

Apparent motion (AM) is induced when two stationary visual stimuli are presented in alternating sequence. Intriguingly, AM leads to an impaired detectability of stimuli along the AM path (i.e., AM-induced masking). It has been hypothesized that AM triggers an internal representation of a moving object in early visual cortex, which competes with stimulus-evoked representations of visual stimuli on the motion path in early visual cortex of 25 human adults (16 female). We tested this hypothesis by measuring BOLD responses in early visual cortex during the process of AM-induced masking, using fMRI and population receptive field methods. Surprisingly, and counter to our hypothesis, we showed that AM suppressed, rather than increased, BOLD responses along early visual (V1 and V2) representations of the AM path, including regions that were not directly activated by the AM inducer stimuli. This activity suppression of the visual response predicted the subsequent reduction in detectability of the target that appeared in the middle of the AM path. Our data thereby provide direct empirical evidence for suppressive neural mechanisms underlying AM and suggest that illusory motion can render us blind to objects on the motion path by suppressing neural activity at the earliest cortical stages of visual perception.SIGNIFICANCE STATEMENT When two spatially distinct visual objects are presented in alternating sequence, apparent motion (AM) occurs and impairs detectability of stimuli along its path. The underlying mechanism is thought to be that increased activation in human early visual cortex evoked by AM interferes with the representation of the stimulus. Strikingly, however, we show that AM suppresses neural activity along the motion path, and the strength of activity suppression predicts the subsequent behavioral performance decrement in terms of detecting a stimulus along the AM path. Our findings provide empirical evidence for a suppressive, rather than faciliatory, mechanism underlying AM.

Neural responses to apparent motion can be predicted by responses to non-moving stimuli

When two objects are presented in alternation at two locations, they are seen as a single object moving from one location to the other. This apparent motion (AM) percept is experienced for objects located at short and also at long distances. However, current models cannot explain how the brain integrates information over large distances to create such long-range AM. This study investigates the neural markers of AM by parcelling out the contribution of spatial and temporal interactions not specific to motion. In two experiments, participants’ EEG was recorded while they viewed two stimuli inducing AM. Different combinations of these stimuli were also shown in a static context to predict an AM neural response where no motion is perceived. We compared the goodness of fit between these different predictions and found consistent results in both experiments. At short-range, the addition of the inhibitory spatial and temporal interactions not specific to motion improved the AM prediction. However, there was no indication that spatial or temporal non-linear interactions were present at long-range. This suggests that short- and long-range AM rely on different neural mechanisms. Importantly, our results also show that at both short- and long-range, responses generated by a moving stimulus could be well predicted from conditions in which no motion is perceived. That is, the EEG response to a moving stimulus is simply a combination of individual responses to non-moving stimuli. This demonstrates a dissociation between the brain response and the subjective percept of motion.

•

EEG responses are inhibited by spatial and temporal stimulus interactions.

•

These interactions are important for motion at short but not at long distances.

•

We find no trace of a specific neural signature of motion perception.

•

Neural responses to motion are well predicted by responses to non-moving stimuli.

Predictive feedback to V1 dynamically updates with sensory input

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

Changes in fMRI BOLD dynamics reflect anticipation to moving objects

The human brain is thought to respond differently to novel versus predictable neural input. In human visual cortex, neural response amplitude to visual input might be determined by the degree of predictability. We investigated how fMRI BOLD responses in human early visual cortex reflect the anticipation of a single moving bar's trajectory. We found that BOLD signals decreased linearly from onset to offset of the stimulus trajectory. Moreover, decreased amplitudes of BOLD responses coincided with an increased initial dip as the stimulus moved along its trajectory. Importantly, motion anticipation effects were absent, when motion coherence was disrupted by means of stimulus contrast reversals. These results show that human early visual cortex anticipates the trajectory of a coherently moving object at the initial stages of visual motion processing. The results can be explained by suppression of predictable input, plausibly underlying the formation of stable visual percepts.

Early effects of previous experience on conscious perception

Constructive theories of brain function such as predictive coding posit that prior knowledge affects our experience of the world quickly and directly. However, it is yet unknown how swiftly prior knowledge impacts the neural processes giving rise to conscious experience. Here we used an experimental paradigm where prior knowledge augmented perception and measured the timing of this effect with magnetoencephalography (MEG). By correlating the perceptual benefits of prior knowledge with the MEG activity, we found that prior knowledge took effect in the time-window 80–95 ms after stimulus onset, thus reflecting an early influence on conscious perception. The sources of this effect were localized to occipital and posterior parietal regions. These results are in line with the predictive coding framework.

Reconstructing representations of dynamic visual objects in early visual cortex

As raw sensory data are partial, our visual system extensively fills in missing details, creating enriched percepts based on incomplete bottom-up information. Despite evidence for internally generated representations at early stages of cortical processing, it is not known whether these representations include missing information of dynamically transforming objects. Long-range apparent motion (AM) provides a unique test case because objects in AM can undergo changes both in position and in features. Using fMRI and encoding methods, we found that the "intermediate" orientation of an apparently rotating grating, never presented in the retinal input but interpolated during AM, is reconstructed in population-level, feature-selective tuning responses in the region of early visual cortex (V1) that corresponds to the retinotopic location of the AM path. This neural representation is absent when AM inducers are presented simultaneously and when AM is visually imagined. Our results demonstrate dynamic filling-in in V1 for object features that are interpolated during kinetic transformations.

Keeping postdiction simple

Postdiction effects are phenomena in which a stimulus influences the appearance of events taking place before it. In metacontrast masking, for instance, a masking stimulus can render a target stimulus shown before the mask invisible. This and other postdiction effects have been considered incompatible with a simple explanation according to which (i) our perceptual experiences are delayed for only the time it takes for a distal stimulus to reach our sensory receptors and for our neural mechanisms to process it, and (ii) the order in which the processing of stimuli is completed corresponds with the apparent temporal order of stimuli. As a result, the theories that account for more than a single postdiction effect reject at least one of these theses. This paper presents a new framework for the timing of experiences-the non-linear latency difference view-in which the three most discussed postdiction effects-apparent motion, the flash-lag effect, and metacontrast masking-can be accounted for while simultaneously holding theses (i) and (ii). This view is grounded in the local reentrant processes, which are known to have a crucial role in perception. Accordingly, the non-linear latency difference view is both more parsimonious and more empirically plausible than the competing theories, all of which remain largely silent about the neural implementation of the mechanisms they postulate.

Visual Benefits in Apparent Motion Displays: Automatically Driven Spatial and Temporal Anticipation Are Partially Dissociated

Many behaviourally relevant sensory events such as motion stimuli and speech have an intrinsic spatio-temporal structure. This will engage intentional and most likely unintentional (automatic) prediction mechanisms enhancing the perception of upcoming stimuli in the event stream. Here we sought to probe the anticipatory processes that are automatically driven by rhythmic input streams in terms of their spatial and temporal components. To this end, we employed an apparent visual motion paradigm testing the effects of pre-target motion on lateralized visual target discrimination. The motion stimuli either moved towards or away from peripheral target positions (valid vs. invalid spatial motion cueing) at a rhythmic or arrhythmic pace (valid vs. invalid temporal motion cueing). Crucially, we emphasized automatic motion-induced anticipatory processes by rendering the motion stimuli non-predictive of upcoming target position (by design) and task-irrelevant (by instruction), and by creating instead endogenous (orthogonal) expectations using symbolic cueing. Our data revealed that the apparent motion cues automatically engaged both spatial and temporal anticipatory processes, but that these processes were dissociated. We further found evidence for lateralisation of anticipatory temporal but not spatial processes. This indicates that distinct mechanisms may drive automatic spatial and temporal extrapolation of upcoming events from rhythmic event streams. This contrasts with previous findings that instead suggest an interaction between spatial and temporal attention processes when endogenously driven. Our results further highlight the need for isolating intentional from unintentional processes for better understanding the various anticipatory mechanisms engaged in processing behaviourally relevant stimuli with predictable spatio-temporal structure such as motion and speech.

The influence of spontaneous brain oscillations on apparent motion perception

A good example of inferential processes in perception is long-range apparent motion (AM), the illusory percept of visual motion that occurs when two spatially distinct stationary visual objects are presented in alternating sequence. The AM illusion is strongest at presentation frequencies around 3 Hz. At lower presentation frequencies, the percept varies from trial to trial between AM and sequential alternation, while at higher frequencies perception varies between AM and two simultaneously flickering objects. Previous studies have demonstrated that prestimulus alpha oscillations explain trial-to-trial variability in detection performance for visual stimuli presented at threshold. In the present study, we investigated whether fluctuations of prestimulus alpha oscillations can also account for variations in AM perception. Prestimulus alpha power was stronger when observers reported AM perception in subsequent trials with low presentation frequencies, while at high presentation frequencies there were no significant differences in alpha power preceding AM and veridical flicker perception. Moreover, when observers perceived AM the prestimulus functional connectivity between frontal and occipital channels was increased in the alpha band, as revealed by the imaginary part of coherency, which is insensitive to artefacts from volume conduction. Dynamic causal modelling of steady-state responses revealed that the most likely direction of this fronto-occipital connectivity was from frontal to occipital sources. These results point to a role of ongoing alpha oscillations in the inferential process that gives rise to the perception of AM and suggest that fronto-occipital interactions bias perception towards internally generated predictions.

Dissociation of Prediction from Conscious Perception

The framework of predictive coding offers a parsimonious explanation for many perceptual phenomena. According to this framework, perception of the outer world is created by the comparison of incoming sensory information with an internal predictive model based on previous experience and context. However, it is unclear whether the predicted percept needs to enter conscious awareness for the internal predictive model to be effective. Here we used an apparent motion paradigm to show that while prediction and conscious awareness of a predicted percept may coincide, a dissociation can be observed. When sensory information provides reliable input for the internal predictive model, the predicted percept does not have to be consciously perceived for successful prediction. However, when sensory input is ambiguous, conscious awareness helps the prediction to take effect.

TMS over V5 disrupts motion prediction

Given the vast amount of sensory information the brain has to deal with, predicting some of this information based on the current context is a resource-efficient strategy. The framework of predictive coding states that higher-level brain areas generate a predictive model to be communicated via feedback connections to early sensory areas. Here, we directly tested the necessity of a higher-level visual area, V5, in this predictive processing in the context of an apparent motion paradigm. We flashed targets on the apparent motion trace in-time or out-of-time with the predicted illusory motion token. As in previous studies, we found that predictable in-time targets were better detected than unpredictable out-of-time targets. However, when we applied functional magnetic resonance imaging-guided, double-pulse transcranial magnetic stimulation (TMS) over left V5 at 13–53 ms before target onset, the detection advantage of in-time targets was eliminated; this was not the case when TMS was applied over the vertex. Our results are causal evidence that V5 is necessary for a prediction effect, which has been shown to modulate V1 activity (Alink et al. 2010). Thus, our findings suggest that information processing between V5 and V1 is crucial for visual motion prediction, providing experimental support for the predictive coding framework.

Apparent motion can impair and enhance target visibility: the role of shape in predicting and postdicting object continuity

Some previous studies have reported that the visibility of a target in the path of an apparent motion sequence is impaired; other studies have reported that it is facilitated. Here we test whether the relation of shape similarity between the inducing and target stimuli has an influence on visibility. Reasoning from a theoretical framework in which there are both predictive and postdictive influences on shape perception, we report experiments involving three-frame apparent motion sequences. In these experiments, we systematically varied the congruence between target shapes and contextual shapes (preceding and following). Experiment 1 established the baseline visibility of the target, when it was presented in isolation and when it was preceded or followed by a single contextual shape. This set the stage for Experiment 2, where the shape congruence between the target and both contextual shapes was varied orthogonally. The results showed a remarkable degree of synergy between predictive and postdictive influences, allowing a backward-masked shape that was almost invisible when presented in isolation to be discriminated with a d′ of 2 when either of the contextual shapes are congruent. In Experiment 3 participants performed a shape-feature detection task with the same stimuli, with the results indicating that the predictive and postdictive effects were now absent. This finding confirms that shape congruence effects on visibility are specific to shape perception and are not due to either general alerting effects for objects in the path of a motion signal nor to low-level perceptual filling-in.

Detection of visual events along the apparent motion trace in patients with paranoid schizophrenia

Dysfunctional prediction in sensory processing has been suggested as a possible causal mechanism in the development of delusions in patients with schizophrenia. Previous studies in healthy subjects have shown that while the perception of apparent motion can mask visual events along the illusory motion trace, such motion masking is reduced when events are spatio-temporally compatible with the illusion, and, therefore, predictable. Here we tested the hypothesis that this specific detection advantage for predictable target stimuli on the apparent motion trace is reduced in patients with paranoid schizophrenia. Our data show that, although target detection along the illusory motion trace is generally impaired, both patients and healthy control participants detect predictable targets more often than unpredictable targets. Patients had a stronger motion masking effect when compared to controls. However, patients showed the same advantage in the detection of predictable targets as healthy control subjects. Our findings reveal stronger motion masking but intact prediction of visual events along the apparent motion trace in patients with paranoid schizophrenia and suggest that the sensory prediction mechanism underlying apparent motion is not impaired in paranoid schizophrenia.

Transfer of predictive signals across saccades

Predicting visual information facilitates efficient processing of visual signals. Higher visual areas can support the processing of incoming visual information by generating predictive models that are fed back to lower visual areas. Functional brain imaging has previously shown that predictions interact with visual input already at the level of the primary visual cortex (V1; Harrison et al., 2007; Alink et al., 2010). Given that fixation changes up to four times a second in natural viewing conditions, cortical predictions are effective in V1 only if they are fed back in time for the processing of the next stimulus and at the corresponding new retinotopic position. Here, we tested whether spatio-temporal predictions are updated before, during, or shortly after an inter-hemifield saccade is executed, and thus, whether the predictive signal is transferred swiftly across hemifields. Using an apparent motion illusion, we induced an internal motion model that is known to produce a spatio-temporal prediction signal along the apparent motion trace in V1 (Muckli et al., 2005; Alink et al., 2010). We presented participants with both visually predictable and unpredictable targets on the apparent motion trace. During the task, participants saccaded across the illusion whilst detecting the target. As found previously, predictable stimuli were detected more frequently than unpredictable stimuli. Furthermore, we found that the detection advantage of predictable targets is detectable as early as 50-100 ms after saccade offset. This result demonstrates the rapid nature of the transfer of a spatio-temporally precise predictive signal across hemifields, in a paradigm previously shown to modulate V1.

Expectations change the signatures and timing of electrophysiological correlates of perceptual awareness

Previous experience allows the brain to predict what comes next. How these expectations affect conscious experience is poorly understood. In particular, it is unknown whether and when expectations interact with sensory evidence in granting access to conscious perception, and how this is reflected electrophysiologically. Here, we parametrically manipulate sensory evidence and expectations while measuring event-related potentials in human subjects to assess the time course of evoked responses that correlate with subjective visibility, the properties of the stimuli, and/or perceptual expectations. We found that expectations lower the threshold of conscious perception and reduce the latency of neuronal signatures differentiating seen and unseen stimuli. Without expectations, this differentiation occurs ∼300 ms and with expectations ∼200 ms after stimulus in occipitoparietal sensors. The amplitude of this differentiating response component (P2) decreases as visibility increases, regardless of whether this increase is attributable to enhanced sensory evidence and/or the gradual buildup of perceptual expectations. Importantly, at matched performance levels, responses to seen and unseen stimuli differed regardless of the physical stimulus properties. These findings indicate that the latency of the neuronal correlates of access to consciousness depend on whether access is driven by stimulus saliency or by a combination of expectations and sensory evidence.

Stimulus predictability reduces responses in primary visual cortex

In this functional magnetic resonance imaging study we tested whether the predictability of stimuli affects responses in primary visual cortex (V1). The results of this study indicate that visual stimuli evoke smaller responses in V1 when their onset or motion direction can be predicted from the dynamics of surrounding illusory motion. We conclude from this finding that the human brain anticipates forthcoming sensory input that allows predictable visual stimuli to be processed with less neural activation at early stages of cortical processing.

Modulation of perception and brain activity by predictable trajectories of facial expressions

People track facial expression dynamics with ease to accurately perceive distinct emotions. Although the superior temporal sulcus (STS) appears to possess mechanisms for perceiving changeable facial attributes such as expressions, the nature of the underlying neural computations is not known. Motivated by novel theoretical accounts, we hypothesized that visual and motor areas represent expressions as anticipated motion trajectories. Using magnetoencephalography, we show predictable transitions between fearful and neutral expressions (compared with scrambled and static presentations) heighten activity in visual cortex as quickly as 165 ms poststimulus onset and later (237 ms) engage fusiform gyrus, STS and premotor areas. Consistent with proposed models of biological motion representation, we suggest that visual areas predictively represent coherent facial trajectories. We show that such representations bias emotion perception of subsequent static faces, suggesting that facial movements elicit predictions that bias perception. Our findings reveal critical processes evoked in the perception of dynamic stimuli such as facial expressions, which can endow perception with temporal continuity.