Feature integration is unaffected by saccade landing point, even when saccades land outside of the range of regular oculomotor variance

Trans-saccadic integration for object recognition peters out with pre-saccadic object eccentricity as target-directed saccades become more saliency-driven

Bringing objects from peripheral locations to fovea via saccades facilitates their recognition. Human observers integrate pre- and post-saccadic information for recognition. This integration has only been investigated using instructed saccades to prescribed locations. Typically, the target has a fixed pre-saccadic location in an uncluttered scene and is viewed by a pre-determined post-saccadic duration. Consequently, whether trans-saccadic integration is limited or absent when the pre-saccadic target eccentricity is too large in cluttered scenes in unknown. Our study revealed this limit during visual exploration, when observers decided themselves when and to where to make their saccades. We asked thirty observers (400 trials each) to find and report as quickly as possible a target amongst 404 non-targets in an image spanning 57.3°×33.8° in visual angle. We measured the target's pre-saccadic eccentricity e, the duration Tpre of the fixation before the saccade, and the post-saccadic foveal viewing duration Tpost. This Tpost increased with e before starting to saturate around eccentricity ep=10°-20°. Meanwhile, Tpre increased much more slowly with e and started decreasing before ep. These observations imply the following at sufficiently large pre-saccadic eccentricities: the trans-saccadic integration ceases, target recognition relies exclusively on post-saccadic foveal vision, decision to saccade to the target relies exclusively on target saliency rather than identification. These implications should be applicable to general behavior, although ep should depend on object and scene properties. They are consistent with the Central-peripheral Dichotomy that central and peripheral vision are specialized for seeing and looking, respectively.

Trans-retinal predictive signals of visual features are precise, saccade-specific and operate over a wide range of spatial frequencies

Saccadic eye movements successively project the saccade target on two retinal locations: a peripheral one before the saccade, and the fovea after the saccade. Typically, performance in discriminating stimulus features changes between these two projections is very poor. However, a short (∼200 ms) blanking of the target upon saccade onset drastically improves performance, demonstrating that a precise signal of the peripheral projection is retained during the saccade. Although little is known about the nature of that transsaccadic signal, previous reports conjectured that it relies on information processed by the magnocellular system. Across two experiments, we investigated the feature blanking effect for a wide range of spatial frequencies (0.5-8 cycles per degree of visual angle, dva), stimulus sizes (1-4 dva), and eccentricities (6-10 dva). In each trial, participants executed a saccade to a high-contrast grating presented either left or right of fixation. During the saccade, the grating changed orientation (clockwise or counter-clockwise) either instantaneously or after a 200-ms blank, and participants reported the change's direction. We contrasted this saccade condition with a trans-retinal fixation condition mimicking the peripheral-then-foveal sequence of the target stimulus occurring across a saccade. Remarkably, blanking improved performance reliably for each spatial frequency, stimulus size, and eccentricity, but only in the saccade condition. Performance with blanking in saccade trials systematically exceeded performance in the fixation condition. Our results demonstrate a robust feature blanking effect across saccades, suggesting that transsaccadic processes involve low-level visual features beyond those processed in the magnocellular system.NEW & NOTEWORTHY Across a saccadic eye movement, the visual system is able to keep track of the signals carrying the visual features of a saccade target. We provide evidence that these signals are sensitive to a wide range of stimulus sizes, can use a wide range spatial frequencies channels and, operate at various saccade amplitudes. Our results suggest an underlying mechanism operating beyond the magnocellular pathway that is contingent to saccade execution.

The role of color in transsaccadic object correspondence

With each saccade, visual information is disrupted, and the visual system is tasked with establishing object correspondence between the presaccadic and postsaccadic representations of the saccade target. There is substantial evidence that the visual system consults spatiotemporal continuity when determining object correspondence across saccades. The evidence for surface feature continuity, however, is mixed. Surface features that are integral to the saccade target object's identity (e.g., shape and contrast polarity) are informative of object continuity, but features that may only imply the state of the object (e.g., orientation) are ignored. The present study tested whether color information is consulted to determine transsaccadic object continuity. We used two variations of the intrasaccadic target displacement task. In Experiments 1 and 2, participants reported the direction of the target displacement. In Experiments 3 and 4, they instead reported whether they detected any target movement. In all experiments, we manipulated the saccade target's continuity by removing it briefly (i.e., blanking) and by changing its color. We found that large color changes can disrupt stability and increase sensitivity to displacements for both direction and movement reports, although not as strongly as long blank durations (250 ms). Interestingly, even smaller color changes, but not blanking, reduced response biases. These results indicate that disrupting surface feature continuity may impact the process of transsaccadic object correspondence more strongly than spatiotemporal disruptions by both increasing the sensitivity and decreasing the response bias.

Familiar objects benefit more from transsaccadic feature predictions

The transsaccadic feature prediction mechanism associates peripheral and foveal information belonging to the same object to make predictions about how an object seen in the periphery would appear in the fovea or vice versa. It is unclear if such transsaccadic predictions require experience with the object such that only familiar objects benefit from this mechanism by virtue of having peripheral-foveal associations. In two experiments, we tested whether familiar objects have an advantage over novel objects in peripheral-foveal matching and transsaccadic change detection tasks. In both experiments, observers were unknowingly familiarized with a small set of stimuli by completing a sham orientation change detection task. In the first experiment, observers subsequently performed a peripheral-foveal matching task, where they needed to pick the foveal test object that matched a briefly presented peripheral target. In the second experiment, observers subsequently performed a transsaccadic object change detection task where a peripheral target was exchanged or not exchanged with another target after the saccade, either immediately or after a 300-ms blank period. We found an advantage of familiar objects over novel objects in both experiments. While foveal-peripheral associations explained the familiarity effect in the matching task of the first experiment, the second experiment provided evidence for the advantage of peripheral-foveal associations in transsaccadic object change detection. Introducing a postsaccadic blank improved change detection performance in general but more for familiar than for novel objects. We conclude that familiar objects benefit from additional object-specific predictions.

Post-saccadic changes disrupt attended pre-saccadic object memory

Trans-saccadic memory consists of keeping track of objects’ locations and features across saccades; pre-saccadic information is remembered and compared with post-saccadic information. It has been shown to have limited resources and involve attention with respect to the selection of objects and features. In support, a previous study showed that recognition of distinct post-saccadic objects in the visual scene is impaired when pre-saccadic objects are relevant and thus already encoded in memory (Poth, Herwig, Schneider, 2015). Here, we investigated the inverse (i.e. how the memory of pre-saccadic objects is affected by abrupt but irrelevant changes in the post-saccadic visual scene). We also modulated the amount of attention to the relevant pre-saccadic object by having participants either make a saccade to it or elsewhere and observed that pre-saccadic attentional facilitation affected how much post-saccadic changes disrupted trans-saccadic memory of pre-saccadic objects.

Participants identified a flashed symbol (d, b, p, or q, among distracters), at one of six placeholders (figures “8”) arranged in circle around fixation while planning a saccade to one of them. They reported the identity of the symbol after the saccade. We changed the post-saccadic scene in Experiment one by removing the entire scene, only the placeholder where the pre-saccadic symbol was presented, or all other placeholders except this one. We observed reduced identification performance when only the saccade-target placeholder disappeared after the saccade. In Experiment two, we changed one placeholder location (inward/outward shift or rotation re. saccade vector) after the saccade and observed that identification performance decreased with increased shift/rotation of the saccade-target placeholder. We conclude that pre-saccadic memory is disrupted by abrupt attention-capturing post-saccadic changes of visual scene, particularly when these changes involve the object prioritized by being the goal of a saccade. These findings support the notion that limited trans-saccadic memory resources are disrupted when object correspondence at saccadic goal is broken through removal or location change.

Transsaccadic perception is affected by saccade landing point deviations after saccadic adaptation

Whenever we move our eyes, some visual information obtained before a saccade is combined with the visual information obtained after a saccade. Interestingly, saccades rarely land exactly on the saccade target, which may pose a problem for transsaccadic perception as it could affect the quality of postsaccadic input. Recently, however, we showed that transsaccadic feature integration is actually unaffected by deviations of saccade landing points. Possibly, transsaccadic integration remains unaffected because the presaccadic shift of attention follows the intended saccade target and not the actual saccade landing point during regular saccades. Here, we investigated whether saccade landing point errors can in fact alter transsaccadic perception when the presaccadic shift of attention follows the saccade landing point deviation. Given that saccadic adaptation not only changes the saccade vector, but also the presaccadic shift of attention, we combined a feature report paradigm with saccadic adaptation. Observers reported the color of the saccade target, which occasionally changed slightly during a saccade to the target. This task was performed before and after saccadic adaptation. The results showed that, after adaptation, presaccadic color information became less precise and transsaccadic perception had a stronger reliance on the postsaccadic color estimate. Therefore, although previous studies have shown that transsaccadic perception is generally unaffected by saccade landing point deviations, our results reveal that this cannot be considered a general property of the visual system. When presaccadic shifts of attention follow altered saccade landing points, transsaccadic perception is affected, suggesting that transsaccadic feature perception might be dependent on visual spatial attention.

A review of interactions between peripheral and foveal vision

Visual processing varies dramatically across the visual field. These differences start in the retina and continue all the way to the visual cortex. Despite these differences in processing, the perceptual experience of humans is remarkably stable and continuous across the visual field. Research in the last decade has shown that processing in peripheral and foveal vision is not independent, but is more directly connected than previously thought. We address three core questions on how peripheral and foveal vision interact, and review recent findings on potentially related phenomena that could provide answers to these questions. First, how is the processing of peripheral and foveal signals related during fixation? Peripheral signals seem to be processed in foveal retinotopic areas to facilitate peripheral object recognition, and foveal information seems to be extrapolated toward the periphery to generate a homogeneous representation of the environment. Second, how are peripheral and foveal signals re-calibrated? Transsaccadic changes in object features lead to a reduction in the discrepancy between peripheral and foveal appearance. Third, how is peripheral and foveal information stitched together across saccades? Peripheral and foveal signals are integrated across saccadic eye movements to average percepts and to reduce uncertainty. Together, these findings illustrate that peripheral and foveal processing are closely connected, mastering the compromise between a large peripheral visual field and high resolution at the fovea.

Object-mediated overwriting across saccades

How are visual sensory representations that are acquired peripherally from a saccade target related to sensory representations generated foveally after the saccade? We tested the hypothesis that, when the two representations are perceived to belong to the same object, the post-saccadic value tends to overwrite the pre-saccadic value. Participants executed a saccade to a colored target object, which sometimes changed during the saccade by ±15°, 30°, or 45° in color space. They were post-cued to report either the pre-saccadic or post-saccadic color in a continuous report procedure. Substantial overwriting of the pre-saccadic color by the post-saccadic color was observed. Moreover, the introduction of a brief post-saccadic blank interval (which disrupted the perception of object correspondence) led to a substantial reduction in overwriting. The results provide the first direct evidence for an object-mediated overwriting mechanism across saccades, in which post-saccadic values automatically replace pre-saccadic values.

Immersive virtual environments and embodied agents for e-learning applications

Massive Open Online Courses are a dominant force in remote-learning yet suffer from persisting problems stemming from lack of commitment and low completion rates. In this initial study we investigate how the use of immersive virtual environments for Power-Point based informational learning may benefit learners and mimic traditional lectures successfully. We examine the role of embodied agent tutors which are frequently implemented within virtual learning environments. We find similar performance on a bespoke knowledge test and metrics for motivation, satisfaction, and engagement by learners in both real and virtual environments, regardless of embodied agent tutor presence. Our results raise questions regarding the viability of using virtual environments for remote-learning paradigms, and we emphasise the need for further investigation to inform the design of effective remote-learning applications.

Effects of Transient Loss of Vision on Head and Eye Movements during Visual Search in a Virtual Environment

Central and peripheral fields of view extract information of different quality and serve different roles during visual tasks. Past research has studied this dichotomy on-screen in conditions remote from natural situations where the scene would be omnidirectional and the entire field of view could be of use. In this study, we had participants looking for objects in simulated everyday rooms in virtual reality. By implementing a gaze-contingent protocol we masked central or peripheral vision (masks of 6 deg. of radius) during trials. We analyzed the impact of vision loss on visuo-motor variables related to fixation (duration) and saccades (amplitude and relative directions). An important novelty is that we segregated eye, head and the general gaze movements in our analyses. Additionally, we studied these measures after separating trials into two search phases (scanning and verification). Our results generally replicate past on-screen literature and teach about the role of eye and head movements. We showed that the scanning phase is dominated by short fixations and long saccades to explore, and the verification phase by long fixations and short saccades to analyze. One finding indicates that eye movements are strongly driven by visual stimulation, while head movements serve a higher behavioral goal of exploring omnidirectional scenes. Moreover, losing central vision has a smaller impact than reported on-screen, hinting at the importance of peripheral scene processing for visual search with an extended field of view. Our findings provide more information concerning how knowledge gathered on-screen may transfer to more natural conditions, and attest to the experimental usefulness of eye tracking in virtual reality.

Stimulus blanking reveals contrast-dependent transsaccadic feature transfer

Across saccadic eye movements, the visual system receives two successive static images corresponding to the pre- and the postsaccadic projections of the visual field on the retina. The existence of a mechanism integrating the content of these images is today still a matter of debate. Here, we studied the transfer of a visual feature across saccades using a blanking paradigm. Participants moved their eyes to a peripheral grating and discriminated a change in its orientation occurring during the eye movement. The grating was either constantly on the screen or briefly blanked during and after the saccade. Moreover, it either was of the same luminance as the background (i.e., isoluminant) or anisoluminant with respect to it. We found that for anisoluminant gratings, the orientation discrimination across saccades was improved when a blank followed the onset of the eye movement. Such effect was however abolished with isoluminant gratings. Additionally, performance was also improved when an anisoluminant grating presented before the saccade was followed by an isoluminant one. These results demonstrate that a detailed representation of the presaccadic image was transferred across saccades allowing participants to perform better on the transsaccadic orientation task. While such a transfer of visual orientation across saccade is masked in real-life anisoluminant conditions, the use of a blank and of an isoluminant postsaccadic grating allowed to reveal its existence.

Stronger saccadic suppression of displacement and blanking effect in children

Humans do not notice small displacements to objects that occur during saccades, termed saccadic suppression of displacement (SSD), and this effect is reduced when a blank is introduced between the pre- and postsaccadic stimulus (Bridgeman, Hendry, & Stark, 1975; Deubel, Schneider, & Bridgeman, 1996). While these effects have been studied extensively in adults, it is unclear how these phenomena are characterized in children. A potentially related mechanism, saccadic suppression of contrast sensitivity-a prerequisite to achieve a stable percept-is stronger for children (Bruno, Brambati, Perani, & Morrone, 2006). However, the evidence for how transsaccadic stimulus displacements may be suppressed or integrated is mixed. While they can integrate basic visual feature information from an early age, they cannot integrate multisensory information (Gori, Viva, Sandini, & Burr, 2008; Nardini, Jones, Bedford, & Braddick, 2008), suggesting a failure in the ability to integrate more complex sensory information. We tested children 7 to 12 years old and adults 19 to 23 years old on their ability to perceive intrasaccadic stimulus displacements, with and without a postsaccadic blank. Results showed that children had stronger SSD than adults and a larger blanking effect. Children also had larger undershoots and more variability in their initial saccade endpoints, indicating greater intrinsic uncertainty, and they were faster in executing corrective saccades to account for these errors. Together, these results suggest that children may have a greater internal expectation or prediction of saccade error than adults; thus, the stronger SSD in children may be due to higher intrinsic uncertainty in target localization or saccade execution.

Spatiotopic and saccade-specific transsaccadic memory for object detail

The content and nature of transsaccadic memory are still a matter of debate. Brief postsaccadic target blanking was demonstrated to recover transsaccadic memory and defeat saccadic suppression of displacement. We examined whether blanking would also support transsaccadic transfer of detailed form information. Observers saccaded to a peripheral, checkerboard-like stimulus and reported whether an intrasaccadic change had occurred in its upper or lower half. On half of the trials, the stimulus was blanked for 200 ms with saccade onset. In a fixation condition, observers kept fixation but the stimulus was displaced from periphery to fixation, mimicking the retinal events of the saccade condition. Results show that stimulus blanking improves transsaccadic change detection, with performance being far superior to the retinally equivalent fixation condition. Our findings argue in favor of a remapped memory trace that can be accessed only in the blanking condition, when not being overwritten by the salient postsaccadic stimulus.

Transsaccadic integration is dominated by early, independent noise

Humans are able to integrate pre- and postsaccadic percepts of an object across saccades to maintain perceptual stability. Previous studies have used Maximum Likelihood Estimation (MLE) to determine that integration occurs in a near-optimal manner. Here, we compared three different models to investigate the mechanism of integration in more detail: an early noise model, where noise is added to the pre- and postsaccadic signals before integration occurs; a late-noise model, where noise is added to the integrated signal after integration occurs; and a temporal summation model, where integration benefits arise from the longer transsaccadic presentation duration compared to pre- and postsaccadic presentation only. We also measured spatiotemporal aspects of integration to determine whether integration can occur for very brief stimulus durations, across two hemifields, and in spatiotopic and retinotopic coordinates. Pre-, post-, and transsaccadic performance was measured at different stimulus presentation durations, both at the saccade target and a location where the pre- and postsaccadic stimuli were presented in different hemifields across the saccade. Results showed that for both within- and between-hemifields conditions, integration could occur when pre- and postsaccadic stimuli were presented only briefly, and that the pattern of integration followed an early noise model. Whereas integration occurred when the pre- and post-saccadic stimuli were presented in the same spatiotopic coordinates, there was no integration when they were presented in the same retinotopic coordinates. This contrast suggests that transsaccadic integration is limited by early, independent, sensory noise acting separately on pre- and postsaccadic signals.

What Color Was It? A Psychophysical Paradigm for Tracking Subjective Progress in Continuous Tasks

When making a sequence of fixations, how does the timing of visual experience compare with the timing of fixation onsets? Previous studies have tracked shifts of attention or perceived gaze direction using self-report methods. We used a similar method, a dynamic color technique, to measure subjective timing in continuous tasks involving fixation sequences. Does the time that observers report reading a word coincide with their fixation on it, or is there an asynchrony, and does this relationship depend on the observer’s task? Observers read sentences that continuously changed in hue and identified the color of a word at the time that they read it using a color palette. We compared responses with a nonreading condition, where observers reproduced their fixations, but viewed nonword stimuli. Results showed a delay between the color of stimuli at fixation onset and the reported color during perception. For nonword tasks, the delay was constant. However, in the reading task, the delay was larger for earlier compared with later words in the sentence. Our results offer a new method for measuring awareness or subjective progress within fixation sequences, which can be extended to other continuous tasks.

Transsaccadic integration benefits are not limited to the saccade target

Across saccades, humans can integrate the low-resolution presaccadic information of an upcoming saccade target with the high-resolution postsaccadic information. There is converging evidence to suggest that transsaccadic integration occurs at the saccade target. However, given divergent evidence on the spatial specificity of related mechanisms such as attention, visual working memory, and remapping, it is unclear whether integration is also possible at locations other than the saccade target. We tested the spatial profile of transsaccadic integration, by testing perceptual performance at six locations around the saccade target and between the saccade target and initial fixation. Results show that integration benefits do not differ between the saccade target and surrounding locations. Transsaccadic integration benefits are not specific to the saccade target and can occur at other locations when they are behaviorally relevant, although there is a trend for worse performance for the location above initial fixation compared with those in the direction of the saccade. This suggests that transsaccadic integration may be a more general mechanism used to reconcile task-relevant pre- and postsaccadic information at attended locations other than the saccade target.

NEW & NOTEWORTHY This study shows that integration of pre- and postsaccadic information across saccades is not restricted to the saccade target. We found performance benefits of transsaccadic integration at attended locations other than the saccade target, and these benefits did not differ from those found at the saccade target. This suggests that transsaccadic integration may be a more general mechanism used to reconcile pre- and postsaccadic information at task-relevant locations.