Domain Specificity of Oculomotor Learning after Changes in Sensory Processing

A systematic review of extended reality (XR) for understanding and augmenting vision loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

The Influence of Presentation Frames of Visualization Information for Safety on Situational Awareness under a Three-Level User-Interface Design

To explore the influence of the construction and presentation frames of visualization information for safety (VIS) on people's situation awareness (SA), we designed a three-level user interface (UI) of VIS based on the three-stage SA theory, including perception (SA1), comprehension (SA2), and projection (SA3). Then, 166 subjects were recruited and divided into three groups to participate in the experiment, in which SA was measured by the situation-present-assessment method (SPAM) and situation-awareness-rating technique (SART), and eye-movement data were recorded. The results show that the level-3 UI design could effectively improve the subjects' SA levels. Although the increase in VIS displayed caused by the higher UI level led to a decrease in the perception-stage score of SA, the level-3 UI fully considered the three stages of human information processing, and helped improve the SA of the subjects; the overall SA score measured using the SART method was not significant, but the result was consistent with the SPAM. There was a framing effect on the presentation of VIS, and subjects perceived different degrees of risk under different presentation frames; that is, less risk under the positive frame, more risk under the negative frame, and a higher level of SA under the positive frame compared with the negative frame. To some extent, the nearest-neighbor-index (NNI) algorithm could be utilized to quantify subjects' eye-tracking fixation mode. While the frames were guided by the high-level interface and the positive presentation frame, the distribution of the subjects' gaze points was more discrete; they could grasp the relevant information more comprehensively and had a relatively high level of SA. To some extent, this study can provide a reference for the design and optimization of the VIS presentation interface.

Humans trade off search costs and accuracy in a combined visual search and perceptual task

To interact with one's environment, relevant objects have to be selected as targets for saccadic eye movements. Previous studies have demonstrated that factors such as visual saliency and reward influence saccade target selection, and that humans can dynamically trade off these factors to maximize expected value during visual search. However, expected value in everyday situations not only depends on saliency and reward, but also on the required time to find objects, and the likelihood of a successful object-interaction after search. Here we studied whether search costs and the accuracy to discriminate an object feature can be traded off to maximize expected value. We designed a combined visual search and perceptual discrimination task, where participants chose whether to search for an easy- or difficult-to-discriminate target in search displays populated by distractors that shared features with either the easy or the difficult target. Participants received a monetary reward for correct discriminations and were given limited time to complete as many trials as they could. We found that participants considered their discrimination performance and the search costs when choosing targets and, by this, maximized expected value. However, the accumulated reward was constrained by noise in both the choice of which target to search for, and which elements to fixate during search. We conclude that humans take into account the prospective search time and the likelihood of successful a object-interaction, when deciding what to search for. However, search performance is constrained by noise in decisions about what to search for and how to search for it.

Peripheral facial features guiding eye movements and reducing fixational variability

Face processing is a fast and efficient process due to its evolutionary and social importance. A majority of people direct their first eye movement to a featureless point just below the eyes that maximizes accuracy in recognizing a person's identity and gender. Yet, the exact properties or features of the face that guide the first eye movements and reduce fixational variability are unknown. Here, we manipulated the presence of the facial features and the spatial configuration of features to investigate their effect on the location and variability of first and second fixations to peripherally presented faces. Our results showed that observers can utilize the face outline, individual facial features, and feature spatial configuration to guide the first eye movements to their preferred point of fixation. The eyes have a preferential role in guiding the first eye movements and reducing fixation variability. Eliminating the eyes or altering their position had the greatest influence on the location and variability of fixations and resulted in the largest detriment to face identification performance. The other internal features (nose and mouth) also contribute to reducing fixation variability. A subsequent experiment measuring detection of single features showed that the eyes have the highest detectability (relative to other features) in the visual periphery providing a strong sensory signal to guide the oculomotor system. Together, the results suggest a flexible multiple-cue approach that might be a robust solution to cope with how the varying eccentricities in the real world influence the ability to resolve individual feature properties and the preferential role of the eyes.

Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets' luminance but also crucially on high-level factors like the expected reward or a targets' relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.

Eye movement strategies in face ethnicity categorization vs. face identification tasks

A quick look at a face allows us to identify the person, their gender, and emotion. Humans direct their first eye movement towards points on the face that vary moderately across these common tasks and maximize performance. However, not known is the extent to which humans alter their oculomotor strategies to maximize accuracy in more specialized face categorization tasks. We studied the eye movements of Indian observers during a North vs. South Indian face categorization task and compared them to those in a person-identification task. We found that observers did not alter their first eye movement strategy for the ethnic categorization task, i.e., they directed their first fixations to a similar preferred point as in the person-identification task. To assess whether using a similar preferred point of fixation for both tasks resulted in a performance cost for the categorization task, we measured performance as a function of fixation position along the face. Fixating away from the preferred point of fixation reduced observer performance in the person identification task, but not in the ethnicity categorization task. We used computational modeling to assess whether the results could be explained by an interaction between the distribution of task information across the face and the foveated properties of the visual system. A foveated ideal observer analysis revealed a spatially more distributed task information and lower dependence of performance on the point of fixation for the ethnicity categorization task relative to the person identification. We conclude that, unlike the person identification task, humans can access the information for the ethnicity categorization task from various points of fixation. Thus, the observer strategy to utilize the typical person identification first eye movement for the ethnicity categorization task is a simple solution that incurs little or no performance cost.

Eye Movements in Macular Degeneration

In healthy vision, the fovea provides high acuity and serves as the locus for fixation achieved through saccadic eye movements. Bilateral loss of the foveal regions in both eyes causes individuals to adopt an eccentric locus for fixation. This review deals with the eye movement consequences of the loss of the foveal oculomotor reference and the ability of individuals to use an eccentric fixation locus as the new oculomotor reference. Eye movements are an integral part of everyday activities, such as reading, searching for an item of interest, eye-hand coordination, navigation, or tracking an approaching car. We consider how these tasks are impacted by the need to use an eccentric locus for fixation and as a reference for eye movements, specifically saccadic and smooth pursuit eye movements. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Initial eye gaze to faces and its functional consequence on face identification abilities in autism spectrum disorder

BACKGROUND

Autism spectrum disorder (ASD) is a neurodevelopmental disorder defined and diagnosed by core deficits in social communication and the presence of restricted and repetitive behaviors. Research on face processing suggests deficits in this domain in ASD but includes many mixed findings regarding the nature and extent of these differences. The first eye movement to a face has been shown to be highly informative and sufficient to achieve high performance in face identification in neurotypical adults. The current study focused on this critical moment shown to be essential in the process of face identification.

METHODS

We applied an established eye-tracking and face identification paradigm to comprehensively characterize the initial eye movement to a face and test its functional consequence on face identification performance in adolescents with and without ASD (n = 21 per group), and in neurotypical adults. Specifically, we presented a series of faces and measured the landing location of the first saccade to each face, while simultaneously measuring their face identification abilities. Then, individuals were guided to look at specific locations on the face, and we measured how face identification performance varied as a function of that location. Adolescent participants also completed a more traditional measure of face identification which allowed us to more fully characterize face identification abilities in ASD.

RESULTS

Our results indicate that the location of the initial look to faces and face identification performance for briefly presented faces are intact in ASD, ruling out the possibility that deficits in face perception, at least in adolescents with ASD, begin with the initial eye movement to the face. However, individuals with ASD showed impairments on the more traditional measure of face identification.

CONCLUSION

Together, the observed dissociation between initial, rapid face perception processes, and other measures of face perception offers new insights and hypotheses related to the timing and perceptual complexity of face processing and how these specific aspects of face identification may be disrupted in ASD.

Spatial Distribution of Eye-Movements After Central Vision Loss is Consistent with an Optimal Visual Search Strategy

The problem of gaze allocation has previously been studied in the framework of eye-movement control models, which require prior knowledge of visibility maps (VMs). These encode the signal-to-noise ratio, at each point in the visual field, which can be used to define an optimal policy of gaze allocation. However, it is not always possible to estimate the VM, in a given experimental setting, as it depends on many factors, including the visual system of the individual observer. Hence, few eye-movement datasets include the corresponding VM estimates. This can be problematic for the analysis of certain clinical conditions, such as Age-related Macular Degeneration (AMD), which are associated with reduced sensitivity in the affected locations of the visual field. The corresponding VMs are highly idiosyncratic, and cannot be modeled by estimates obtained from healthy observers. We propose an algorithm for maximum likelihood VM estimation, working directly from eye-movement sequences. We apply this algorithm to two eye-tracking datasets, based on visual search tasks, obtained from AMD patients. We show that the inferred VMs are spatially consistent with the measured visual field sensitivities. We also show that simulations with the estimated VMs can account for the asymmetric distribution of saccade vectors, which is typical of AMD patients.

Eye movements and retinotopic tuning in developmental prosopagnosia

Despite extensive investigation, the causes and nature of developmental prosopagnosia (DP)-a severe face identification impairment in the absence of acquired brain injury-remain poorly understood. Drawing on previous work showing that individuals identified as being neurotypical (NT) show robust individual differences in where they fixate on faces, and recognize faces best when the faces are presented at this location, we defined and tested four novel hypotheses for how atypical face-looking behavior and/or retinotopic face encoding could impair face recognition in DP: (a) fixating regions of poor information, (b) inconsistent saccadic targeting, (c) weak retinotopic tuning, and (d) fixating locations not matched to the individual's own face tuning. We found no support for the first three hypotheses, with NTs and DPs consistently fixating similar locations and showing similar retinotopic tuning of their face perception performance. However, in testing the fourth hypothesis, we found preliminary evidence for two distinct phenotypes of DP: (a) Subjects characterized by impaired face memory, typical face perception, and a preference to look high on the face, and (b) Subjects characterized by profound impairments to both face memory and perception and a preference to look very low on the face. Further, while all NTs and upper-looking DPs performed best when faces were presented near their preferred fixation location, this was not true for lower-looking DPs. These results suggest that face recognition deficits in a substantial proportion of people with DP may arise not from aberrant face gaze or compromised retinotopic tuning, but from the suboptimal matching of gaze to tuning.

Task-driven visual exploration at the foveal scale

Humans use saccades to inspect objects of interest with the foveola, the small region of the retina with highest acuity. This process of visual exploration is normally studied over large scenes. However, in everyday tasks, the stimulus within the foveola is complex, and the need for visual exploration may extend to this smaller scale. We have previously shown that fixational eye movements, in particular microsaccades, play an important role in fine spatial vision. Here, we investigate whether task-driven visual exploration occurs during the fixation pauses in between large saccades. Observers judged the expression of faces covering approximately 1°, as if viewed from a distance of many meters. We use a custom system for accurately localizing the line of sight and continually track gaze position at high resolution. Our findings reveal that active spatial exploration, a process driven by the goals of the task, takes place at the foveal scale. The scanning strategies used at this scale resemble those used when examining larger scenes, with idiosyncrasies maintained across spatial scales. These findings suggest that the visual system possesses not only a coarser priority map of the extrafoveal space to guide saccades, but also a finer-grained priority map that is used to guide microsaccades once the region of interest is foveated.