In the eye of the beholder: a survey of models for eyes and gaze

A low-computational approach on gaze estimation with eye touch system

Among various approaches to eye tracking systems, light-reflection based systems with non-imaging sensors, e.g., photodiodes or phototransistors, are known to have relatively low complexity; yet, they provide moderately accurate estimation of the point of gaze. In this paper, a low-computational approach on gaze estimation is proposed using the Eye Touch system, which is a light-reflection based eye tracking system, previously introduced by the authors. Based on the physical implementation of Eye Touch, the sensor measurements are now utilized in low-computational least-squares algorithms to estimate arbitrary gaze directions, unlike the existing light reflection-based systems, including the initial Eye Touch implementation, where only limited predefined regions were distinguished. The system also utilizes an effective pattern classification algorithm to be able to perform left, right, and double clicks based on respective eye winks with significantly high accuracy. In order to avoid accuracy problems for sensitive sensor biasing hardware, a robust custom microcontroller-based data acquisition system is developed. Consequently, the physical size and cost of the overall Eye Touch system are considerably reduced while the power efficiency is improved. The results of the experimental analysis over numerous subjects clearly indicate that the proposed eye tracking system can classify eye winks with 98% accuracy, and attain an accurate gaze direction with an average angular error of about 0.93 °. Due to its lightweight structure, competitive accuracy and low-computational requirements relative to video-based eye tracking systems, the proposed system is a promising human-computer interface for both stationary and mobile eye tracking applications.

Calibration-free gaze tracking for automatic measurement of visual acuity in human infants

Most existing vision-based methods for gaze tracking need a tedious calibration process. In this process, subjects are required to fixate on a specific point or several specific points in space. However, it is hard to cooperate, especially for children and human infants. In this paper, a new calibration-free gaze tracking system and method is presented for automatic measurement of visual acuity in human infants. As far as I know, it is the first time to apply the vision-based gaze tracking in the measurement of visual acuity. Firstly, a polynomial of pupil center-cornea reflections (PCCR) vector is presented to be used as the gaze feature. Then, Gaussian mixture models (GMM) is employed for gaze behavior classification, which is trained offline using labeled data from subjects with healthy eyes. Experimental results on several subjects show that the proposed method is accurate, robust and sufficient for the application of measurement of visual acuity in human infants.

Art critic: Multisignal vision and speech interaction system in a gaming context

True immersion of a player within a game can only occur when the world simulated looks and behaves as close to reality as possible. This implies that the game must correctly read and understand, among other things, the player's focus, attitude toward the objects/persons in focus, gestures, and speech. In this paper, we proposed a novel system that integrates eye gaze estimation, head pose estimation, facial expression recognition, speech recognition, and text-to-speech components for use in real-time games. Both the eye gaze and head pose components utilize underlying 3-D models, and our novel head pose estimation algorithm uniquely combines scene flow with a generic head model. The facial expression recognition module uses the local binary patterns with three orthogonal planes approach on the 2-D shape index domain rather than the pixel domain, resulting in improved classification. Our system has also been extended to use a pan-tilt-zoom camera driven by the Kinect, allowing us to track a moving player. A test game, Art Critic, is also presented, which not only demonstrates the utility of our system but also provides a template for player/non-player character (NPC) interaction in a gaming context. The player alters his/her view of the 3-D world using head pose, looks at paintings/NPCs using eye gaze, and makes an evaluation based on the player's expression and speech. The NPC artist will respond with facial expression and synthetic speech based on its personality. Both qualitative and quantitative evaluations of the system are performed to illustrate the system's effectiveness.

GazeParser: an open-source and multiplatform library for low-cost eye tracking and analysis

Eye movement analysis is an effective method for research on visual perception and cognition. However, recordings of eye movements present practical difficulties related to the cost of the recording devices and the programming of device controls for use in experiments. GazeParser is an open-source library for low-cost eye tracking and data analysis; it consists of a video-based eyetracker and libraries for data recording and analysis. The libraries are written in Python and can be used in conjunction with PsychoPy and VisionEgg experimental control libraries. Three eye movement experiments are reported on performance tests of GazeParser. These showed that the means and standard deviations for errors in sampling intervals were less than 1 ms. Spatial accuracy ranged from 0.7° to 1.2°, depending on participant. In gap/overlap tasks and antisaccade tasks, the latency and amplitude of the saccades detected by GazeParser agreed with those detected by a commercial eyetracker. These results showed that the GazeParser demonstrates adequate performance for use in psychological experiments.

Dichoptic Viewing Methods for Binocular Rivalry Research: Prospects for Large-Scale Clinical and Genetic Studies

Binocular rivalry (BR) is an intriguing phenomenon that occurs when two different images are presented, one to each eye, resulting in alternation orrivalrybetween the percepts. The phenomenon has been studied for nearly 200 years, with renewed and intensive investigation over recent decades. Therateof perceptual switching has long been known to vary widely between individuals but to be relatively stable within individuals. A recent twin study demonstrated that individual variation in BR rate is under substantial genetic control, a finding that also represented the first report, using a large study, of genetic contribution for any post-retinal visual processing phenomenon. The twin study had been prompted by earlier work showing BR rate was slow in the heritable psychiatric condition, bipolar disorder (BD). Together, these studies suggested that slow BR may represent an endophenotype for BD, and heralded the advent of modern clinical and genetic studies of rivalry. This new focus has coincided with rapid advances in 3D display technology, but despite such progress, specific development of technology for rivalry research has been lacking. This review therefore compares different display methods for BR research across several factors, including viewing parameters, image quality, equipment cost, compatibility with other investigative methods, subject group, and sample size, with a focus on requirements specific to large-scale clinical and genetic studies. It is intended to be a resource for investigators new to BR research, such as clinicians and geneticists, and to stimulate the development of 3D display technology for advancing interdisciplinary studies of rivalry.

Eye Tracking and Head Movement Detection: A State-of-Art Survey

Eye-gaze detection and tracking have been an active research field in the past years as it adds convenience to a variety of applications. It is considered a significant untraditional method of human computer interaction. Head movement detection has also received researchers' attention and interest as it has been found to be a simple and effective interaction method. Both technologies are considered the easiest alternative interface methods. They serve a wide range of severely disabled people who are left with minimal motor abilities. For both eye tracking and head movement detection, several different approaches have been proposed and used to implement different algorithms for these technologies. Despite the amount of research done on both technologies, researchers are still trying to find robust methods to use effectively in various applications. This paper presents a state-of-art survey for eye tracking and head movement detection methods proposed in the literature. Examples of different fields of applications for both technologies, such as human-computer interaction, driving assistance systems, and assistive technologies are also investigated.

Eyeglasses lens contour extraction from facial images using an efficient shape description

This paper presents a system that automatically extracts the position of the eyeglasses and the accurate shape and size of the frame lenses in facial images. The novelty brought by this paper consists in three key contributions. The first one is an original model for representing the shape of the eyeglasses lens, using Fourier descriptors. The second one is a method for generating the search space starting from a finite, relatively small number of representative lens shapes based on Fourier morphing. Finally, we propose an accurate lens contour extraction algorithm using a multi-stage Monte Carlo sampling technique. Multiple experiments demonstrate the effectiveness of our approach.

Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades

Ternary eye movement classification, which separates fixations, saccades, and smooth pursuit from the raw eye positional data, is extremely challenging. This article develops new and modifies existing eye-tracking algorithms for the purpose of conducting meaningful ternary classification. To this end, a set of qualitative and quantitative behavior scores is introduced to facilitate the assessment of classification performance and to provide means for automated threshold selection. Experimental evaluation of the proposed methods is conducted using eye movement records obtained from 11 subjects at 1000 Hz in response to a step-ramp stimulus eliciting fixations, saccades, and smooth pursuits. Results indicate that a simple hybrid method that incorporates velocity and dispersion thresholding allows producing robust classification performance. It is concluded that behavior scores are able to aid automated threshold selection for the algorithms capable of successful classification.

Long-Range Gaze Tracking System for Large Movements

In the vision-based remote gaze tracking systems, the most challenging topics are to allow natural movement of a user and to increase the working volume and distance of the system. Several eye gaze estimation methods considering the natural movement of a user have been proposed. However, their working volume and distance are narrow and close. In this paper, we propose a novel 2-D mapping-based gaze estimation method that allows large-movement of user. Conventional 2-D mapping-based methods utilize mapping function between calibration points on the screen and pupil center corneal reflection (PCCR) vectors obtained in user calibration step. However, PCCR vectors and their associated mapping function are only valid at or near to the position where the user calibration is performed. The proposed movement mapping function, complementing the user's movement, estimates scale factors between two PCCR vector sets: one obtained at the user calibration position and another obtained at the new user position. The proposed system targets a longer range gaze tracking which operates from 1.4 to 3 m. A narrow-view camera mounted on a pan and tilt unit is used by the proposed system to capture high-resolution eye image, providing a wide and long working volume of about 100 cm × 40 cm × 100 cm. The experimental results show that the proposed method successfully compensated the poor performance due to user's large movement. Average angular error was 0.8° and only 0.07° of angular error was increased while the user moved around 81 cm.

Joint attention by gaze interpolation and saliency

Joint attention, which is the ability of coordination of a common point of reference with the communicating party, emerges as a key factor in various interaction scenarios. This paper presents an image-based method for establishing joint attention between an experimenter and a robot. The precise analysis of the experimenter's eye region requires stability and high-resolution image acquisition, which is not always available. We investigate regression-based interpolation of the gaze direction from the head pose of the experimenter, which is easier to track. Gaussian process regression and neural networks are contrasted to interpolate the gaze direction. Then, we combine gaze interpolation with image-based saliency to improve the target point estimates and test three different saliency schemes. We demonstrate the proposed method on a human-robot interaction scenario. Cross-subject evaluations, as well as experiments under adverse conditions (such as dimmed or artificial illumination or motion blur), show that our method generalizes well and achieves rapid gaze estimation for establishing joint attention.

Appearance-based gaze estimation using visual saliency

We propose a gaze sensing method using visual saliency maps that does not need explicit personal calibration. Our goal is to create a gaze estimator using only the eye images captured from a person watching a video clip. Our method treats the saliency maps of the video frames as the probability distributions of the gaze points. We aggregate the saliency maps based on the similarity in eye images to efficiently identify the gaze points from the saliency maps. We establish a mapping between the eye images to the gaze points by using Gaussian process regression. In addition, we use a feedback loop from the gaze estimator to refine the gaze probability maps to improve the accuracy of the gaze estimation. The experimental results show that the proposed method works well with different people and video clips and achieves a 3.5-degree accuracy, which is sufficient for estimating a user's attention on a display.

A free geometry model-independent neural eye-gaze tracking system

Background

Eye Gaze Tracking Systems (EGTSs) estimate the Point Of Gaze (POG) of a user. In diagnostic applications EGTSs are used to study oculomotor characteristics and abnormalities, whereas in interactive applications EGTSs are proposed as input devices for human computer interfaces (HCI), e.g. to move a cursor on the screen when mouse control is not possible, such as in the case of assistive devices for people suffering from locked-in syndrome. If the user’s head remains still and the cornea rotates around its fixed centre, the pupil follows the eye in the images captured from one or more cameras, whereas the outer corneal reflection generated by an IR light source, i.e. glint, can be assumed as a fixed reference point. According to the so-called pupil centre corneal reflection method (PCCR), the POG can be thus estimated from the pupil-glint vector.

Methods

A new model-independent EGTS based on the PCCR is proposed. The mapping function based on artificial neural networks allows to avoid any specific model assumption and approximation either for the user’s eye physiology or for the system initial setup admitting a free geometry positioning for the user and the system components. The robustness of the proposed EGTS is proven by assessing its accuracy when tested on real data coming from: i) different healthy users; ii) different geometric settings of the camera and the light sources; iii) different protocols based on the observation of points on a calibration grid and halfway points of a test grid.

Results

The achieved accuracy is approximately 0.49°, 0.41°, and 0.62° for respectively the horizontal, vertical and radial error of the POG.

Conclusions

The results prove the validity of the proposed approach as the proposed system performs better than EGTSs designed for HCI which, even if equipped with superior hardware, show accuracy values in the range 0.6°-1°.

Automatic Calibration Method for Driver's Head Orientation in Natural Driving Environment

Gaze tracking is crucial for studying driver's attention, detecting fatigue, and improving driver assistance systems, but it is difficult in natural driving environments due to nonuniform and highly variable illumination and large head movements. Traditional calibrations that require subjects to follow calibrators are very cumbersome to be implemented in daily driving situations. A new automatic calibration method, based on a single camera for determining the head orientation and which utilizes the side mirrors, the rear-view mirror, the instrument board, and different zones in the windshield as calibration points, is presented in this paper. Supported by a self-learning algorithm, the system tracks the head and categorizes the head pose in 12 gaze zones based on facial features. The particle filter is used to estimate the head pose to obtain an accurate gaze zone by updating the calibration parameters. Experimental results show that, after several hours of driving, the automatic calibration method without driver's corporation can achieve the same accuracy as a manual calibration method. The mean error of estimated eye gazes was less than 5°in day and night driving.

Multi-Initialized States Referred Work Parameter Calibration for Gaze Tracking Human-Robot Interaction

In order to adaptively calibrate the work parameters in the infrared-TV based eye gaze tracking Human-Robot Interaction (HRI) system, a kind of gaze direction sensing model has been provided for detecting the eye gaze identified parameters. We paid more attention to situations where the user's head was in a different position to the interaction interface. Furthermore, the algorithm for automatically correcting work parameters of the system has also been put up by defining certain initial reference system states and analysing the historical information of the interaction between a user and the system. Moreover, considering some application cases and factors, and relying on minimum error rate Bayesian decision-making theory, a mechanism for identifying system state and adaptively calibrating parameters has been proposed. Finally, some experiments have been done with the established system and the results suggest that the proposed mechanism and algorithm can identify the system work state in multi-situations, and can automatically correct the work parameters to meet the demands of a gaze tracking HRI system.

Gaze estimation interpolation methods based on binocular data

Video oculography (VOG) is one of the most commonly used techniques for gaze tracking because it enables nonintrusive eye detection and tracking. Improving the eye tracking's accuracy and tolerance to user head movements is a common task in the field of gaze tracking; thus, a thorough study of how binocular information can improve a gaze tracking system's accuracy and tolerance to user head movements has been carried out. The analysis is focused on interpolation-based methods and systems with one and two infrared lights. New mapping features are proposed based on the commonly used pupil-glint vector using different distances as the normalization factor. For this study, an experimental procedure with six users based on a real VOG gaze tracking system was performed, and the results were contrasted with an eye simulator. Important conclusions have been obtained in terms of configuration, equation, and mapping features, such as the outperformance of the interglint distance as the normalization factor. Furthermore, the binocular gaze tracking system was found to have a similar or improved level of accuracy compared to that of the monocular gaze tracking system.

A New Biased Discriminant Analysis Using Composite Vectors for Eye Detection

We propose a new biased discriminant analysis (BDA) using composite vectors for eye detection. A composite vector consists of several pixels inside a window on an image. The covariance of composite vectors is obtained from their inner product and can be considered as a generalization of the covariance of pixels. The proposed composite BDA (C-BDA) method is a BDA using the covariance of composite vectors. We construct a hybrid cascade detector for eye detection, using Haar-like features in the earlier stages and composite features obtained from C-BDA in the later stages. The proposed detector runs in real time; its execution time is 5.5 ms on a typical PC. The experimental results for the CMU PIE database and our own real-world data set show that the proposed detector provides robust performance to several kinds of variations such as facial pose, illumination, eyeglasses, and partial occlusion. On the whole, the detection rate per pair of eyes is 98.0% for the 3604 face images of the CMU PIE database and 95.1% for the 2331 face images of the real-world data set. In particular, it provides a 99.7% detection rate for the 2120 CMU PIE images without glasses. Face recognition performance is also investigated using the eye coordinates from the proposed detector. The recognition results for the real-world data set show that the proposed detector gives similar performance to the method using manually located eye coordinates, showing that the accuracy of the proposed eye detector is comparable with that of the ground-truth data.

Accuracy of a remote eye tracker for radiologic observer studies: effects of calibration and recording environment

RATIONALE AND OBJECTIVES

To determine the accuracy and reproducibility of a remote eye-tracking system for studies of observer gaze while displaying volumetric chest computed tomography (CT) images.

MATERIALS AND METHODS

Four participants performed calibrations using three different gray-scale backgrounds (black, gray, and white). Each participant then observed a three-dimensional 10-point test pattern embedded in five Digital Imaging and Communications in Medicine (DICOM) datasets (test backgrounds): a full 190-section chest CT scan, 190 copies of a single chest CT section, and three 190-section datasets of homogeneous intensity (black, gray, and white).

RESULTS

Significant variances between participants, calibration backgrounds, and test backgrounds were observed. The least mean systematic error (deviation of recorded gaze position from target) was obtained when the calibration background and test background were black (27 pixels). Systematic error increased when displaying a test background that deviated from the calibration background intensity. Hence, the largest mean systematic error occurred when calibrating to a black background and displaying a white background (67 pixels). For complex chest CT volumes the white calibration background performed best (38 pixels). An angular analysis of the systematic error was performed and demonstrated that the systemic error primarily affects the vertical position of the estimated gaze position.

CONCLUSION

Our findings indicate a potential source of systematic error during gaze recording in a dynamic environment and highlight the importance of configuring the calibration procedure according to the brightness of the display. We recommend that investigators develop routines for postcalibration accuracy measurement and report the effective accuracy for the display environment in which the data are collected.

A psychotechnological review on eye-tracking systems: towards user experience

PURPOSE

The aim of the present work is to show a critical review of the international literature on eye-tracking technologies by focusing on those features that characterize them as 'psychotechnologies'.

METHOD

A critical literature review was conducted through the main psychology, engineering, and computer sciences databases by following specific inclusion and exclusion criteria. A total of 46 matches from 1998 to 2010 were selected for content analysis. Results have been divided into four broad thematic areas.

RESULTS

We found that, although there is a growing attention to end-users, most of the studies reviewed in this work are far from being considered as adopting holistic human-computer interaction models that include both individual differences and needs of users. User is often considered only as a measurement object of the functioning of the technological system and not as a real alter-ego of the intrasystemic interaction.

CONCLUSION

In order to fully benefit from the communicative functions of gaze, the research on eye-tracking must emphasize user experience. Eye-tracking systems would become an effective assistive technology for integration, adaptation and neutralization of the environmental barrier only when a holistic model can be applied for both design processes and assessment of the functional components of the interaction.

Face salient points and eyes tracking for robust drowsiness detection

Measuring a driver's level of attention and drowsiness is fundamental to reducing the number of traffic accidents that often involve bus and truck drivers, who must work for long periods of time under monotonous road conditions. Determining a driver's state of alert in a noninvasive way can be achieved using computer vision techniques. However, two main difficulties must be solved in order to measure drowsiness in a robust way: first, detecting the driver's face location despite variations in pose or illumination; secondly, recognizing the driver's facial cues, such as blinks, yawns, and eyebrow rising. To overcome these challenges, our approach combines the well-known Viola–Jones face detector with the motion analysis of Shi–Tomasi salient features within the face. The location of the eyes and blinking is important to refine the tracking of the driver's head and compute the so-called PERCLOS, which is the percentage of time the eyes are closed over a given time interval. The latter cue is essential for noninvasive driver's alert state estimation as it has a high correlation with drowsiness. To further improve the location of the eyes under different conditions of illumination, the proposed method takes advantage of the high reflectivity of the retina to near infrared illumination employing a camera with an 850 nm wavelength filter. The paper shows that motion analysis of the salient points, in particular cluster mass centers and spatial distributions, yields better head tracking results compared to the state-of-the-art and provides measures of the driver's alert state.

Eye tracking using artificial neural networks for human computer interaction

This paper describes an ongoing project that has the aim to develop a low cost application to replace a computer mouse for people with physical impairment. The application is based on an eye tracking algorithm and assumes that the camera and the head position are fixed. Color tracking and template matching methods are used for pupil detection. Calibration is provided by neural networks as well as by parametric interpolation methods. Neural networks use back-propagation for learning and bipolar sigmoid function is chosen as the activation function. The user's eye is scanned with a simple web camera with backlight compensation which is attached to a head fixation device. Neural networks significantly outperform parametric interpolation techniques: 1) the calibration procedure is faster as they require less calibration marks and 2) cursor control is more precise. The system in its current stage of development is able to distinguish regions at least on the level of desktop icons. The main limitation of the proposed method is the lack of head-pose invariance and its relative sensitivity to illumination (especially to incidental pupil reflections).

Iris center corneal reflection method for gaze tracking using visible light

Very few attempts, if any, have been made to use visible light in corneal reflection approaches to the problem of gaze tracking. The reasons usually given to justify the limited application of this type of illumination are that the required image features are less accurately depicted, and that visible light may disturb the user. The aim of this paper is to show that it is possible to overcome these difficulties and build an accurate and robust gaze tracker under these circumstances. For this purpose, visible light is used to obtain the corneal reflection or glint in a way analogous to the well-known pupil center corneal reflection technique. Due to the lack of contrast, the center of the iris is tracked instead of the center of the pupil. The experiments performed in our laboratory have shown very satisfactory results, allowing free-head movement and no need of recalibration.