In the eye of the beholder: a survey of models for eyes and gaze

Appearance-Based Gaze Estimation via Uncalibrated Gaze Pattern Recovery

Aiming at reducing the restrictions due to person/scene dependence, we deliver a novel method that solves appearance-based gaze estimation in a novel fashion. First, we introduce and solve an "uncalibrated gaze pattern" solely from eye images independent of the person and scene. The gaze pattern recovers gaze movements up to only scaling and translation ambiguities, via nonlinear dimension reduction and pixel motion analysis, while no training/calibration is needed. This is new in the literature and enables novel applications. Second, our method allows simple calibrations to align the gaze pattern to any gaze target. This is much simpler than conventional calibrations which rely on sufficient training data to compute person and scene-specific nonlinear gaze mappings. Through various evaluations, we show that: 1) the proposed uncalibrated gaze pattern has novel and broad capabilities; 2) the proposed calibration is simple and efficient, and can be even omitted in some scenarios; and 3) quantitative evaluations produce promising results under various conditions.

3-D-Gaze-Based Robotic Grasping Through Mimicking Human Visuomotor Function for People With Motion Impairments

OBJECTIVE

The goal of this paper is to achieve a novel 3-D-gaze-based human-robot-interaction modality, with which a user with motion impairment can intuitively express what tasks he/she wants the robot to do by directly looking at the object of interest in the real world. Toward this goal, we investigate 1) the technology to accurately sense where a person is looking in real environments and 2) the method to interpret the human gaze and convert it into an effective interaction modality. Looking at a specific object reflects what a person is thinking related to that object, and the gaze location contains essential information for object manipulation.

METHODS

A novel gaze vector method is developed to accurately estimate the 3-D coordinates of the object being looked at in real environments, and a novel interpretation framework that mimics human visuomotor functions is designed to increase the control capability of gaze in object grasping tasks.

RESULTS

High tracking accuracy was achieved using the gaze vector method. Participants successfully controlled a robotic arm for object grasping by directly looking at the target object.

CONCLUSION

Human 3-D gaze can be effectively employed as an intuitive interaction modality for robotic object manipulation.

SIGNIFICANCE

It is the first time that 3-D gaze is utilized in a real environment to command a robot for a practical application. Three-dimensional gaze tracking is promising as an intuitive alternative for human-robot interaction especially for disabled and elderly people who cannot handle the conventional interaction modalities.

A crucial temporal accuracy test of combining EEG and Tobii eye tracker

Eye tracking and event-related potentials have been widely used in the field of cognitive psychology and neuroscience. Both techniques have the ability to refine cognitive models through a precise timeline description; nevertheless, they also have severe limitations. Combining measures of event-related potentials and eye movements can contribute to cognitive process capture, which provides the possibility to determine precisely when and in which order different cognitive operations occur. Combining of event-related potentials and eye movements has been recently conducted by synchronizing measures from an infrared eye tracker with an electroencephalograph to allow simultaneous data recording. Here, we describe in detail 4 types of co-registration methods for event-related potentials and eye movements on the Tobii platform. Moreover, the present investigation was designed to evaluate the temporal accuracy of data obtained using the 4 methods. We found that the method based on the Tobii Pro Analytics software development kit had a higher degree of temporal accuracy than the other co-registration methods. Furthermore, the reasons for the different temporal accuracies were assessed, and potential measures to correct clock drift were taken. General suggestions are made regarding timing in the co-registration of the electroencephalograph and eye tracker.

Novel quantitative pigmentation phenotyping enhances genetic association, epistasis, and prediction of human eye colour

Success of genetic association and the prediction of phenotypic traits from DNA are known to depend on the accuracy of phenotype characterization, amongst other parameters. To overcome limitations in the characterization of human iris pigmentation, we introduce a fully automated approach that specifies the areal proportions proposed to represent differing pigmentation types, such as pheomelanin, eumelanin, and non-pigmented areas within the iris. We demonstrate the utility of this approach using high-resolution digital eye imagery and genotype data from 12 selected SNPs from over 3000 European samples of seven populations that are part of the EUREYE study. In comparison to previous quantification approaches, (1) we achieved an overall improvement in eye colour phenotyping, which provides a better separation of manually defined eye colour categories. (2) Single nucleotide polymorphisms (SNPs) known to be involved in human eye colour variation showed stronger associations with our approach. (3) We found new and confirmed previously noted SNP-SNP interactions. (4) We increased SNP-based prediction accuracy of quantitative eye colour. Our findings exemplify that precise quantification using the perceived biological basis of pigmentation leads to enhanced genetic association and prediction of eye colour. We expect our approach to deliver new pigmentation genes when applied to genome-wide association testing.

Optics of the human cornea influence the accuracy of stereo eye-tracking methods: a simulation study

Current stereo eye-tracking methods model the cornea as a sphere with one refractive surface. However, the human cornea is slightly aspheric and has two refractive surfaces. Here we used ray-tracing and the Navarro eye-model to study how these optical properties affect the accuracy of different stereo eye-tracking methods. We found that pupil size, gaze direction and head position all influence the reconstruction of gaze. Resulting errors range between ± 1.0 degrees at best. This shows that stereo eye-tracking may be an option if reliable calibration is not possible, but the applied eye-model should account for the actual optics of the cornea.

A New Human Eye Tracking Algorithm of Optimized TLD Based on Improved Mean-Shift

In this paper, an improved Mean-shift algorithm was integrated with standard tracking–learning–detection (TLD) model tracker for improving the tracking effects of standard TLD model and enhancing the anti-occlusion capability and the recognition capability of similar objectives. The target region obtained by the improved Mean-shift algorithm and the target region obtained by the TLD model tracker are integrated to achieve favorable tracking effects. Then the optimized TLD tracking system was applied to human eye tracking. In the tests, the model can be self-adopted to partial occlusion, such as eye-glasses, closed eyes and hand occlusion. And the roll angle can approach 90[Formula: see text], raw angle can approach 45[Formula: see text] and pitch angle can approach 60[Formula: see text]. In addition, the model never mistakenly transfers the tracking region to another eye (similar target on the same face) in longtime tracking. Experimental results indicate that: (1) the optimized TLD model shows sound tracking stability even when targets are partially occluded or rotated; (2) tracking speed and accuracy are superior to those of the standard TLD and some mainstream tracking methods. In summary, the optimized TLD model show higher robustness, stability and better responding to complex eye tracking requirement.

Driver Distraction Using Visual-Based Sensors and Algorithms

Driver distraction, defined as the diversion of attention away from activities critical for safe driving toward a competing activity, is increasingly recognized as a significant source of injuries and fatalities on the roadway. Additionally, the trend towards increasing the use of in-vehicle information systems is critical because they induce visual, biomechanical and cognitive distraction and may affect driving performance in qualitatively different ways. Non-intrusive methods are strongly preferred for monitoring distraction, and vision-based systems have appeared to be attractive for both drivers and researchers. Biomechanical, visual and cognitive distractions are the most commonly detected types in video-based algorithms. Many distraction detection systems only use a single visual cue and therefore, they may be easily disturbed when occlusion or illumination changes appear. Moreover, the combination of these visual cues is a key and challenging aspect in the development of robust distraction detection systems. These visual cues can be extracted mainly by using face monitoring systems but they should be completed with more visual cues (e.g., hands or body information) or even, distraction detection from specific actions (e.g., phone usage). Additionally, these algorithms should be included in an embedded device or system inside a car. This is not a trivial task and several requirements must be taken into account: reliability, real-time performance, low cost, small size, low power consumption, flexibility and short time-to-market. The key points for the development and implementation of sensors to carry out the detection of distraction will also be reviewed. This paper shows a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction. Some key points considered as both future work and challenges ahead yet to be solved will also be addressed.

Masking Strategies for Image Manifolds

We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold's geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the power expense grows with the number of pixels acquired. Our goal is for the manifold learned from masked images to resemble its full image counterpart as closely as possible. More precisely, we show that one can indeed accurately learn an image manifold without having to consider a large majority of the image pixels. In doing so, we consider two masking methods that preserve the local and global geometric structure of the manifold, respectively. In each case, the process of finding the optimal masking pattern can be cast as a binary integer program, which is computationally expensive but can be approximated by a fast greedy algorithm. Numerical experiments show that the relevant manifold structure is preserved through the data-dependent masking process, even for modest mask sizes.

Real-Time Detection and Measurement of Eye Features from Color Images

The accurate extraction and measurement of eye features is crucial to a variety of domains, including human-computer interaction, biometry, and medical research. This paper presents a fast and accurate method for extracting multiple features around the eyes: the center of the pupil, the iris radius, and the external shape of the eye. These features are extracted using a multistage algorithm. On the first stage the pupil center is localized using a fast circular symmetry detector and the iris radius is computed using radial gradient projections, and on the second stage the external shape of the eye (of the eyelids) is determined through a Monte Carlo sampling framework based on both color and shape information. Extensive experiments performed on a different dataset demonstrate the effectiveness of our approach. In addition, this work provides eye annotation data for a publicly-available database.

Size-Invariant Detection of Cell Nuclei in Microscopy Images

Accurate detection of individual cell nuclei in microscopy images is an essential and fundamental task for many biological studies. In particular, multivariate fluorescence microscopy is used to observe different aspects of cells in cultures. Manual detection of individual cell nuclei by visual inspection is time consuming, and prone to induce subjective bias. This makes automatic detection of cell nuclei essential for large-scale, objective studies of cell cultures. Blur, clutter, bleed-through, imaging noise and touching and partially overlapping nuclei with varying sizes and shapes make automated detection of individual cell nuclei a challenging task using image analysis. In this paper we propose a new automated method for fast and robust detection of individual cell nuclei based on their radial symmetric nature in fluorescence in-situ hybridization (FISH) images obtained via confocal microscopy. The main contributions are two-fold. 1) This work presents a more accurate cell nucleus detection system using the fast radial symmetry transform (FRST). 2) The proposed cell nucleus detection system is robust against most occlusions and variations in size and moderate shape deformations. We evaluate the performance of the proposed algorithm using precision/recall rates, Fβ-score and root-mean-squared distance (RMSD) and show that our algorithm provides improved detection accuracy compared to existing algorithms.

A simple algorithm for the offline recalibration of eye-tracking data through best-fitting linear transformation

Poor calibration and inaccurate drift correction can pose severe problems for eye-tracking experiments requiring high levels of accuracy and precision. We describe an algorithm for the offline correction of eye-tracking data. The algorithm conducts a linear transformation of the coordinates of fixations that minimizes the distance between each fixation and its closest stimulus. A simple implementation in MATLAB is also presented. We explore the performance of the correction algorithm under several conditions using simulated and real data, and show that it is particularly likely to improve data quality when many fixations are included in the fitting process.

Low Cost Eye Tracking: The Current Panorama

Despite the availability of accurate, commercial gaze tracker devices working with infrared (IR) technology, visible light gaze tracking constitutes an interesting alternative by allowing scalability and removing hardware requirements. Over the last years, this field has seen examples of research showing performance comparable to the IR alternatives. In this work, we survey the previous work on remote, visible light gaze trackers and analyze the explored techniques from various perspectives such as calibration strategies, head pose invariance, and gaze estimation techniques. We also provide information on related aspects of research such as public datasets to test against, open source projects to build upon, and gaze tracking services to directly use in applications. With all this information, we aim to provide the contemporary and future researchers with a map detailing previously explored ideas and the required tools.

Pupil size influences the eye-tracker signal during saccades

While it is known that scleral search coils-measuring the rotation of the eye globe--and modern, video based eye trackers-tracking the center of the pupil and the corneal reflection (CR)--produce signals with different properties, the mechanisms behind the differences are less investigated. We measure how the size of the pupil affects the eye-tracker signal recorded during saccades with a common pupil-CR eye-tracker. Eye movements were collected from four healthy participants and one person with an aphakic eye while performing self-paced, horizontal saccades at different levels of screen luminance and hence pupil size. Results show that pupil-, and gaze-signals, but not the CR-signal, are affected by the size of the pupil; changes in saccade peak velocities in the gaze signal of more than 30% were found. It is important to be aware of this pupil size dependent change when comparing fine grained oculomotor behavior across participants and conditions.

A practical efficient human computer interface based on saccadic eye movements for people with disabilities

Human computer interfaces (HCI) provide new channels of communication for people with severe motor disabilities to state their needs, and control their environment. Some HCI systems are based on eye movements detected from the electrooculogram. In this study, a wearable HCI, which implements a novel adaptive algorithm for detection of saccadic eye movements in eight directions, was developed, considering the limitations that people with disabilities have. The adaptive algorithm eliminated the need for calibration of the system for different users and in different environments. A two-stage typing environment and a simple game for training people with disabilities to work with the system were also developed. Performance of the system was evaluated in experiments with the typing environment performed by six participants without disabilities. The average accuracy of the system in detecting eye movements and blinking was 82.9% at first tries with an average typing rate of 4.5cpm. However an experienced user could achieve 96% accuracy and 7.2cpm typing rate. Moreover, the functionality of the system for people with movement disabilities was evaluated by performing experiments with the game environment. Six people with tetraplegia and significant levels of speech impairment played with the computer game several times. The average success rate in performing the necessary eye movements was 61.5%, which increased significantly with practice up to 83% for one participant. The developed system is 2.6×4.5cm in size and weighs only 15g, assuring high level of comfort for the users.

Mobile gaze tracking system for outdoor walking behavioral studies

Most gaze tracking techniques estimate gaze points on screens, on scene images, or in confined spaces. Tracking of gaze in open-world coordinates, especially in walking situations, has rarely been addressed. We use a head-mounted eye tracker combined with two inertial measurement units (IMU) to track gaze orientation relative to the heading direction in outdoor walking. Head movements relative to the body are measured by the difference in output between the IMUs on the head and body trunk. The use of the IMU pair reduces the impact of environmental interference on each sensor. The system was tested in busy urban areas and allowed drift compensation for long (up to 18 min) gaze recording. Comparison with ground truth revealed an average error of 3.3° while walking straight segments. The range of gaze scanning in walking is frequently larger than the estimation error by about one order of magnitude. Our proposed method was also tested with real cases of natural walking and it was found to be suitable for the evaluation of gaze behaviors in outdoor environments.

Gaze Estimation From Eye Appearance: A Head Pose-Free Method via Eye Image Synthesis

In this paper, we address the problem of free head motion in appearance-based gaze estimation. This problem remains challenging because head motion changes eye appearance significantly, and thus, training images captured for an original head pose cannot handle test images captured for other head poses. To overcome this difficulty, we propose a novel gaze estimation method that handles free head motion via eye image synthesis based on a single camera. Compared with conventional fixed head pose methods with original training images, our method only captures four additional eye images under four reference head poses, and then, precisely synthesizes new training images for other unseen head poses in estimation. To this end, we propose a single-directional (SD) flow model to efficiently handle eye image variations due to head motion. We show how to estimate SD flows for reference head poses first, and then use them to produce new SD flows for training image synthesis. Finally, with synthetic training images, joint optimization is applied that simultaneously solves an eye image alignment and a gaze estimation. Evaluation of the method was conducted through experiments to assess its performance and demonstrate its effectiveness.

Robust Eye Center Localization through Face Alignment and Invariant Isocentric Patterns

The localization of eye centers is a very useful cue for numerous applications like face recognition, facial expression recognition, and the early screening of neurological pathologies. Several methods relying on available light for accurate eye-center localization have been exploited. However, despite the considerable improvements that eye-center localization systems have undergone in recent years, only few of these developments deal with the challenges posed by the profile (non-frontal face). In this paper, we first use the explicit shape regression method to obtain the rough location of the eye centers. Because this method extracts global information from the human face, it is robust against any changes in the eye region. We exploit this robustness and utilize it as a constraint. To locate the eye centers accurately, we employ isophote curvature features, the accuracy of which has been demonstrated in a previous study. By applying these features, we obtain a series of eye-center locations which are candidates for the actual position of the eye-center. Among these locations, the estimated locations which minimize the reconstruction error between the two methods mentioned above are taken as the closest approximation for the eye centers locations. Therefore, we combine explicit shape regression and isophote curvature feature analysis to achieve robustness and accuracy, respectively. In practical experiments, we use BioID and FERET datasets to test our approach to obtaining an accurate eye-center location while retaining robustness against changes in scale and pose. In addition, we apply our method to non-frontal faces to test its robustness and accuracy, which are essential in gaze estimation but have seldom been mentioned in previous works. Through extensive experimentation, we show that the proposed method can achieve a significant improvement in accuracy and robustness over state-of-the-art techniques, with our method ranking second in terms of accuracy. According to our implementation on a PC with a Xeon 2.5Ghz CPU, the frame rate of the eye tracking process can achieve 38 Hz.

CIDER: Enabling Robustness-Power Tradeoffs on a Computational Eyeglass

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the complexity of continuous high-rate sensing from a camera and processing the image stream to estimate eye parameters, and b) dealing with the wide variability in illumination conditions in the natural environment. This paper explores the power-robustness tradeoffs inherent in the design of a wearable eye tracker, and proposes a novel staged architecture that enables graceful adaptation across the spectrum of real-world illumination. We propose CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared, b) error in estimating pupil center and pupil dilation, and c) model training procedures that involve zero effort from a user. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

Attention-Aware Robotic Laparoscope Based on Fuzzy Interpretation of Eye-Gaze Patterns

Laparoscopic robots have been widely adopted in modern medical practice. However, explicitly interacting with these robots may increase the physical and cognitive load on the surgeon. An attention-aware robotic laparoscope system has been developed to free the surgeon from the technical limitations of visualization through the laparoscope. This system can implicitly recognize the surgeon's visual attention by interpreting the surgeon's natural eye movements using fuzzy logic and then automatically steer the laparoscope to focus on that viewing target. Experimental results show that this system can make the surgeon–robot interaction more effective, intuitive, and has the potential to make the execution of the surgery smoother and faster.

An investigation of pupil-based cognitive load measurement with low cost infrared webcam under light reflex interference

Using the task-evoked pupillary response (TEPR) to index cognitive load can contribute significantly to the assessment of memory function and cognitive skills in patients. However, the measurement of pupillary response is currently limited to a well-controlled lab environment due to light reflex and also relies heavily on expensive video-based eye trackers. Furthermore, commercial eye trackers are usually dedicated to gaze direction measurement, and their calibration procedure and computing resource are largely redundant for pupil-based cognitive load measurement (PCLM). In this study, we investigate the validity of cognitive load measurement with (i) pupil light reflex in a less controlled luminance background; (ii) a low-cost infrared (IR) webcam for the TEPR in a controlled luminance background. ANOVA results show that with an appropriate baseline selection and subtraction, the light reflex is significantly reduced, suggesting the possibility of less constrained practical applications of PCLM. Compared with the TEPR from a commercial remote eye tracker, a low-cost IR webcam achieved a similar TEPR pattern and no significant difference was found between the two devices in terms of cognitive load measurement across five induced load levels.

A new perspective on how humans assess their surroundings; derivation of head orientation and its role in 'framing' the environment

Understanding the way humans inform themselves about their environment is pivotal in helping explain our susceptibility to stimuli and how this modulates behaviour and movement patterns. We present a new device, the Human Interfaced Personal Observation Platform (HIPOP), which is a head-mounted (typically on a hat) unit that logs magnetometry and accelerometry data at high rates and, following appropriate calibration, can be used to determine the heading and pitch of the wearer’s head. We used this device on participants visiting a botanical garden and noted that although head pitch ranged between −80° and 60°, 25% confidence limits were restricted to an arc of about 25° with a tendency for the head to be pitched down (mean head pitch ranged between −43° and 0°). Mean rates of change of head pitch varied between −0.00187°/0.1 s and 0.00187°/0.1 s, markedly slower than rates of change of head heading which varied between −0.3141°/0.1 s and 0.01263°/0.1 s although frequency distributions of both parameters showed them to be symmetrical and monomodal. Overall, there was considerable variation in both head pitch and head heading, which highlighted the role that head orientation might play in exposing people to certain features of the environment. Thus, when used in tandem with accurate position-determining systems, the HIPOP can be used to determine how the head is orientated relative to gravity and geographic North and in relation to geographic position, presenting data on how the environment is being ‘framed’ by people in relation to environmental content.

SET: a pupil detection method using sinusoidal approximation

Mobile eye-tracking in external environments remains challenging, despite recent advances in eye-tracking software and hardware engineering. Many current methods fail to deal with the vast range of outdoor lighting conditions and the speed at which these can change. This confines experiments to artificial environments where conditions must be tightly controlled. Additionally, the emergence of low-cost eye tracking devices calls for the development of analysis tools that enable non-technical researchers to process the output of their images. We have developed a fast and accurate method (known as "SET") that is suitable even for natural environments with uncontrolled, dynamic and even extreme lighting conditions. We compared the performance of SET with that of two open-source alternatives by processing two collections of eye images: images of natural outdoor scenes with extreme lighting variations ("Natural"); and images of less challenging indoor scenes ("CASIA-Iris-Thousand"). We show that SET excelled in outdoor conditions and was faster, without significant loss of accuracy, indoors. SET offers a low cost eye-tracking solution, delivering high performance even in challenging outdoor environments. It is offered through an open-source MATLAB toolkit as well as a dynamic-link library ("DLL"), which can be imported into many programming languages including C# and Visual Basic in Windows OS (www.eyegoeyetracker.co.uk).

Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays

In recent years optical see-through head-mounted displays (OST-HMDs) have moved from conceptual research to a market of mass-produced devices with new models and applications being released continuously. It remains challenging to deploy augmented reality (AR) applications that require consistent spatial visualization. Examples include maintenance, training and medical tasks, as the view of the attached scene camera is shifted from the user's view. A calibration step can compute the relationship between the HMD-screen and the user's eye to align the digital content. However, this alignment is only viable as long as the display does not move, an assumption that rarely holds for an extended period of time. As a consequence, continuous recalibration is necessary. Manual calibration methods are tedious and rarely support practical applications. Existing automated methods do not account for user-specific parameters and are error prone. We propose the combination of a pre-calibrated display with a per-frame estimation of the user's cornea position to estimate the individual eye center and continuously recalibrate the system. With this, we also obtain the gaze direction, which allows for instantaneous uncalibrated eye gaze tracking, without the need for additional hardware and complex illumination. Contrary to existing methods, we use simple image processing and do not rely on iris tracking, which is typically noisy and can be ambiguous. Evaluation with simulated and real data shows that our approach achieves a more accurate and stable eye pose estimation, which results in an improved and practical calibration with a largely improved distribution of projection error.

An Optokinetic Nystagmus Detection Method for Use With Young Children

The detection of vision problems in early childhood can prevent neurodevelopmental disorders such as amblyopia. However, accurate clinical assessment of visual function in young children is challenging. optokinetic nystagmus (OKN) is a reflexive sawtooth motion of the eye that occurs in response to drifting stimuli, that may allow for objective measurement of visual function in young children if appropriate child-friendly eye tracking techniques are available. In this paper, we present offline tools to detect the presence and direction of the optokinetic reflex in children using consumer grade video equipment. Our methods are tested on video footage of children ([Formula: see text] children and 20 trials) taken as they freely observed visual stimuli that induced horizontal OKN. Using results from an experienced observer as a baseline, we found the sensitivity and specificity of our OKN detection method to be 89.13% and 98.54%, respectively, across all trials. Our OKN detection results also compared well (85%) with results obtained from a clinically trained assessor. In conclusion, our results suggest that OKN presence and direction can be measured objectively in children using consumer grade equipment, and readily implementable algorithms.

Acquiring visual information for locomotion by older adults: a systematic review

Developments in technology have facilitated quantitative examination of gaze behavior in relation to locomotion. The objective of this systematic review is to provide a critical evaluation of available evidence and to explore the role of gaze behavior among older adults during different forms of locomotion. Database searches were conducted to identify research papers that met the inclusion criteria of (1) study variables that included direct measurement of gaze and at least one form of locomotion, (2) participants who were older adults aged 60 years and above, and (3) reporting original research. Twenty-five papers related to walking on a straight path and turning (n=4), stair navigation (n=3), target negotiation and obstacle circumvention (n=13) and perturbation-evoked sudden loss of balance (n=5) were identified for the final quality assessment. The reviewed articles were found to have acceptable quality, with scores ranging from 47.06% to 94.12%. Overall, the current literature suggests that differences in gaze behavior during locomotion appear to change in late adulthood, especially with respect to transfer of gaze to and from a target, saccade-step latency, fixation durations on targets and viewing patterns. These changes appear to be particularly pronounced for older adults with high risk of falling and impaired executive functioning.

A probabilistic approach to online eye gaze tracking without explicit personal calibration

Existing eye gaze tracking systems typically require an explicit personal calibration process in order to estimate certain person-specific eye parameters. For natural human computer interaction, such a personal calibration is often inconvenient and unnatural. In this paper, we propose a new probabilistic eye gaze tracking system without explicit personal calibration. Unlike the conventional eye gaze tracking methods, which estimate the eye parameter deterministically using known gaze points, our approach estimates the probability distributions of the eye parameter and eye gaze. Using an incremental learning framework, the subject does not need personal calibration before using the system. His/her eye parameter estimation and gaze estimation can be improved gradually when he/she is naturally interacting with the system. The experimental result shows that the proposed system can achieve <3° accuracy for different people without explicit personal calibration.

Why have microsaccades become larger? Investigating eye deformations and detection algorithms

The reported size of microsaccades is considerably larger today compared to the initial era of microsaccade studies during the 1950s and 1960s. We investigate whether this increase in size is related to the fact that the eye-trackers of today measure different ocular structures than the older techniques, and that the movements of these structures may differ during a microsaccade. In addition, we explore the impact such differences have on subsequent analyzes of the eye-tracker signals. In Experiment I, the movement of the pupil as well as the first and fourth Purkinje reflections were extracted from series of eye images recorded during a fixation task. Results show that the different ocular structures produce different microsaccade signatures. In Experiment II, we found that microsaccade amplitudes computed with a common detection algorithm were larger compared to those reported by two human experts. The main reason was that the overshoots were not systematically detected by the algorithm and therefore not accurately accounted for. We conclude that one reason to why the reported size of microsaccades has increased is due to the larger overshoots produced by the modern pupil-based eye-trackers compared to the systems used in the classical studies, in combination with the lack of a systematic algorithmic treatment of the overshoot. We hope that awareness of these discrepancies in microsaccade dynamics across eye structures will lead to more generally accepted definitions of microsaccades.

Efficient and robust pupil size and blink estimation from near-field video sequences for human-machine interaction

Monitoring pupil and blink dynamics has applications in cognitive load measurement during human-machine interaction. However, accurate, efficient, and robust pupil size and blink estimation pose significant challenges to the efficacy of real-time applications due to the variability of eye images, hence to date, require manual intervention for fine tuning of parameters. In this paper, a novel self-tuning threshold method, which is applicable to any infrared-illuminated eye images without a tuning parameter, is proposed for segmenting the pupil from the background images recorded by a low cost webcam placed near the eye. A convex hull and a dual-ellipse fitting method are also proposed to select pupil boundary points and to detect the eyelid occlusion state. Experimental results on a realistic video dataset show that the measurement accuracy using the proposed methods is higher than that of widely used manually tuned parameter methods or fixed parameter methods. Importantly, it demonstrates convenience and robustness for an accurate and fast estimate of eye activity in the presence of variations due to different users, task types, load, and environments. Cognitive load measurement in human-machine interaction can benefit from this computationally efficient implementation without requiring a threshold calibration beforehand. Thus, one can envisage a mini IR camera embedded in a lightweight glasses frame, like Google Glass, for convenient applications of real-time adaptive aiding and task management in the future.

Adaptive Linear Regression for Appearance-Based Gaze Estimation

We investigate the appearance-based gaze estimation problem, with respect to its essential difficulty in reducing the number of required training samples, and other practical issues such as slight head motion, image resolution variation, and eye blinking. We cast the problem as mapping high-dimensional eye image features to low-dimensional gaze positions, and propose an adaptive linear regression (ALR) method as the key to our solution. The ALR method adaptively selects an optimal set of sparsest training samples for the gaze estimation via ℓ(1)-optimization. In this sense, the number of required training samples is significantly reduced for high accuracy estimation. In addition, by adopting the basic ALR objective function, we integrate the gaze estimation, subpixel alignment and blink detection into a unified optimization framework. By solving these problems simultaneously, we successfully handle slight head motion, image resolution variation and eye blinking in appearance-based gaze estimation. We evaluated the proposed method by conducting experiments with multiple users and variant conditions to verify its effectiveness.

Unsupervised eye pupil localization through differential geometry and local self-similarity matching

The automatic detection and tracking of human eyes and, in particular, the precise localization of their centers (pupils), is a widely debated topic in the international scientific community. In fact, the extracted information can be effectively used in a large number of applications ranging from advanced interfaces to biometrics and including also the estimation of the gaze direction, the control of human attention and the early screening of neurological pathologies. Independently of the application domain, the detection and tracking of the eye centers are, currently, performed mainly using invasive devices. Cheaper and more versatile systems have been only recently introduced: they make use of image processing techniques working on periocular patches which can be specifically acquired or preliminarily cropped from facial images. In the latter cases the involved algorithms must work even in cases of non-ideal acquiring conditions (e.g in presence of noise, low spatial resolution, non-uniform lighting conditions, etc.) and without user's awareness (thus with possible variations of the eye in scale, rotation and/or translation). Getting satisfying results in pupils' localization in such a challenging operating conditions is still an open scientific topic in Computer Vision. Actually, the most performing solutions in the literature are, unfortunately, based on supervised machine learning algorithms which require initial sessions to set the working parameters and to train the embedded learning models of the eye: this way, experienced operators have to work on the system each time it is moved from an operational context to another. It follows that the use of unsupervised approaches is more and more desirable but, unfortunately, their performances are not still satisfactory and more investigations are required. To this end, this paper proposes a new unsupervised approach to automatically detect the center of the eye: its algorithmic core is a representation of the eye's shape that is obtained through a differential analysis of image intensities and the subsequent combination with the local variability of the appearance represented by self-similarity coefficients. The experimental evidence of the effectiveness of the method was demonstrated on challenging databases containing facial images. Moreover, its capabilities to accurately detect the centers of the eyes were also favourably compared with those of the leading state-of-the-art methods.

iShadow: Design of a Wearable, Real-Time Mobile Gaze Tracker

Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We instantiate this idea in a prototype hardware platform equipped with a low-power image sensor that provides random access to pixel values, a low-power ARM Cortex M3 microcontroller, and a bluetooth radio to communicate with a mobile phone. The sparse pixel-based gaze estimation algorithm is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function that minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 3 degrees.

Eye-tracking data quality as affected by ethnicity and experimental design

Lack of accuracy in eye-tracking data can be critical. If the point of gaze is not recorded accurately and reliably, the information obtained or action executed might be different from what the user intended. This study reports trackability, accuracy, and precision as indicators of eye-tracking data quality as measured at various head positions and light conditions for a sample of participants from three different ethnic groups. It was found that accuracy and precision for Asian participants was worse than that for African and Caucasian participants. No significant differences were found between the latter two ethnic groups. Operating distance had the largest effect on data quality, since it affected all indicators for all ethnic groups. Illumination had no significant effect on accuracy or precision, but the accuracy achieved by African and Caucasian participants was better when the stimulus was presented on a dark background. Large gaze angles proved to be detrimental for trackability for African participants, while accuracy and precision were also affected adversely by larger gaze angles for two of the ethnicities.

An investigation on the feasibility of uncalibrated and unconstrained gaze tracking for human assistive applications by using head pose estimation

This paper investigates the possibility of accurately detecting and tracking human gaze by using an unconstrained and noninvasive approach based on the head pose information extracted by an RGB-D device. The main advantages of the proposed solution are that it can operate in a totally unconstrained environment, it does not require any initial calibration and it can work in real-time. These features make it suitable for being used to assist human in everyday life (e.g., remote device control) or in specific actions (e.g., rehabilitation), and in general in all those applications where it is not possible to ask for user cooperation (e.g., when users with neurological impairments are involved). To evaluate gaze estimation accuracy, the proposed approach has been largely tested and results are then compared with the leading methods in the state of the art, which, in general, make use of strong constraints on the people movements, invasive/additional hardware and supervised pattern recognition modules. Experimental tests demonstrated that, in most cases, the errors in gaze estimation are comparable to the state of the art methods, although it works without additional constraints, calibration and supervised learning.

Medical tongue piercing - development and evaluation of a surgical protocol and the perception of procedural discomfort of the participants

BACKGROUND

A system providing disabled persons with control of various assistive devices with the tongue has been developed at Aalborg University in Denmark. The system requires an activation unit attached to the tongue with a small piercing. The aim of this study was to establish and evaluate a safe and tolerable procedure for medical tongue piercing and to evaluate the expected and perceived procedural discomfort.

METHODS

Four tetraplegic subjects volunteered for the study. A surgical protocol for a safe insertion of a tongue barbell piercing was presented using sterilized instruments and piercing parts. Moreover, post-procedural observations of participant complications such as bleeding, edema, and infection were recorded. Finally, procedural discomforts were monitored by VAS scores of pain, changes in taste and speech as well as problems related to hitting the teeth.

RESULTS

The piercings were all successfully inserted in less than 5 min and the pain level was moderate compared with oral injections. No bleeding, infection, embedding of the piercing, or tooth/gingival injuries were encountered; a moderate edema was found in one case without affecting the speech. In two cases the piercing rod later had to be replaced by a shorter rod, because participants complained that the rod hit their teeth. The replacements prevented further problems. Moreover, loosening of balls was encountered, which could be prevented with the addition of dental glue. No cases of swallowing or aspiration of the piercing parts were recorded.

CONCLUSIONS

The procedure proved simple, fast, and safe for insertion of tongue piercings for tetraplegic subjects in a clinical setting. The procedure represented several precautions in order to avoid risks in these susceptible participants with possible co-morbidity. No serious complications were encountered, and the procedure was found tolerable to the participants. The procedure may be used in future studies with tongue piercings being a prerequisite for similar systems, and this may include insertion in an out-patient setting.