Person-Specific Gaze Estimation from Low-Quality Webcam Images

Gaze estimation is an established research problem in computer vision. It has various applications in real life, from human-computer interactions to health care and virtual reality, making it more viable for the research community. Due to the significant success of deep learning techniques in other computer vision tasks-for example, image classification, object detection, object segmentation, and object tracking-deep learning-based gaze estimation has also received more attention in recent years. This paper uses a convolutional neural network (CNN) for person-specific gaze estimation. The person-specific gaze estimation utilizes a single model trained for one individual user, contrary to the commonly-used generalized models trained on multiple people's data. We utilized only low-quality images directly collected from a standard desktop webcam, so our method can be applied to any computer system equipped with such a camera without additional hardware requirements. First, we used the web camera to collect a dataset of face and eye images. Then, we tested different combinations of CNN parameters, including the learning and dropout rates. Our findings show that building a person-specific eye-tracking model produces better results with a selection of good hyperparameters when compared to universal models that are trained on multiple users' data. In particular, we achieved the best results for the left eye with 38.20 MAE (Mean Absolute Error) in pixels, the right eye with 36.01 MAE, both eyes combined with 51.18 MAE, and the whole face with 30.09 MAE, which is equivalent to approximately 1.45 degrees for the left eye, 1.37 degrees for the right eye, 1.98 degrees for both eyes combined, and 1.14 degrees for full-face images.

Review and Evaluation of Eye Movement Event Detection Algorithms

Eye tracking is a technology aimed at understanding the direction of the human gaze. Event detection is a process of detecting and classifying eye movements that are divided into several types. Nowadays, event detection is almost exclusively done by applying a detection algorithm to the raw recorded eye-tracking data. However, due to the lack of a standard procedure for how to perform evaluations, evaluating and comparing various detection algorithms in eye-tracking signals is very challenging. In this paper, we used data from a high-speed eye-tracker SMI HiSpeed 1250 system and compared event detection performance. The evaluation focused on fixations, saccades and post-saccadic oscillation classification. It used sample-by-sample comparisons to compare the algorithms and inter-agreement between algorithms and human coders. The impact of varying threshold values on threshold-based algorithms was examined and the optimum threshold values were determined. This evaluation differed from previous evaluations by using the same dataset to evaluate the event detection algorithms and human coders. We evaluated and compared the different algorithms from threshold-based, machine learning-based and deep learning event detection algorithms. The evaluation results show that all methods perform well for fixation and saccade detection; however, there are substantial differences in classification results. Generally, CNN (Convolutional Neural Network) and RF (Random Forest) algorithms outperform threshold-based methods.

Biometric Identification Based on Keystroke Dynamics

The purpose of the paper is to study how changes in neural network architecture and its hyperparameters affect the results of biometric identification based on keystroke dynamics. The publicly available dataset of keystrokes was used, and the models with different parameters were trained using this data. Various neural network layers-convolutional, recurrent, and dense-in different configurations were employed together with pooling and dropout layers. The results were compared with the state-of-the-art model using the same dataset. The results varied, with the best-achieved accuracy equal to 82% for the identification (1 of 20) task.

Time Signature Detection: A Survey

This paper presents a thorough review of methods used in various research articles published in the field of time signature estimation and detection from 2003 to the present. The purpose of this review is to investigate the effectiveness of these methods and how they perform on different types of input signals (audio and MIDI). The results of the research have been divided into two categories: classical and deep learning techniques, and are summarized in order to make suggestions for future study. More than 110 publications from top journals and conferences written in English were reviewed, and each of the research selected was fully examined to demonstrate the feasibility of the approach used, the dataset, and accuracy obtained. Results of the studies analyzed show that, in general, the process of time signature estimation is a difficult one. However, the success of this research area could be an added advantage in a broader area of music genre classification using deep learning techniques. Suggestions for improved estimates and future research projects are also discussed.

Biometric Identification Based on Eye Movement Dynamic Features

The paper presents studies on biometric identification methods based on the eye movement signal. New signal features were investigated for this purpose. They included its representation in the frequency domain and the largest Lyapunov exponent, which characterizes the dynamics of the eye movement signal seen as a nonlinear time series. These features, along with the velocities and accelerations used in the previously conducted works, were determined for 100-ms eye movement segments. 24 participants took part in the experiment, composed of two sessions. The users’ task was to observe a point appearing on the screen in 29 locations. The eye movement recordings for each point were used to create a feature vector in two variants: one vector for one point and one vector including signal for three consecutive locations. Two approaches for defining the training and test sets were applied. In the first one, 75% of randomly selected vectors were used as the training set, under a condition of equal proportions for each participant in both sets and the disjointness of the training and test sets. Among four classifiers: kNN (k = 5), decision tree, naïve Bayes, and random forest, good classification performance was obtained for decision tree and random forest. The efficiency of the last method reached 100%. The outcomes were much worse in the second scenario when the training and testing sets when defined based on recordings from different sessions; the possible reasons are discussed in the paper.

Application of Time-Scale Decomposition of Entropy for Eye Movement Analysis

The methods for nonlinear time series analysis were used in the presented research to reveal eye movement signal characteristics. Three measures were used: approximate entropy, fuzzy entropy, and the Largest Lyapunov Exponent, for which the multilevel maps (MMs), being their time-scale decomposition, were defined. To check whether the estimated characteristics might be useful in eye movement events detection, these structures were applied in the classification process conducted with the usage of the kNN method. The elements of three MMs were used to define feature vectors for this process. They consisted of differently combined MM segments, belonging either to one or several selected levels, as well as included values either of one or all the analysed measures. Such a classification produced an improvement in the accuracy for saccadic latency and saccade, when compared with the previously conducted studies using eye movement dynamics.

Understanding Eye Movement Signal Characteristics Based on Their Dynamical and Fractal Features

Eye movement is one of the biological signals whose exploration may reveal substantial information, enabling greater understanding of the biology of the brain and its mechanisms. In this research, eye movement dynamics were studied in terms of chaotic behavior and self-similarity assessment to provide a description of young, healthy, oculomotor system characteristics. The first of the investigated features is present and advantageous for many biological objects or physiological phenomena, and its vanishing or diminishment may indicate a system pathology. Similarly, exposed self-similarity may prove useful for indicating a young and healthy system characterized by adaptability. For this research, 24 young people with normal vision were involved. Their eye movements were registered with the usage of a head-mounted eye tracker, using infrared oculography, embedded in the sensor, measuring the rotations of the left and the right eye. The influence of the preprocessing step in the form of the application of various filtering methods on the assessment of the final dynamics was also explored. The obtained results confirmed the existence of chaotic behavior in some parts of eye movement signal; however, its strength turned out to be dependent on the filter used. They also exposed the long-range correlation representing self-similarity, although the influence of the applied filters on these outcomes was not unveiled.

An Analysis of Entropy-Based Eye Movement Events Detection

Analysis of eye movement has attracted a lot of attention recently in terms of exploring areas of people’s interest, cognitive ability, and skills. The basis for eye movement usage in these applications is the detection of its main components—namely, fixations and saccades, which facilitate understanding of the spatiotemporal processing of a visual scene. In the presented research, a novel approach for the detection of eye movement events is proposed, based on the concept of approximate entropy. By using the multiresolution time-domain scheme, a structure entitled the Multilevel Entropy Map was developed for this purpose. The dataset was collected during an experiment utilizing the “jumping point” paradigm. Eye positions were registered with a 1000 Hz sampling rate. For event detection, the knn classifier was applied. The best classification efficiency in recognizing the saccadic period ranged from 83% to 94%, depending on the sample size used. These promising outcomes suggest that the proposed solution may be used as a potential method for describing eye movement dynamics.

Implicit Calibration Using Probable Fixation Targets

Proper calibration of eye movement signal registered by an eye tracker seems to be one of the main challenges in popularizing eye trackers as yet another user-input device. Classic calibration methods taking time and imposing unnatural behavior on eyes must be replaced by intelligent methods that are able to calibrate the signal without conscious cooperation by the user. Such an implicit calibration requires some knowledge about the stimulus a user is looking at and takes into account this information to predict probable gaze targets. This paper describes a possible method to perform implicit calibration: it starts with finding probable fixation targets (PFTs), then it uses these targets to build a mapping-probable gaze path. Various algorithms that may be used for finding PFTs and mappings are presented in the paper and errors are calculated using two datasets registered with two different types of eye trackers. The results show that although for now the implicit calibration provides results worse than the classic one, it may be comparable with it and sufficient for some applications.

Vision Diagnostics and Treatment System for Children with Disabilities

Vision plays a crucial role in children's mental development. Therefore, early diagnosis of any vision disparities and implementation of a correct therapy is very important. However, carrying out such a procedure in case of young children and especially children with brain dysfunctions poses some limitations due to cooperation problems. The vision diagnostics and treatment (VisDaT) system presented in this paper is meant to help therapists in proper diagnosis and treatment involving such children. It utilizes a computer connected to two monitors and equipped with a specialized software. The main system components are as follows: an eye tracker recording child's eye movements and a digital camera monitoring online child's reactions. The system is equipped with a specialized software, which creates the opportunity to stimulate children's vision with a dedicated stimulus and post hoc analyses of recorded sessions, which enable making decision as to the future treatment.

Gaze Self-Similarity Plot - A New Visualization Technique

Eye tracking has become a valuable way for extending knowledge of human behavior based on visual patterns. One of the most important elements of such an analysis is the presentation of obtained results, which proves to be a challenging task. Traditional visualization techniques such as scan-paths or heat maps may reveal interesting information, nonetheless many useful features are still not visible, especially when temporal characteristics of eye movement is taken into account. This paper introduces a technique called gaze self-similarity plot (GSSP) that may be applied to visualize both spatial and temporal eye movement features on the single two-dimensional plot. The technique is an extension of the idea of recurrence plots, commonly used in time series analysis. The paper presents the basic concepts of the proposed approach (two types of GSSP) complemented with some examples of what kind of information may be disclosed and finally showing areas of the GSSP possible applications.

Application of eye tracking in medicine: A survey, research issues and challenges

The performance and quality of medical procedures and treatments are inextricably linked to technological development. The application of more advanced techniques provides the opportunity to gain wider knowledge and deeper understanding of the human body and mind functioning. The eye tracking methods used to register eye movement to find the direction and targets of a person's gaze are well in line with the nature of the topic. By providing methods for capturing and processing images of the eye it has become possible not only to reveal abnormalities in eye functioning but also to conduct cognitive studies focused on learning about peoples' emotions and intentions. The usefulness of the application of eye tracking technology in medicine was proved in many research studies. The aim of this paper is to give an insight into those studies and the way they utilize eye imaging in medical applications. These studies were differentiated taking their purpose and experimental paradigms into account. Additionally, methods for eye movement visualization and metrics for its quantifying were presented. Apart from presenting the state of the art, the aim of the paper was also to point out possible applications of eye tracking in medicine that have not been exhaustively investigated yet, and are going to be a perspective long-term direction of research.