Modeling Eye Movements During Decision Making: A Review

SPEED: A Graphical User Interface Software for Processing Eye Tracking Data

Eye tracking is a tool that is widely used in scientific research, enabling the acquisition of precise and detailed data on an individual's eye movements during interaction with visual stimuli, thus offering a rich source of information on visual perception and associated cognitive processes. In this work, a new software called SPEED (labScoc Processing and Extraction of Eye tracking Data) is presented to process data acquired by Pupil Lab Neon (Pupil Labs, Berlin, Germany). The software is written in Python which helps researchers with the feature extraction step without any coding skills. This work also presents a pilot study in which five healthy subjects were included in research investigating oculomotor correlates during MDMT (Moral Decision-Making Task) and testing possible autonomic predictors of participants' performance. A statistically significant difference was observed in reaction times and in the number of blinks made during the choice between the conditions of the personal and impersonal dilemma.

Egoistic versus Prosocial Decision Making: an Eye Movement Data Report

Eye tracking data are highly promising in revealing novel and valuable evidence on human behavior and decision making. Data descripted in this article were collected in fourteen experiments with SMI eye tracking glasses in individual and social decision making conditions. The dataset is available on Harvard Dataverse and include data of 14 subjects with 4,180 visual behavior metrics summary and 3,744 eye moment records in decision-related areas of attention. Data may be applicable in computational models of oculomotor activity to explain decision process and predict its outcomes.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Feasibility of virtual reality and machine learning to assess personality traits in an organizational environment

Introduction

Personality plays a crucial role in shaping an individual's interactions with the world. The Big Five personality traits are widely used frameworks that help describe people's psychological behaviours. These traits predict how individuals behave within an organizational setting.

Methods

In this article, we introduce a virtual reality (VR) strategy for relatively scoring an individual's personality to evaluate the feasibility of predicting personality traits from implicit measures captured from users interacting in VR simulations of different organizational situations. Specifically, eye-tracking and decision-making patterns were used to classify individuals according to their level in each of the Big Five dimensions using statistical machine learning (ML) methods. The virtual environment was designed using an evidence-centered design approach.

Results

The dimensions were assessed using NEO-FFI inventory. A random forest ML model provided 83% accuracy in predicting agreeableness. A k-nearest neighbour ML model provided 75%, 75%, and 77% accuracy in predicting openness, neuroticism, and conscientiousness, respectively. A support vector machine model provided 85% accuracy for predicting extraversion. These analyses indicated that the dimensions could be differentiated by eye-gaze patterns and behaviours during immersive VR.

Discussion

Eye-tracking measures contributed more significantly to this differentiation than the behavioural metrics. Currently, we have obtained promising results with our group of participants, but to ensure the robustness and generalizability of our findings, it is imperative to replicate the study with a considerably larger sample. This study demonstrates the potential of VR and ML to recognize personality traits.

Attention biases the process of risky decision-making: Evidence from eye-tracking

Attention determines what kind of option information is processed during risky choices owing to the limitation of visual attention. This paper reviews research on the relationship between higher-complexity risky decision-making and attention as illustrated by eye-tracking to explain the process of risky decision-making by the effect of attention. We demonstrate this process from three stages: the pre-phase guidance of options on attention, the process of attention being biased, and the impact of attention on final risk preference. We conclude that exogenous information can capture attention directly to salient options, thereby altering evidence accumulation. In particular, for multi-attribute risky decision-making, attentional advantages increase the weight of specific attributes, thus biasing risk preference in different directions. We highlight the significance of understanding how people use available information to weigh risks from an information-processing perspective via process data.

Active Predictive Coding: A Unifying Neural Model for Active Perception, Compositional Learning, and Hierarchical Planning

There is growing interest in predictive coding as a model of how the brain learns through predictions and prediction errors. Predictive coding models have traditionally focused on sensory coding and perception. Here we introduce active predictive coding (APC) as a unifying model for perception, action, and cognition. The APC model addresses important open problems in cognitive science and AI, including (1) how we learn compositional representations (e.g., part-whole hierarchies for equivariant vision) and (2) how we solve large-scale planning problems, which are hard for traditional reinforcement learning, by composing complex state dynamics and abstract actions from simpler dynamics and primitive actions. By using hypernetworks, self-supervised learning, and reinforcement learning, APC learns hierarchical world models by combining task-invariant state transition networks and task-dependent policy networks at multiple abstraction levels. We illustrate the applicability of the APC model to active visual perception and hierarchical planning. Our results represent, to our knowledge, the first proof-of-concept demonstration of a unified approach to addressing the part-whole learning problem in vision, the nested reference frames learning problem in cognition, and the integrated state-action hierarchy learning problem in reinforcement learning.

Evidence for human-centric in-vehicle lighting: part 3-Illumination preferences based on subjective ratings, eye-tracking behavior, and EEG features

Within this third part of our mini-series, searching for the best and worst automotive in-vehicle lighting settings, we aim to extend our previous finding about white light illumination preferences by adding local cortical area activity as one key indicator. Frontal electrical potential asymmetry, measured using an electroencephalogram (EEG), is a highly correlated index for identifying positive and negative emotional behavior, primarily in the alpha band. It is rarely understood to what extent this observation can be applied to the evaluation of subjective preference or dislike based on luminaire variations in hue, chroma, and lightness. Within a controlled laboratory study, we investigated eight study participants who answered this question after they were shown highly immersive 360° image renderings. By so doing, we first subjectively defined, based on four different external driving scenes varying in location and time settings, the best and worst luminaire settings by changing six unlabeled luminaire sliders. Emotional feedback was collected based on semantic differentials and an emotion wheel. Furthermore, we recorded 120 Hz gaze data to identify the most important in-vehicle area of interest during the luminaire adaptation process. In the second study session, we recorded EEG data during a binocular observation task of repeated images arbitrarily paired by previously defined best and worst lighting settings and separated between all four driving scenes. Results from gaze data showed that the central vehicle windows with the left-side orientated colorful in-vehicle fruit table were both significantly longer fixed than other image areas. Furthermore, the previously identified cortical EEG feature describing the maximum power spectral density could successfully separate positive and negative luminaire settings based only on cortical activity. Within the four driving scenes, two external monotonous scenes followed trendlines defined by highly emotionally correlated images. More interesting external scenes contradicted this trend, suggesting an external emotional bias stronger than the emotional changes created by luminaires. Therefore, we successfully extended our model to define the best and worst in-vehicle lighting with cortical features by touching the field of neuroaesthetics.

Distinctive visual tasks for characterizing mild cognitive impairment and dementia using oculomotor behavior

Introduction

One's eye movement (in response to visual tasks) provides a unique window into the cognitive processes and higher-order cognitive functions that become adversely affected in cases with cognitive decline, such as those mild cognitive impairment (MCI) and dementia. MCI is a transitional stage between normal aging and dementia.

Methods

In the current work, we have focused on identifying visual tasks (such as horizontal and vertical Pro-saccade, Anti-saccade and Memory Guided Fixation tasks) that can differentiate individuals with MCI and dementia from their cognitively unimpaired healthy aging counterparts based on oculomotor Performance indices. In an attempt to identify the optimal combination of visual tasks that can be used to differentiate the participant groups, clustering was performed using the oculomotor Performance indices.

Results

Results of our study with a group of 60 cognitively unimpaired healthy aging individuals, a group with 60 individuals with MCI and a group with 60 individuals with dementia indicate that the horizontal and vertical Anti-saccade tasks provided the optimal combination that could differentiate individuals with MCI and dementia from their cognitively unimpaired healthy aging counterparts with clustering accuracy of ∼92% based on the saccade latencies. Also, the saccade latencies during both of these Anti-saccade tasks were found to strongly correlate with the Neuropsychological test scores.

Discussion

This suggests that the Anti-saccade tasks can hold promise in clinical practice for professionals working with individuals with MCI and dementia.

EEG-Based Emotion Classification in Financial Trading Using Deep Learning: Effects of Risk Control Measures

Day traders in the financial markets are under constant pressure to make rapid decisions and limit capital losses in response to fluctuating market prices. As such, their emotional state can greatly influence their decision-making, leading to suboptimal outcomes in volatile market conditions. Despite the use of risk control measures such as stop loss and limit orders, it is unclear if these strategies have a substantial impact on the emotional state of traders. In this paper, we aim to determine if the use of limit orders and stop loss has a significant impact on the emotional state of traders compared to when these risk control measures are not applied. The paper provides a technical framework for valence-arousal classification in financial trading using EEG data and deep learning algorithms. We conducted two experiments: the first experiment employed predetermined stop loss and limit orders to lock in profit and risk objectives, while the second experiment did not employ limit orders or stop losses. We also proposed a novel hybrid neural architecture that integrates a Conditional Random Field with a CNN-BiLSTM model and employs Bayesian Optimization to systematically determine the optimal hyperparameters. The best model in the framework obtained classification accuracies of 85.65% and 85.05% in the two experiments, outperforming previous studies. Results indicate that the emotions associated with Low Valence and High Arousal, such as fear and worry, were more prevalent in the second experiment. The emotions associated with High Valence and High Arousal, such as hope, were more prevalent in the first experiment employing limit orders and stop loss. In contrast, High Valence and Low Arousal (calmness) emotions were most prominent in the control group which did not engage in trading activities. Our results demonstrate the efficacy of our proposed framework for emotion classification in financial trading and aid in the risk-related decision-making abilities of day traders. Further, we present the limitations of the current work and directions for future research.

Visuospatial information foraging describes search behavior in learning latent environmental features

In the real world, making sequences of decisions to achieve goals often depends upon the ability to learn aspects of the environment that are not directly perceptible. Learning these so-called latent features requires seeking information about them. Prior efforts to study latent feature learning often used single decisions, used few features, and failed to distinguish between reward-seeking and information-seeking. To overcome this, we designed a task in which humans and monkeys made a series of choices to search for shapes hidden on a grid. On our task, the effects of reward and information outcomes from uncovering parts of shapes could be disentangled. Members of both species adeptly learned the shapes and preferred to select tiles expected to be informative earlier in trials than previously rewarding ones, searching a part of the grid until their outcomes dropped below the average information outcome-a pattern consistent with foraging behavior. In addition, how quickly humans learned the shapes was predicted by how well their choice sequences matched the foraging pattern, revealing an unexpected connection between foraging and learning. This adaptive search for information may underlie the ability in humans and monkeys to learn latent features to support goal-directed behavior in the long run.