EEG-based index for engagement level monitoring during sustained attention

A Subject-Specific Attention Index Based on the Weighted Spectral Power

As an essential cognitive function, attention has been widely studied and various indices based on EEG have been proposed for its convenience and easy availability for real-time attention monitoring. Although existing indices based on spectral power of empirical frequency bands are able to describe the attentional state in some way, the reliability still needs to be improved. This paper proposed a subject-specific attention index based on the weighted spectral power. Unlike traditional indices, the ranges of frequency bands are not empirical but obtained from subject-specific change patterns of spectral power of electroencephalograph (EEG) to overcome the great inter-subject variance. In addition, the contribution of each frequency component in the frequency band is considered different. Specifically, the ratio of power spectral density (PSD) function in attentional and inattentional state is utilized to calculate the weight to enhance the effectiveness of the proposed index. The proposed subject-specific attention index based on the weighted spectral power is evaluated on two open datasets including EEG data of a total of 44 subjects. The results of the proposed index are compared with 3 traditional attention indices using various statistical analysis methods including significance tests and distribution variance measurements. According to the experimental results, the proposed index can describe the attentional state more accurately. The proposed index respectively achieves accuracies of 86.21% and 70.00% at the 1% significance level in both the t-test and Wilcoxon rank-sum test for two datasets, which obtains improvements of 41.38% and 20.00% compared to the best result of the traditional indices. These results indicate that the proposed index provides an efficient way to measure attentional state.

More than the sum of its parts: investigating episodic memory as a multidimensional cognitive process

Why do we remember some events but forget others? Previous studies attempting to decode successful vs. unsuccessful brain states to investigate this question have met with limited success, potentially due, in part, to assessing episodic memory as a unidimensional process, despite evidence that multiple domains contribute to episodic encoding. Using a novel machine learning algorithm known as "transfer learning", we leveraged visual perception, sustained attention, and selective attention brain states to better predict episodic memory performance from trial-to-trial encoding electroencephalography (EEG) activity. We found that this multidimensional treatment of memory decoding improved prediction performance compared to traditional, unidimensional, methods, with each cognitive domain explaining unique variance in decoding of successful encoding-related neural activity. Importantly, this approach could be applied to cognitive domains outside of memory. Overall, this study provides critical insight into the underlying reasons why some events are remembered while others are not.

Viewer Engagement in Response to Mixed and Uniform Emotional Content in Marketing Videos-An Electroencephalographic Study

This study presents the results of an experiment designed to investigate whether marketing videos containing mixed emotional content can sustain consumers interest longer compared to videos conveying a consistent emotional message. During the experiment, thirteen participants, wearing EEG (electroencephalographic) caps, were exposed to eight marketing videos with diverse emotional tones. Participant engagement was measured with an engagement index, a metric derived from the power of brain activity recorded over the frontal and parietal cortex and computed within three distinct frequency bands: theta (4-8 Hz), alpha (8-13 Hz), and beta (13-30 Hz). The outcomes indicated a statistically significant influence of emotional content type (mixed vs. consistent) on the duration of user engagement. Videos containing a mixed emotional message were notably more effective in sustaining user engagement, whereas the engagement level for videos with a consistent emotional message declined over time. The principal inference drawn from the study is that advertising materials conveying a consistent emotional message should be notably briefer than those featuring a mixed emotional message to achieve an equivalent level of message effectiveness, measured through engagement duration.

Ratio Indexes Based on Spectral Electroencephalographic Brainwaves for Assessment of Mental Involvement: A Systematic Review

BACKGROUND

This review systematically examined the scientific literature about electroencephalogram-derived ratio indexes used to assess human mental involvement, in order to deduce what they are, how they are defined and used, and what their best fields of application are. (2) Methods: The review was carried out according to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines. (3) Results: From the search query, 82 documents resulted. The majority (82%) were classified as related to mental strain, while 12% were classified as related to sensory and emotion aspects, and 6% to movement. The electroencephalographic electrode montage used was low-density in 13%, high-density in 6% and very-low-density in 81% of documents. The most used electrode positions for computation of involvement indexes were in the frontal and prefrontal cortex. Overall, 37 different formulations of involvement indexes were found. None of them could be directly related to a specific field of application. (4) Conclusions: Standardization in the definition of these indexes is missing, both in the considered frequency bands and in the exploited electrodes. Future research may focus on the development of indexes with a unique definition to monitor and characterize mental involvement.

EEG and peripheral markers of viewer ratings: a study of short films

Introduction

Cinema is an important part of modern culture, influencing millions of viewers. Research suggested many models for the prediction of film success, one of them being the use of neuroscientific tools. The aim of our study was to find physiological markers of viewer perception and correlate them to short film ratings given by our subjects. Short films are used as a test case for directors and screenwriters and can be created to raise funding for future projects; however, they have not been studied properly with physiological methods.

Methods

We recorded electroencephalography (18 sensors), facial electromyography (corrugator supercilii and zygomaticus major), photoplethysmography, and skin conductance in 21 participants while watching and evaluating 8 short films (4 dramas and 4 comedies). Also, we used machine learning (CatBoost, SVR) to predict the exact rating of each film (from 1 to 10), based on all physiological indicators. In addition, we classified each film as low or high rated by our subjects (with Logistic Regression, KNN, decision tree, CatBoost, and SVC).

Results

The results showed that ratings did not differ between genres. Corrugator supercilii activity ("frowning" muscle) was larger when watching dramas; whereas zygomaticus major ("smiling" muscle) activity was larger during the watching of comedies. Of all somatic and vegetative markers, only zygomaticus major activity, PNN50, SD1/SD2 (heart rate variability parameters) positively correlated to the film ratings. The EEG engagement indices, beta/(alpha+theta) and beta/alpha correlated positively with the film ratings in the majority of sensors. Arousal (beta_F3 + beta_F4)/(alpha_F3 + alpha_F4), and valence (alpha_F4/beta_F4) - (alpha_F3/beta_F3) indices also correlated positively to film ratings. When we attempted to predict exact ratings, MAPE was 0.55. As for the binary classification, logistic regression yielded the best values (area under the ROC curve = 0.62) than other methods (0.51-0.60).

Discussion

Overall, we revealed EEG and peripheral markers, which reflect viewer ratings and can predict them to a certain extent. In general, high film ratings can reflect a fusion of high arousal and different valence, positive valence being more important. These findings broaden our knowledge about the physiological basis of viewer perception and can be potentially used at the stage of film production.

Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills

Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.

Central EEG Beta/Alpha Ratio Predicts the Population-Wide Efficiency of Advertisements

Recent studies have demonstrated that the brain activity of a group of people can be used to forecast choices at the population level. In this study, we attempted to neuroforecast aggregate consumer behavior of Internet users. During our electroencephalography (EEG) and eye-tracking study, participants were exposed to 10 banners that were also used in the real digital marketing campaign. In the separate online study, we additionally collected self-reported preferences for the same banners. We explored the relationship between the EEG, eye-tracking, and behavioral indexes obtained in our studies and the banners' aggregate efficiency provided by the large food retailer based on the decisions of 291,301 Internet users. An EEG-based engagement index (central beta/alpha ratio) significantly correlated with the aggregate efficiency of banners. Furthermore, our multiple linear regression models showed that a combination of eye-tracking, EEG and behavioral measurements better explained the market-level efficiency of banner advertisements than each measurement alone. Overall, our results confirm that neural signals of a relatively small number of individuals can forecast aggregate behavior at the population level.

An Evaluation of the EEG Alpha-to-Theta and Theta-to-Alpha Band Ratios as Indexes of Mental Workload

Many research works indicate that EEG bands, specifically the alpha and theta bands, have been potentially helpful cognitive load indicators. However, minimal research exists to validate this claim. This study aims to assess and analyze the impact of the alpha-to-theta and the theta-to-alpha band ratios on supporting the creation of models capable of discriminating self-reported perceptions of mental workload. A dataset of raw EEG data was utilized in which 48 subjects performed a resting activity and an induced task demanding exercise in the form of a multitasking SIMKAP test. Band ratios were devised from frontal and parietal electrode clusters. Building and model testing was done with high-level independent features from the frequency and temporal domains extracted from the computed ratios over time. Target features for model training were extracted from the subjective ratings collected after resting and task demand activities. Models were built by employing Logistic Regression, Support Vector Machines and Decision Trees and were evaluated with performance measures including accuracy, recall, precision and f1-score. The results indicate high classification accuracy of those models trained with the high-level features extracted from the alpha-to-theta ratios and theta-to-alpha ratios. Preliminary results also show that models trained with logistic regression and support vector machines can accurately classify self-reported perceptions of mental workload. This research contributes to the body of knowledge by demonstrating the richness of the information in the temporal, spectral and statistical domains extracted from the alpha-to-theta and theta-to-alpha EEG band ratios for the discrimination of self-reported perceptions of mental workload.

Multimodal Virtual Reality-Based Assessment of Adult ADHD: A Feasibility Study in Healthy Subjects

Neuropsychological assessments are often surprisingly inaccurate in mapping clinically-reported attention-deficit hyperactivity disorder (ADHD) symptoms, presumably due to their low ecological validity. Virtual reality (VR) might offer a potential solution for this problem, given its capability to generate standardized and yet highly realistic virtual environments. As the first adaptation of existing virtual classroom scenarios to an adult population, we developed a Virtual Seminar Room (VSR) for multimodal characterization of ADHD symptoms. To test its feasibility, N = 35 healthy participants were immersed into the VSR via a head-mounted display and carried out a VR-embedded continuous performance task (CPT) under varying levels of distractions in two experimental blocks (24 min each). CPT performance, electroencephalography (EEG) measures, and head movements (actigraphy) were simultaneously recorded and analyzed offline. Although CPT performance remained constant throughout the task, head movements increased significantly from Block 1 to Block 2. In addition, EEG theta (4-7 Hz) and beta (13-30 Hz) power was higher during Block 1 than Block 2, and during distractor-present than distractor-absent phases. Moreover, P300 amplitudes were higher during Block 1 than Block 2, and P300 latencies were prolonged in distractor-absent compared with distractor-present phases. Although the paradigm awaits further improvements, this study confirms the general feasibility of the VSR and provides a first step toward a multimodal, ecologically valid, and reliable VR-based adult ADHD assessment.

“Sound” Decisions: The Combined Role of Ambient Noise and Cognitive Regulation on the Neurophysiology of Food Cravings

Our ability to evaluate long-term goals over immediate rewards is manifested in the brain’s decision circuit. Simplistically, it can be divided into a fast, impulsive, reward “system 1” and a slow, deliberate, control “system 2.” In a noisy eating environment, our cognitive resources may get depleted, potentially leading to cognitive overload, emotional arousal, and consequently more rash decisions, such as unhealthy food choices. Here, we investigated the combined impact of cognitive regulation and ambient noise on food cravings through neurophysiological activity. Thirty-seven participants were recruited for an adapted version of the Regulation of Craving (ROC) task. All participants underwent two sessions of the ROC task; once with soft ambient restaurant noise (∼50 dB) and once with loud ambient restaurant noise (∼70 dB), while data from electroencephalography (EEG), electrodermal activity (EDA), and self-reported craving were collected for all palatable food images presented in the task. The results indicated that thinking about future (“later”) consequences vs. immediate (“now”) sensations associated with the food decreased cravings, which were mediated by frontal EEG alpha power. Likewise, “later” trials also increased frontal alpha asymmetry (FAA) —an index for emotional motivation. Furthermore, loud (vs. soft) noise increased alpha, beta, and theta activity, but for theta activity, this was solely occurring during “later” trials. Similarly, EDA signal peak probability was also higher during loud noise. Collectively, our findings suggest that the presence of loud ambient noise in conjunction with prospective thinking can lead to the highest emotional arousal and cognitive load as measured by EDA and EEG, respectively, both of which are important in regulating cravings and decisions. Thus, exploring the combined effects of interoceptive regulation and exteroceptive cues on food-related decision-making could be methodologically advantageous in consumer neuroscience and entail theoretical, commercial, and managerial implications.

Neurophysiological Verbal Working Memory Patterns in Children: Searching for a Benchmark of Modality Differences in Audio/Video Stimuli Processing

Exploration of specific brain areas involved in verbal working memory (VWM) is a powerful but not widely used tool for the study of different sensory modalities, especially in children. In this study, for the first time, we used electroencephalography (EEG) to investigate neurophysiological similarities and differences in response to the same verbal stimuli, expressed in the auditory and visual modality during the n-back task with varying memory load in children. Since VWM plays an important role in learning ability, we wanted to investigate whether children elaborated the verbal input from auditory and visual stimuli through the same neural patterns and if performance varies depending on the sensory modality. Performance in terms of reaction times was better in visual than auditory modality (p = 0.008) and worse as memory load increased regardless of the modality (p < 0.001). EEG activation was proportionally influenced by task level and was evidenced in theta band over the prefrontal cortex (p = 0.021), along the midline (p = 0.003), and on the left hemisphere (p = 0.003). Differences in the effects of the two modalities were seen only in gamma band in the parietal cortices (p = 0.009). The values of a brainwave-based engagement index, innovatively used here to test children in a dual-modality VWM paradigm, varied depending on n-back task level (p = 0.001) and negatively correlated (p = 0.002) with performance, suggesting its computational effectiveness in detecting changes in mental state during memory tasks involving children. Overall, our findings suggest that auditory and visual VWM involved the same brain cortical areas (frontal, parietal, occipital, and midline) and that the significant differences in cortical activation in theta band were more related to memory load than sensory modality, suggesting that VWM function in the child's brain involves a cross-modal processing pattern.

Sensor-based indicators of performance changes between sessions during robotic surgery training

Training of surgeons is essential for safe and effective use of robotic surgery, yet current assessment tools for learning progression are limited. The objective of this study was to measure changes in trainees' cognitive and behavioral states as they progressed in a robotic surgeon training curriculum at a medical institution. Seven surgical trainees in urology who had no formal robotic training experience participated in the simulation curriculum. They performed 12 robotic skills exercises with varying levels of difficulty repetitively in separate sessions. EEG (electroencephalogram) activity and eye movements were measured throughout to calculate three metrics: engagement index (indicator of task engagement), pupil diameter (indicator of mental workload) and gaze entropy (indicator of randomness in gaze pattern). Performance scores (completion of task goals) and mental workload ratings (NASA-Task Load Index) were collected after each exercise. Changes in performance scores between training sessions were calculated. Analysis of variance, repeated measures correlation, and machine learning classification were used to diagnose how cognitive and behavioral states associate with performance increases or decreases between sessions. The changes in performance were correlated with changes in engagement index (rrm=-.25,p<.001) and gaze entropy (rrm=-.37,p<.001). Changes in cognitive and behavioral states were able to predict training outcomes with 72.5% accuracy. Findings suggest that cognitive and behavioral metrics correlate with changes in performance between sessions. These measures can complement current feedback tools used by medical educators and learners for skills assessment in robotic surgery training.

Neurophysiological Differences Between Women With Fibromyalgia and Healthy Controls During Dual Task: A Pilot Study

Background

Women with FM have a reduced ability to perform two simultaneous tasks. However, the impact of dual task (DT) on the neurophysiological response of women with FM has not been studied.

Objective

To explore both the neurophysiological response and physical performance of women with FM and healthy controls while performing a DT (motor–cognitive).

Design

Cross-sectional study.

Methods

A total of 17 women with FM and 19 age- and sex-matched healthy controls (1:1 ratio) were recruited. The electroencephalographic (EEG) activity was recorded while participants performed two simultaneous tasks: a motor (30 seconds arm-curl test) and a cognitive (remembering three unrelated words). Theta (4–7 Hz), alpha (8–12 Hz), and beta (13–30) frequency bands were analyzed by using EEGLAB.

Results

Significant differences were obtained in the healthy control group between single task (ST) and DT in the theta, alpha, and beta frequency bands (p-value < 0.05). Neurophysiological differences between ST and DT were not found in women with FM. In addition, between-group differences were found in the alpha and beta frequency bands between healthy and FM groups, with lower values of beta and alpha in the FM group. Therefore, significant group^∗condition interactions were detected in the alpha and beta frequency bands. Regarding physical condition performance, between groups, analyses showed that women with FM obtained significantly worse results in the arm curl test than healthy controls, in both ST and DT.

Conclusion

Women with FM showed the same electrical brain activity pattern during ST and DT conditions, whereas healthy controls seem to adapt their brain activity to task commitment. This is the first study that investigates the neurophysiological response of women with FM while simultaneously performing a motor and a cognitive task.

Prediction of Motor Imagery Performance based on Pre-Trial Spatio-Spectral Alertness Features

Electroencephalogram (EEG) based brain-computer interfaces (BCIs) enable communication by interpreting the user intent based on measured brain electrical activity. Such interpretation is usually performed by supervised classifiers constructed in training sessions. However, changes in cognitive states of the user, such as alertness and vigilance, during test sessions lead to variations in EEG patterns, causing classification performance decline in BCI systems. This research focuses on effects of alertness on the performance of motor imagery (MI) BCI as a common mental control paradigm. It proposes a new protocol to predict MI performance decline by alertness-related pre-trial spatio-spectral EEG features. The proposed protocol can be used for adapting the classifier or restoring alertness based on the cognitive state of the user during BCI applications.

The Low-Frequency Fluctuation of Trial-by-Trial Frontal Theta Activity and Its Correlation With Reaction-Time Variability in Sustained Attention

Reaction-time variability is a critical index of sustained attention. However, researchers still lack effective measures to establish the association between neurophysiological activity and this behavioral variability. Here, the present study recorded reaction time (RT) and cortical electroencephalogram (EEG) in healthy subjects when they continuously performed an alternative responding task. The frontal theta activity and reaction-time variability were examined trial by trial using the measures of standard deviation (SD) in the time domain and amplitude of low-frequency fluctuation (ALFF) in the frequency domain. Our results showed that the SD of reaction-time variability did not have any correlation with the SD of trial-by-trial frontal theta activity, and the ALFF of reaction-time variability has a significant correlation with the ALFF of trial-by-trial frontal theta activity in 0.01–0.027 Hz. These results suggested the methodological significance of ALFF in establishing the association between neurophysiological activity and reaction-time variability. Furthermore, these findings also support the low-frequency fluctuation as a potential feature of sustained attention.

AttentivU: a Biofeedback Device to Monitor and Improve Engagement in the Workplace

Everyday work is becoming increasingly complex and cognitively demanding. A person's level of attention influences how effectively their brain prepares itself for action, and how much effort they apply to a task. However, the various distractions of the modern work environment often make it hard to pay and sustain attention. To address this issue, we present AttentivU - a system that uses wearable electroencephalography (EEG) to measure the attention of a person in real-time. When the user's attention level is low, the system provides real-time, subtle feedback to nudge the person to become attentive again. Users can choose to turn the device on or off based on whether their current task requires focused attention. We tested the system on 12 adults in a real workplace setting. The preliminary results show that the biofeedback redirects the attention of the participants to the task at hand and improves their performance.

Electroencephalographic Workload Indicators During Teleoperation of an Unmanned Aerial Vehicle Shepherding a Swarm of Unmanned Ground Vehicles in Contested Environments

Background: Although many electroencephalographic (EEG) indicators have been proposed in the literature, it is unclear which of the power bands and various indices are best as indicators of mental workload. Spectral powers (Theta, Alpha, and Beta) and ratios (Beta/(Alpha + Theta), Theta/Alpha, Theta/Beta) were identified in the literature as prominent indicators of cognitive workload.

Objective: The aim of the present study is to identify a set of EEG indicators that can be used for the objective assessment of cognitive workload in a multitasking setting and as a foundational step toward a human-autonomy augmented cognition system.

Methods: The participants' perceived workload was modulated during a teleoperation task involving an unmanned aerial vehicle (UAV) shepherding a swarm of unmanned ground vehicles (UGVs). Three sources of data were recorded from sixteen participants (n = 16): heart rate (HR), EEG, and subjective indicators of the perceived workload using the Air Traffic Workload Input Technique (ATWIT).

Results: The HR data predicted the scores from ATWIT. Nineteen common EEG features offered a discriminatory power of the four workload setups with high classification accuracy (82.23%), exhibiting a higher sensitivity than ATWIT and HR.

Conclusion: The identified set of features represents EEG indicators for the objective assessment of cognitive workload across subjects. These common indicators could be used for augmented intelligence in human-autonomy teaming scenarios, and form the basis for our work on designing a closed-loop augmented cognition system for human-swarm teaming.

Finger Temperature: A Psychophysiological Assessment of the Attentional State

Attention is a key cognitive phenomenon that is studied to understand cognitive disorders or even to estimate workloads to prevent accidents. Usually, it is studied using brain activity, even though it has many psychophysiological correlates. In the present study, we aim to evaluate if finger temperature, as a surrogate of peripheral vasoconstriction, can be used to obtain similar and complementary information to electroencephalography (EEG) brain activity measurements. To conduct this, 34 participants were recruited and submitted to performing four tasks-one as a baseline, and three attentional tasks. These three attentional tasks measured sustained attention, resilience to distractors, and attentional resources. During the tasks, the room, forehead, tympanic, and finger temperatures were measured. Furthermore, we included a 32-channel EEG recording. Our results showed a strong monotonic association between the finger temperature and the Alpha and Beta EEG spectral bands. When predicting attentional performance, the finger temperature was complementary to the EEG spectral measurements, through the prediction of aspects of attentional performance that had not been assessed by spectral EEG activity, or through the improvement of the model's fit. We also found that during the baseline task (non-goal-oriented task), the spectral EEG activity has an inverted correlation, as compared to a goal-oriented task. Our current results suggest that the psychophysiological assessment of attention is complementary to classic EEG approach, while also having the advantage of easy implementation of analysis tools in environments of reducing control (workplaces, student classrooms).