Measuring mental workload using physiological measures: A systematic review

Physiological Predictors of Operator Performance: The Role of Mental Effort and Its Link to Task Performance

ObjectiveThe present study investigated how pupil size and heart rate variability (HRV) can contribute to the prediction of operator performance. We illustrate how focusing on mental effort as the conceptual link between physiological measures and task performance can align relevant empirical findings across research domains.BackgroundPhysiological measures are often treated as indicators of operators' mental state. Thereby, they could enable a continuous and unobtrusive assessment of operators' current ability to perform the task.MethodFifty participants performed a process monitoring task consisting of ten 9-minute task blocks. Blocks alternated between low and high task demands, and the last two blocks introduced a task reward manipulation. We measured response times as primary performance indicator, pupil size and HRV as physiological measures, and mental fatigue, task engagement, and perceived effort as subjective ratings.ResultsBoth increased pupil size and increased HRV significantly predicted better task performance. However, the underlying associations between physiological measures and performance were influenced by task demands and time on task. Pupil size, but not HRV, results were consistent with subjective ratings.ConclusionThe empirical findings suggest that, by capturing variance in operators' mental effort, physiological measures, specifically pupil size, can contribute to the prediction of task performance. Their predictive value is limited by confounding effects that alter the amount of effort required to achieve a given level of performance.ApplicationThe outlined conceptual approach and empirical results can guide study designs and performance prediction models that examine physiological measures as the basis for dynamic operator assistance.

Effects of anthropomorphic features of co-robots on users' perception and emotional responses during human-robot collaboration

Human-robot collaboration (HRC) is growing across sectors like manufacturing and healthcare. This study explores how anthropomorphic features, specifically robot appearance (human-like face, robot-like face and blank screen) and voice (human-like voice, robot-like voice and beeping sound), influence user perceptions and emotional responses. Subjective assessments included the Robotic Social Attributes Scale (RoSAS) and self-report, while physiological responses were measured using electrocardiogram (ECG), galvanic skin response (GSR) and eye-tracking. Results showed that robot-like faces and human-like voices were linked to more positive perceptions, while human-like faces and beeping sounds led to greater negative emotional responses. Physiologically, both beeping sound and human-like voices increased skin conductance, and human-like voices also raised heart rate variability (HRV), suggesting participants felt less stressed under human-like voice conditions. Eye-tracking data showed shorter blink durations with robot-like faces, indicating greater sustained attention. These findings highlight the importance of designing robot features to improve user experience and collaboration in HRC.

Cognitive Load Prediction From Multimodal Physiological Signals Using Multiview Learning

Predicting cognitive load is a crucial issue in the emerging field of human-computer interaction and holds significant practical value, particularly in flight scenarios. Although previous studies have realized efficient cognitive load classification, new research is still needed to adapt the current state-of-the-art multimodal fusion methods. Here, we proposed a feature selection framework based on multiview learning to address the challenges of information redundancy and reveal the common physiological mechanisms underlying cognitive load. Specifically, the multimodal signal features [electroencephalogram (EEG), electrodermal activity (EDA), electrocardiogram (ECG), electrooculogram (EOG), & eye movements] at three cognitive load levels were estimated during multiattribute task battery (MATB) tasks performed by 22 healthy participants and fed into a feature selection-multiview classification with cohesion and diversity (FS-MCCD) framework. The optimized feature set was extracted from the original feature set by integrating the weight of each view and the feature weights to formulate the ranking criteria. The cognitive load prediction model, evaluated using real-time classification results, achieved an average accuracy of 81.08% and an average F1-score of 80.94% for three-class classification among 22 participants. Furthermore, the weights of the physiological signal features revealed the physiological mechanisms related to cognitive load. Specifically, heightened cognitive load was linked to amplified $\delta$ and $\theta$ power in the frontal lobe, reduced $\alpha$ power in the parietal lobe, and an increase in pupil diameter. Thus, the proposed multimodal feature fusion framework emphasizes the effectiveness and efficiency of using these features to predict cognitive load.

Training and Assessment of Nontechnical Skills in Cardiothoracic Surgery: Current Literature Review

BACKGROUND

This review explores the surgical and psychological literature on cardiothoracic (CT) surgeons' intraoperative nontechnical skills (NTS). Successful task performance depends on the seamless integration of technical skills (TS) and NTS. This paper aims to identify the training and assessment modalities for NTS in cardiothoracic surgery (CTS).

METHODS

PubMed, Cochrane Library, BioMed Central, Medline, EDINA BIOSIS, Web-of-Knowledge, PsychINFO, and ScienceDirect were searched through to December 2023. Additionally, a manual search of the referenced work of each included article and relevant reviews in PubMed were performed to avoid missing any important data. Original research articles were included if they described NTS training modalities and their assessment in cardiac or thoracic surgeries. Exclusion criteria included any study without full-text availability, non-peer-reviewed publications, studies without extractable data, and duplicates. Retrieved articles were screened, and data extracted in duplicate by 2 independent reviewers.

RESULTS

The literature search found 371 relevant references related to cardiothoracic surgery, 17 articles were included in this literature review, and 12 training tools were identified.

CONCLUSIONS

This review demonstrates the need for additional work to create reliable training and assessment methods for NTS in CTS.

Systematic review of neurophysiological assessment techniques and metrics for mental workload evaluation in real-world settings

Introduction

Mental Workload (MWL) is a concept that has garnered increasing interest in professional settings but remains challenging to define consensually. The literature reports a plurality of operational definitions and assessment methods, with no established unified framework. This review aims to identify objective and validated measurement methods for evaluating MWL in real-world work contexts. Particular attention is given to neurophysiological methods, recognized for their efficiency and robustness, enabling real-time assessment without disrupting operator activity.

Method

To conduct this analysis, a systematic search was performed in three databases (PubMed, ScienceDirect, and IEEEXplore), covering studies published from their inception until March 30, 2023. Selection criteria included research focusing on MWL and its derivatives, as well as neurophysiological measures applied in real-world conditions. An initial screening based on titles and abstracts was followed by an in-depth review, assisted by the bibliometric software Rayyan.

Results

The explored concepts, applied methods, and study results were compiled into a synthesis table. Ultimately, 35 studies were included, highlighting the diversity of measurement tools used in field settings, often combined with subjective assessments.

Discussion

Furthermore, key physiological indicators such as ECG, eye data, EEG and the relationship between MWL metrics and those uses to measure stress are emphasized and discussed. A better understanding of these interrelations could refine the assessment of their respective impacts and help anticipate their consequences on workers' mental health and safety.

The application of fNIRS in studies on occupational workload: a systematic review

Background

Occupational workload can contribute to significant health problems such as chronic stress, fatigue and burnout. To investigate the underlying mechanisms, it is necessary to monitor brain activity in real work environments. Functional near-infrared spectroscopy (fNIRS) is a portable, non-invasive neuroimaging method that captures neural correlates of occupational workload under natural conditions. However, despite its increasing application, a comprehensive overview of fNIRS-based research in this field is lacking. Therefore, this systematic review examines how fNIRS can be utilized to investigate occupational workload.

Methods

Following PRISMA 2020 guidelines, we conducted our systematic review by searching Web of Science, PubMed, and Scopus between November 15, 2023 and March 20, 2025. We included all studies published in English or German at any date, as long as they examined healthy adult professionals performing occupational tasks with functional near-infrared spectroscopy (fNIRS). Extracted data included study characteristics, workload details, signal processing methods, main fNIRS findings, and study quality, assessed using the JBI Critical Appraisal Tool.

Results

We included 41 studies. Of these, 23 reported a significant increase in oxygenated hemoglobin (HbO) concentration and functional connectivity in the prefrontal cortex (PFC) under higher occupational workload conditions. Only five studies examined typical office tasks. Nine studies analyzed differences in cortical activation between experts and novices, with experts showing increased HbO concentration in the PFC than novices. Regarding methodology, 26 studies used standardized optode placements, while only 17 applied systemic and extracerebral artifact correction. Small sample sizes and the absence of randomized controlled trials limited the reliability and reproducibility of the findings.

Conclusion

Functional near-infrared spectroscopy effectively detects neural correlates of occupational workload and provides objective insights into cognitive demands in real-world work settings. Standardizing optode placement, harmonizing signal-processing methods, and increasing sample sizes would enhance the validity and comparability of future research. Expanding investigations to typical office environments is also crucial for understanding daily workload and for developing interventions that promote employee well-being and productivity. Overall, fNIRS represents a promising tool for establishing evidence-based workplace health promotion strategies across diverse occupational settings.

Detection of Cognitive Load Modulation by EDA and HRV

Electrodermal activity (EDA) and heart rate variability (HRV) offer opportunities to grasp critical manifestations of the nervous autonomic system using low-intrusive sensing tools. A key question relies on the capacity to adequately process EDA and HRV signals to extract cognitive load markers, a multifaceted construct with intricate neural networks functioning, where emotions interfere with cognition. Here, 34 participants (20 males, 19.2 ± 1.3 years) were exposed to two-back mental tasking and watching emotionally charged images while recording EDA and HRV. HRV signals were processed using variable frequency complex demodulation (VFCDM) and wavelet packet transform (WPT) to provide high- and low-frequency (HF and LF) markers. Three methods were used to extract EDA indices: VFCDM (EDATVSYMP), WPT (EDAWPT), and convex-optimization (EDACVX). Cognitive load and emotion epochs were distinguished by significant differences in NASA-TLX scores, mental fatigue, and stress, on the one hand; and by EDACVX and, remarkably, EDATVSYMP and HF-HRVVFCDM on the other hand. A linear mixed-effects model and stepwise backward selection procedure showed that these two markers were main predictors of the NASA-TLX score (cognitive load). The individual perception of cognitive load was finally discriminated by k-means clustering, showing three profiles of autonomic responses relying, respectively, on EDATVSYMP, HF-HRVVFCDM, or a mix of these two markers. The existence of EDA-, HRV-, and EDA/HRV-derived profiles might explain why previous attempts that have predominantly employed a single biosignal often remained unconclusive in evaluating the perceived cognitive load, thereby demonstrating the added value of the present approach to monitor mental-related workload in human operators.

Numerosity Perception and Perceptual Load: Exploring Sex Differences Through Eye-Tracking

This study investigates sex differences in numerosity perception and visuospatial abilities in adults using eye-tracking methodology. We report the results of a controlled dual-task experiment that assessed the participants' visuospatial and numerosity estimation abilities. We did not observe sex differences in reaction times and accuracy. However, we found that females consistently underestimated numerosity. This underestimation correlated with higher perceptual load in females, as evidenced by shorter fixation durations and increased fixation rates. These findings suggest that perceptual load, rather than visual or spatial abilities, significantly influences numerosity estimation. Our study contributes novel insights into sex differences in both numerosity estimation and visuospatial abilities. These results provide a foundation for future research on numerosity perception across various populations and contexts, with implications for educational strategies and cognitive training programs.

Utilising raw psycho-physiological data and functional data analysis for estimating mental workload in human drivers

Recent studies have focused on accurately estimating mental workload using machine learning algorithms and extracting features from physiological measures. However, feature extraction leads to the loss of valuable information and often results in binary classifications that lack specificity in the identification of optimum mental workload. This study investigates the feasibility of using raw physiological data (EEG, facial EMG, ECG, EDA, pupillometry) combined with Functional Data Analysis (FDA) to estimate the mental workload of human drivers. A driving scenario with five tasks was employed, and subjective ratings were collected. Results demonstrate that the FDA applied nine different combinations of raw physiological signals achieving a maximum 90% accuracy, outperforming extracted features by 73%. This study shows that the mental workload of human drivers can be accurately estimated without utilising burdensome feature extraction. The approach proposed in this study offers promise for mental workload assessment in real-world applications.

Relationship between feedback frequency and task performance: evidence on the mediating role of heart rate

This study looks into the impact of feedback frequency, a source of cognitive load, on heart rate (HR) and heart rate variability (HRV). The experiment was undertaken using a sample of 96 university students, wherein the feedback frequency was manipulated during an arithmetic task performance. During the task, participants were instructed to wear the E4 Empatica device, which is used to assess HR and HRV. The metrics used to measure HRV were the Standard Deviation of NN Intervals (SDNN) and Root Mean Square of Successive Differences (RMSSD). The study revealed notable disparities in the average HR values across the three feedback conditions. Optimal performance was achieved when HR was elevated (indicating the highest level of cognitive load) and feedback frequency was moderate. Further, HR mediated the association between feedback frequency and task performance. However, no significant impact of feedback frequency on HRV was found.

Characterizing eye gaze and mental workload for assistive device control

Eye gaze tracking is increasingly popular due to improved technology and availability. In the domain of assistive device control, however, eye gaze tracking is often used in discrete ways (e.g., activating buttons on a screen), and does not harness the full potential of the gaze signal. In this article, we present a method for collecting both reactionary and controlled eye gaze signals, via screen-based tasks designed to isolate various types of eye movements. The resulting data allows us to build an individualized characterization for eye gaze interface use. Results from a study conducted with participants with motor impairments are presented, offering insights into maximizing the potential of eye gaze for assistive device control. Importantly, we demonstrate the potential for incorporating direct continuous eye gaze inputs into gaze-based interface designs; generally seen as intractable due to the 'Midas touch' problem of differentiating between gaze movements for perception versus for interface operation. Our key insight is to make use of an individualized measure of smooth pursuit characteristics to differentiate between gaze for control and gaze for environment scanning. We also present results relating to gaze-based metrics for mental workload and show the potential for the concurrent use of eye gaze for control input as well as assessing a user's mental workload both offline and in real-time. These findings might inform the development of continuous control paradigms using eye gaze, as well as the use of eye tracking as the sole input modality to systems that share control between human-generated and autonomy-generated inputs.

Neural impact of anti-G suits on pilots: Analyzing microstates and functional connectivity

Overload represents a significant challenge for pilots in flight, with a substantial impact on flight safety. Currently, the primary method of protection is the utilization of inflatable anti-G suit to address instances where blood is concentrated in the lower extremities. The inflatable air pressure of the anti-G suit varies in response to different overload conditions, which in turn affects the pilot's sensory and brain loads. However, this change has not yet been fully explored. To investigate the neural effects of pressure from the anti-G suit under different degrees of overload, this paper employs a pressurized simulation methodology. The subjects' brain state changes during the simulation are measured through electroencephalogram (EEG), and comparative calculations are performed using microstate and functional connectivity. The final results demonstrate that varying inflation levels of the bladder anti-G suit can influence the microstate and functional connectivity. The Duration, Coverage, Occurrence, and transition probability (TP) characteristics of microstate C demonstrated significant variance across three distinct levels of overload. The mean increase in Phase Locking Value (PLV) for overload 3 relative to the absence of overload was 13.8%, and the number of channel synchronizations underwent a transition from 7 to 62.

Integrating information from off-board sensors into the submarine control room: workload, situation awareness, and performance consequences

Off-board sensors have the potential to provide submarine control rooms with otherwise inaccessible information to supplement data gathered by on-board sensors. Using Uninhabited Aerial Vehicles (UAVs) as a representative test case, we examined how team performance, workload and situation awareness were impacted by receiving raw, compared to processed, sensor data from one or two off-board UAVs. Sixteen teams of three participants were trained to compile a tactical picture and to complete missions. Teams receiving processed off-board sensor data compiled more accurate tactical pictures, completed missions faster, and reported lower workload, compared to teams receiving raw off-board sensor data. Situation awareness, objective workload, and perceived usability were comparable between conditions. Scenarios with two UAVs took longer to complete, with other outcomes unaffected. These outcomes provide evidence that humans can integrate additional and distinct sources of information to form a cohesive combined picture, even when those data are partially redundant.

Predicting Return-to-Manual Performance in Lower- and Higher-Degree Automation

OBJECTIVE

To examine operator state variables (workload, fatigue, trust in automation, task engagement) that potentially predict return-to-manual (RTM) performance after automation fails to complete a task action.

BACKGROUND

Limited research has examined the extent to which within-person variability in operator states predicts RTM performance, a prerequisite to adapting work systems based on expected performance degradation/operator strain. We examine whether operator states differentially predict RTM performance as a function of degree of automation (DOA).

METHOD

Participants completed a simulated air traffic control task. Conflict detection was assisted by either a higher- or lower-DOA. When automation failed to resolve a conflict, participants needed to prevent that conflict (i.e., RTM). Participants' self-reported workload, fatigue, trust in automation, and task engagement were periodically measured.

RESULTS

Participants using lower DOA were faster to resolve conflicts (RTM RT) missed by automation than those using higher DOA. DOA did not moderate the relationship between operator states and RTM performance. Collapsed across DOA, increased workload (relative to participants' own average) and increased fatigue (relative to sample average, or relative to own average) led to the resolution of fewer conflicts missed by automation (poorer RTM accuracy). Participants with higher trust (relative to own average) had higher RTM accuracy.

CONCLUSIONS

Variation in operator state measures of workload, fatigue, and trust can predict RTM performance. However, given some identified inconsistency in which states are predictive across studies, further research is needed.

APPLICATIONS

Adaptive work systems could be designed to respond to vulnerable operator states to minimise RTM performance decrements.

Prediction of intrinsic and extraneous cognitive load with oculometric and biometric indicators

This study focused on the prediction of intrinsic and extraneous cognitive load using eye-tracking metrics, heart rate variability, and galvanic skin response. Intrinsic cognitive load is associated with the inherent complexity of the mental task, whereas extraneous cognitive load is related to the distracting and unrelated elements in the task. Thirty-three participants (aged [Formula: see text]) performed different levels of mental calculations to induce intrinsic cognitive load in the first task and a visual search task to manipulate extraneous cognitive load in the second task. During both tasks, participants' eye movements, heart rate, and galvanic skin response were continuously recorded. Participants' working memory was controlled. Subjective cognitive load was also assessed following each experimental task. A discriminant model, consisting of oculo- and bio-metric indicators, could discriminate between cognitive loads (intrinsic vs.extraneous) and levels (low vs.high). In particular, average fixation duration, average saccade amplitude, and [Formula: see text] coefficient each have an impact on the model. In addition, task difficulty may be distinguished by the Low-High Index of Pupillary Activity (LHIPA) and heart rate variability.

Gaze behaviours, situation awareness and cognitive workload of air traffic controllers in radar screen monitoring tasks with varying task complexity

Objectives. Air traffic controllers should maintain high situational awareness (SA) and low cognitive workload to ensure aviation safety. However, increased task complexity may influence air traffic controllers' SA and cognitive workload. Meanwhile, eye-tracking provides insights into the gaze patterns that might signify SA. This article investigates the gaze behaviours, SA and cognitive workload of different radar screen monitoring tasks with varying task complexity. Methods. Twenty-eight participants performed three radar screen monitoring tasks, including call-sign association, position identification and heading projection. Cognitive workload and SA were evaluated for each task using the National Aeronautics and Space Administration task load index (NASA-TLX) and situational awareness global assessment technique (SAGAT), respectively. The Gaussian mixture model was used to cluster the SA into high/low. Eye-tracking reveals the gaze behaviours that may contribute to the SA formation. Results. Cognitive workload and SA significantly differ between different levels of task complexity. While task complexity has a significant main effect on the fixations of human operators, it does not significantly influence the pupil diameter. Conclusions. Fixation-related metrics may signify the changes in SA under varying task complexity, while the side effects of cognitive workload should be mitigated.

Assessment of mental workload states in monitors using functional near-infrared spectroscopy

BACKGROUND

As technology develops rapidly and complex systems become more widespread, the study of mental workload (MWL) for monitors has become increasingly important.

OBJECTIVE

To explore the signaling features of functional near-infrared spectroscopy (fNIRS) in different MWL states.

METHODS

Twenty participants performed the N-back task and fNIRS data were collected to measure changes in blood oxygenation levels in the prefrontal cortex. Additionally, performance metrics and NASA Task Load Index (NASA-TLX) questionnaire results were recorded.

RESULTS

NASA-TLX scores increased progressively from 0-back (M = 8.750) to 1-back (M = 34.792), 2-back (M = 52.292), and 3-back (M = 75.625). Significant differences were observed in oxygenated hemoglobin (HbO) concentrations (F3, 57 = 10.557, p < 0.001) across these tasks. HbO increased from 0-back (M = -0.008) to 1-back (M = -0.002) and 2-back (M = -0.001), but decreased to 3-back (M = -0.002). Channels CH16, CH17, and CH18, located in the left dorsolateral prefrontal cortex (DLPFC), were significantly activated in the 1-back, 2-back, and 3-back conditions. CH1 and CH2, which are located in the right DLPFC, showed significant activation in both the 2-back and 3-back tasks.

CONCLUSIONS

Our results showed that different levels of the N-back tasks triggered different MWL states. Significant differences in HbO concentrations and activated channels were observed across tasks with varying MWLs. HbO followed an "inverted U" pattern, increasing as the MWL shifted from underload to normal, and subsequently decreasing as it reached overload. Additionally, the left DLPFC was activated earlier than the right DLPFC, with the latter gradually engaging as the MWL increased. This study provides valuable insights into the assessment of MWL states.

Effects of Mental Workload Manipulation on Electroencephalography Spectrum Oscillation and Microstates in Multitasking Environments

INTRODUCTION

Multitasking during flights leads to a high mental workload, which is detrimental for maintaining task performance. Electroencephalography (EEG) power spectral analysis based on frequency-band oscillations and microstate analysis based on global brain network activation can be used to evaluate mental workload. This study explored the effects of a high mental workload during simulated flight multitasking on EEG frequency-band power and microstate parameters.

METHODS

Thirty-six participants performed multitasking with low and high mental workloads after 4 consecutive days of training. Two levels of mental workload were set by varying the number of subtasks. EEG signals were acquired during the task. Power spectral and microstate analyses were performed on the EEG. The indices of four frequency bands (delta, theta, alpha, and beta) and four microstate classes (A-D) were calculated, changes in the frequency-band power and microstate parameters under different mental workloads were compared, and the relationships between the two types of EEG indices were analyzed.

RESULTS

The theta-, alpha-, and beta-band powers were higher under the high than under the low mental workload condition. Compared with the low mental workload condition, the high mental workload condition had a lower global explained variance and time parameters of microstate B but higher time parameters of microstate D. Less frequent transitions between microstates A and B and more frequent transitions between microstates C and D were observed during high mental workload conditions. The time parameters of microstate B were positively correlated with the delta-, theta-, and beta-band powers, whereas the duration of microstate C was negatively correlated with the beta-band power.

CONCLUSION

EEG frequency-band power and microstate parameters can be used to detect a high mental workload. Power spectral analyses based on frequency-band oscillations and microstate analyses based on global brain network activation were not completely isolated during multitasking.

Effects of dynamic thermal conditioning on cognitive load and performance in an office environment

In the design of buildings with minimal environmental impact, the incorporation of higher energy flexibility is becoming increasingly relevant. This approach is associated with dynamic modulations in setpoint temperatures. Until now, a link between indoor temperatures and cognitive performance of workers has been assumed, leading to high energy consumption and overcooling of office environments in summer conditions. However, research focusing on the relationship between thermal indoor environments and cognitive performance has rarely considered the influence of dynamic temperatures or temporal effects. This is the first experimental study aiming to understand the impact of temperature in relation to time of day on the subjective perception of cognitive load and performance under various thermal conditions in real-world office environments. The results indicated no observable relationship between temperature setpoints (25-30 °C) and cognitive performance. Instead, the temporal dynamics of cooling rather than fixed and static temperature setpoints appeared to have an impact.

Evidence of compensatory neural hyperactivity in a subgroup of breast cancer survivors treated with chemotherapy and its association with brain aging

Introduction

Chemotherapy-related cognitive impairment (CRCI) remains poorly understood in terms of the mechanisms of cognitive decline. Neural hyperactivity has been reported on average in cancer survivors, but it is unclear which patients demonstrate this neurophenotype, limiting precision medicine in this population.

Methods

We evaluated a retrospective sample of 80 breast cancer survivors and 80 non-cancer controls, aged 35-73, for which we had previously identified and validated three data-driven, biological subgroups (biotypes) of CRCI. We measured neural activity using the z-normalized percent amplitude of fluctuation from resting-state functional magnetic resonance imaging (MRI). We tested established, quantitative criteria to determine whether hyperactivity can accurately be considered compensatory. We also calculated the brain age gap by applying a previously validated algorithm to anatomic MRI.

Results

We found that neural activity differed across the three CRCI biotypes and controls (F = 13.5, p < 0.001), with Biotype 2 demonstrating significant hyperactivity compared to the other groups (p < 0.004, corrected), primarily in prefrontal regions. Alternatively, Biotypes 1 and 3 demonstrated significant hypoactivity (p < 0.02, corrected). Hyperactivity in Biotype 2 met several of the criteria to be considered compensatory. However, we also found a positive relationship between neural activity and the brain age gap in these patients (r = 0.45, p = 0.042).

Discussion

Our results indicated that neural hyperactivity is specific to a subgroup of breast cancer survivors and, while it seems to support preserved cognitive function, it could also increase the risk of accelerated brain aging. These findings could inform future neuromodulatory interventions with respect to the risks and benefits of upregulation or downregulation of neural activity.

EDA, PPG and Skin Temperature as Predictive Signals for Mental Failure by a Statistical Analysis on Stress and Mental Workload

Objective: The growth of autonomous systems interacting with humans leads to assessing operators' stress and mental workload (MWL), especially in safety-critical situations. Therefore, a system providing information about the psychophysiological workers' condition is fundamental and still missing. This paper aims to study the statistical relationship between the variation of Photoplethysmogram signal (PPG), Electrodermal Activity (EDA), and skin temperature with respect to stress and MWL levels, assessed through an ad-hoc developed subjective questionnaire. Results: 43 features were calculated from these signals during the execution of two cognitive tests and processed through a statistical analysis based on Kruskal-Wallis and Mann-Whitney U tests. This analysis proved that about 50% of them offered statistical evidence in differentiating relaxed and altered emotional conditions. Moreover, fifteen features were found to provide sufficient information to detect at the same time stress and MWL. Conclusions: These results demonstrate the feasibility of this approach and push to continue this research about the relationship between physiological signals and the variation of stress and MWL by enhancing the population and considering more biosignals.

An Objective Handling Qualities Assessment Framework of Electric Vertical Takeoff and Landing

Assessing handling qualities is crucial for ensuring the safety and operational efficiency of aircraft control characteristics. The growing interest in Urban Air Mobility (UAM) has increased the focus on electric Vertical Takeoff and Landing (eVTOL) aircraft; however, a comprehensive assessment of eVTOL handling qualities remains a challenge. This paper proposed a handling qualities framework to assess eVTOL handling qualities, integrating pilot compensation, task performance, and qualitative comments. An experiment was conducted, where eye-tracking data and subjective ratings from 16 participants as they performed various Mission Task Elements (MTEs) in an eVTOL simulator were analyzed. The relationship between pilot compensation and task workload was investigated based on eye metrics. Data mining results revealed that pilots’ eye movement patterns and workload perception change when performing Mission Task Elements (MTEs) that involve aircraft deficiencies. Additionally, pupil size, pupil diameter, iris diameter, interpupillary distance, iris-to-pupil ratio, and gaze entropy are found to be correlated with both handling qualities and task workload. Furthermore, a handling qualities and pilot workload recognition model is developed based on Long-Short Term Memory (LSTM), which is subsequently trained and evaluated with experimental data, achieving an accuracy of 97%. A case study was conducted to validate the effectiveness of the proposed framework. Overall, the proposed framework addresses the limitations of the existing Handling Qualities Rating Method (HQRM), offering a more comprehensive approach to handling qualities assessment.

Movement Sensing Opportunities for Monitoring Dynamic Cognitive States

In occupational domains such as sports, healthcare, driving, and military, both individuals and small groups are expected to perform challenging tasks under adverse conditions that induce transient cognitive states such as stress, workload, and uncertainty. Wearable and standoff 6DOF sensing technologies are advancing rapidly, including increasingly miniaturized yet robust inertial measurement units (IMUs) and portable marker-less infrared optical motion tracking. These sensing technologies may offer opportunities to track overt physical behavior and classify cognitive states relevant to human performance in diverse human-machine domains. We describe progress in research attempting to distinguish cognitive states by tracking movement behavior in both individuals and small groups, examining potential applications in sports, healthcare, driving, and the military. In the context of military training and operations, there are no generally accepted methods for classifying transient mental states such as uncertainty from movement-related data, despite its importance for shaping decision-making and behavior. To fill this gap, an example data set is presented including optical motion capture of rifle trajectories during a dynamic marksmanship task that elicits variable uncertainty; using machine learning, we demonstrate that features of weapon trajectories capturing the complexity of motion are valuable for classifying low versus high uncertainty states. We argue that leveraging metrics of human movement behavior reveals opportunities to complement relatively costly and less portable neurophysiological sensing technologies and enables domain-specific human-machine interfaces to support a wide range of cognitive functions.

A numerical evaluation of real-time workloads for ramp controller through optimization of multi-type feature combinations derived from eye tracker, respiratory, and fatigue patterns

Ramp controllers are required to manage their workloads effectively while handling complex operational tasks, a crucial part of improving aviation safety. The ability to detect their instantaneous workload is vital for ensuring operational effectiveness and preventing hazardous incidents. This paper introduces a novel methodology aimed at enhancing the evaluation of the ramp controller's cumulative workload by incorporating and optimizing the feature combination from eye movement, respiratory, and fatigue characteristics. Specifically, a 90-minute simulated experiment related to ramp control tasks, using real data from Shanghai Hongqiao Airport, is conducted to collect multi-type data from 8 controllers. Following data construction and the extraction of multi-type, the workloads of all samples are categorized through unsupervised learning. Subsequently, supervised learning techniques are used to calculate feature weights and train classifiers after data alignment. The optimal feature combination is established by calculating feature weights, and the best classification accuracy is over 98%, achieved by the KNN classifier. Furthermore, numerical evaluation and threshold calculations for different workload levels are interpreted. It is promising to provide insights into future works towards human-centered data construction, processing, and interpretation to promote the progress of workload assessment within the aviation industry.

Cognitive load during driving: EEG microstate metrics are sensitive to task difficulty and predict safety outcomes

Engaging in phone conversations or other cognitively challenging tasks while driving detrimentally impacts cognitive functions and has been associated with increased risk of accidents. Existing EEG methods have been shown to differentiate between load and no load, but not between different levels of cognitive load. Furthermore, it has not been investigated whether EEG measurements of load can be used to predict safety outcomes in critical events. EEG microstates analysis, categorizing EEG signals into a concise set of prototypical functional states, has been used in other task contexts with good results, but has not been applied in the driving context. Here, this gap is addressed by means of a driving simulation experiment. Three phone use conditions (no phone use, hands-free, and handheld), combined with two task difficulty levels (single- or double-digit addition and subtraction), were tested before and during a rear-end collision conflict. Both conventional EEG spectral power and EEG microstates were analyzed. The results showed that different levels of cognitive load influenced EEG microstates differently, while EEG spectral power remained unaffected. A distinct EEG pattern emerged when drivers engaged in phone tasks while driving, characterized by a simultaneous increase and decrease in two of the EEG microstates, suggesting a heightened focus on auditory information, potentially at a cost to attention reorientation ability. The increase and decrease in these two microstates follow a monotonic sequence from baseline to hands-free simple, hands-free complex, handheld simple, and finally handheld complex, showing sensitivity to task difficulty. This pattern was found both before and after the lead vehicle braked. Furthermore, EEG microstates prior to the lead vehicle braking improved predictions of safety outcomes in terms of minimum time headway after the lead vehicle braked, clearly suggesting that these microstates measure brain states which are indicative of impaired driving. Additionally, EEG microstates are more predictive of safety outcomes than task difficulty, highlighting individual differences in task effects. These findings enhance our understanding of the neural dynamics involved in distracted driving and can be used in methods for evaluating the cognitive load induced by in-vehicle systems.

Searching Across Realities: Investigating ERPs and Eye-Tracking Correlates of Visual Search in Mixed Reality

Mixed Reality allows us to integrate virtual and physical content into users' environments seamlessly. Yet, how this fusion affects perceptual and cognitive resources and our ability to find virtual or physical objects remains uncertain. Displaying virtual and physical information simultaneously might lead to divided attention and increased visual complexity, impacting users' visual processing, performance, and workload. In a visual search task, we asked participants to locate virtual and physical objects in Augmented Reality and Augmented Virtuality to understand the effects on performance. We evaluated search efficiency and attention allocation for virtual and physical objects using event-related potentials, fixation and saccade metrics, and behavioral measures. We found that users were more efficient in identifying objects in Augmented Virtuality, while virtual objects gained saliency in Augmented Virtuality. This suggests that visual fidelity might increase the perceptual load of the scene. Reduced amplitude in distractor positivity ERP, and fixation patterns supported improved distractor suppression and search efficiency in Augmented Virtuality. We discuss design implications for mixed reality adaptive systems based on physiological inputs for interaction.

Investigating mental workload across task modalities: a multimodal analysis using pupillometry

Understanding mental workload is challenging due to its multidimensional nature and varying sensitivities of its primary measures: task performance, perceived workload, and physiological responses. This study investigates the effects of task load on performance, perceived workload, and pupil measures across different information modalities. A within-subjects experiment involving three tasks (digit span, matrix span, and dual n-back) was conducted with three task load levels. Workload measures included accuracy/sensitivity, reaction time, NASA-TLX, peak pupil diameter, and peak pupil latency. Consistent patterns of associations between task load and these workload measures were found across the three modalities. Workload measures revealed a nuanced interplay between task modality and task load. Robustness of peak pupil latency, accuracy, and NASA-TLX was highlighted across verbal and visual modalities, while peak pupil diameter showed a weaker impact with differences between modalities. The findings encourage multivariate assessment of mental workload to account for different task modalities.

How flight experience impacts pilots' decision-making and visual scanning pattern in low-visibility approaches: preliminary evidence from eye tracking

The visual approach is the most accident-prone phase of a flight, especially in low-visibility conditions. This preliminary study aimed to examine the effects of flight experience on pilots' decision-making and visual scanning pattern in low-visibility approaches. Twenty pilots were separated into two groups based on their flight experience and completed the high- and low-visibility approaches in balanced order using a high-fidelity flight simulator. Pilots' mental workload and visual scanning patterns were recorded via an eye tracker. The results showed that, compared to less flight-experienced pilots (20%, 3/15), experienced pilots (80%, 4/5) were more likely to make go-around decisions in the low-visibility approaches. Furthermore, they exhibited a more flexible and adaptable visual scanning pattern by quickly shifting their attention, as evidenced by decreased fixations and increased saccades. These findings suggest that the integration of visual scanning strategy and training solution with a marginally meteorological approach may enhance decision-making safety for novice pilots.

o-CLEAN: a novel multi-stage algorithm for the ocular artifacts' correction from EEG data in out-of-the-lab applications

In the context of electroencephalographic (EEG) signal processing, artifacts generated by ocular movements, such as blinks, are significant confounding factors. These artifacts overwhelm informative EEG features and may occur too frequently to simply remove affected epochs without losing valuable data. Correcting these artifacts remains a challenge, particularly in out-of-lab and online applications using wearable EEG systems (i.e. with low number of EEG channels, without any additional channels to track EOG).Objective.The main objective of the present work consisted in validating a novel ocular blinks artefacts correction method, named multi-stage OCuLar artEfActs deNoising algorithm (o-CLEAN), suitable for online processing with minimal EEG channels.Approach.The research was conducted considering one EEG dataset collected in highly controlled environment, and a second one collected in real environment. The analysis was performed by comparing the o-CLEAN method with previously validated state-of-art techniques, and by evaluating its performance along two dimensions: (a) the ocular artefacts correction performance (IN-Blink), and (b) the EEG signal preservation when the method was applied without any ocular artefacts occurrence (OUT-Blink).Main results.Results highlighted that (i) o-CLEAN algorithm resulted to be, at least, significantly reliable as the most validated approaches identified in scientific literature in terms of ocular blink artifacts correction, (ii) o-CLEAN showed the best performances in terms of EEG signal preservation especially with a low number of EEG channels.Significance.The testing and validation of the o-CLEAN addresses a relevant open issue in bioengineering EEG processing, especially within out-of-the-lab application. In fact, the method offers an effective solution for correcting ocular artifacts in EEG signals with a low number of available channels, for online processing, and without any specific template of the EOG. It was demonstrated to be particularly effective for EEG data gathered in real environments using wearable systems, a rapidly expanding area within applied neuroscience.

Clarifying the effects of sequential item presentation in the police lineup task

Previous research has reported diverging patterns of results with respect to discriminability and response bias when comparing the simultaneous lineup to two different lineup procedures in which items are presented sequentially, the sequential stopping rule lineup and the UK lineup. In a single large sample experiment, we compared discriminability and response bias in six-item photographic lineups presented either simultaneously, sequentially with a stopping rule, or sequentially requiring two full laps through the items before making an identification and including the ability to revisit items, analogous to the UK lineup procedure. Discriminability was greater for the simultaneous lineup compared to the sequential stopping rule lineup, despite a non-significant difference in empirical discriminability between the procedures. There was no significant difference in discriminability when comparing the simultaneous lineup to the sequential two lineup and the sequential two lap lineup to the sequential stopping rule lineup. Responding was most lenient for the sequential two lap lineup, followed by the simultaneous lineup, followed by the sequential lineup. These results imply that sequential item presentation may not exert a large effect in isolation on discriminability and response bias. Rather, discriminability and response bias in the sequential stopping rule lineup and UK lineup result from the interaction of sequential item presentation with other aspects of these procedures.

Designing a practical fatigue detection system: A review on recent developments and challenges

INTRODUCTION

Fatigue is considered to have a life-threatening effect on human health and it has been an active field of research in different sectors. Deploying wearable physiological sensors helps to detect the level of fatigue objectively without any concern of bias in subjective assessment and interfering with work.

METHODS

This paper provides an in-depth review of fatigue detection approaches using physiological signals to pinpoint their main achievements, identify research gaps, and recommend avenues for future research. The review results are presented under three headings, including: signal modality, experimental environments, and fatigue detection models. Fatigue detection studies are first divided based on signal modality into uni-modal and multi-modal approaches. Then, the experimental environments utilized for fatigue data collection are critically analyzed. At the end, the machine learning models used for the classification of fatigue state are reviewed.

PRACTICAL APPLICATIONS

The directions for future research are provided based on critical analysis of past studies. Finally, the challenges of objective fatigue detection in the real-world scenario are discussed.

Evaluation of physiological responses to mental workload in n-back and arithmetic tasks

Working memory tasks, such as n-back and arithmetic tasks, are frequently used in studying mental workload. The present study investigated and compared the sensitivity of several physiological measures at three levels of difficulty of n-back and arithmetic tasks. The results showed significant differences in fixation duration and pupil diameter among three task difficulty levels for both n-back and arithmetic tasks. Pupil diameters increase with increasing mental workload, whereas fixation duration decreases. Blink duration and heart rate (HR) were significantly increased as task difficulty increased in the n-back task, while root mean square of successive differences (RMSSD) and standard deviation of R-R intervals (SDNN) were significantly decreased in the arithmetic task. On the other hand, blink rate and Galvanic Skin Response (GSR) were not sensitive enough to assess the differences in task difficulty for both tasks. All significant physiological measures yielded significant differences between low and high task difficulty except for SDNN.Practitioner summary: This study aimed to assess the sensitivity levels of several physiological measures of mental workload in n-back and arithmetic tasks. It showed that pupil diameter was the most sensitive in both tasks. This study also found that most physiological indices are sensitive to an extreme change in task difficulty levels.

Unraveling the interplay between mental workload, occupational fatigue, physiological responses and cognitive performance in office workers

Recently, cognitive demands in workplaces have surged significantly. This study explored the intricate relationship among mental workload (MWL), occupational fatigue, physiological responses, and cognitive performance in office workers by using collective semi-parametric models. One hundred office workers were selected from twenty offices involved in cognitive performance. MWL was assessed through the NASA Task Load Index (NASA-TLX), and occupational fatigue was measured using the Persian version of the Swedish Occupational Fatigue Inventory. Physiological responses, including respiratory rate, the electrical conductivity of the skin (ECS), Heart Rate (HR), and other heart-related parameters, were recorded from the participants during a work shift. Selective and Divided Attention tests were chosen to evaluate workers' cognitive function based on cognitive task analysis. The mean of MWL and occupational fatigue scores were 66.28 ± 11.76 and 1.62 ± 1.07, respectively. There was a significant moderate correlation between two dimensions, mental demand (0.429) and frustration (0.409), with functional fatigue. Also, Significant and, of course, nonlinear relationships were observed between MWL and HR (R2 = 0.44, P-value < 0.001) and ECS (R2 = 0.45, P-value < 0.001) and reaction time in selected (R2 = 0.34, P-value < 0.001) and divided test (R2 = 0.48, P-value < 0.001). Similarly, nonlinear relationships were observed between physiological responses and cognitive performance with fatigue among participants who had experienced higher levels of occupational fatigue. The MWL and fatigue seem to have a significant and non-linear effect on physiological parameters such as HR and ECS and cognitive parameters such as reaction time. Moreover, MWL can influence the dimension of functional fatigue of workers.

Combined Effects of Moderate Hypoxia and Sleep Restriction on Mental Workload

Aircraft pilots face a high mental workload (MW) under environmental constraints induced by high altitude and sometimes sleep restriction (SR). Our aim was to assess the combined effects of hypoxia and sleep restriction on cognitive and physiological responses to different MW levels using the Multi-Attribute Test Battery (MATB)-II with an additional auditory Oddball-like task. Seventeen healthy subjects were subjected in random order to three 12-min periods of increased MW level (low, medium, and high): sleep restriction (SR, <3 h of total sleep time (TST)) vs. habitual sleep (HS, >6 h TST), hypoxia (HY, 2 h, FIO2 = 13.6%, ~3500 m vs. normoxia, NO, FIO2 = 21%). Following each MW level, participants completed the NASA-TLX subjective MW scale. Increasing MW decreases performance on the MATB-II Tracking task (p = 0.001, MW difficulty main effect) and increases NASA-TLX (p = 0.001). In the combined HY/SR condition, MATB-II performance was lower, and the NASA-TLX score was higher compared with the NO/HS condition, while no effect of hypoxia alone was observed. In the accuracy of the auditory task, there is a significant interaction between hypoxia and MW difficulty (F(2-176) = 3.14, p = 0.04), with lower values at high MW under hypoxic conditions. Breathing rate, pupil size, and amplitude of pupil dilation response (PDR) to auditory stimuli are associated with increased MW. These parameters are the best predictors of increased MW, independently of physiological constraints. Adding ECG, SpO2, or electrodermal conductance does not improve model performance. In conclusion, hypoxia and sleep restriction have an additive effect on MW. Physiological and electrophysiological responses must be taken into account when designing a MW predictive model and cross-validation.

Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method Study

Electrodermal Activity (EDA), which primarily indicates arousal through sympathetic nervous system activity, serves as a tool to measure constructs like engagement, cognitive load, performance, and stress. Despite its potential, empirical studies have often yielded mixed results and found it of limited use. To better understand EDA, we conducted a mixed-methods study in which quantitative EDA profiles and survey data were investigated using qualitative interviews. This study furnishes an EDA dataset measuring the engagement levels of seven participants who watched three videos for 4-10 min. The subsequent interviews revealed five EDA morphologies with varying short-term signatures and long-term trends. We used this dataset to demonstrate the moving average crossover, a novel metric for EDA analysis, in predicting engagement-disengagement dynamics in such data. Our contributions include the creation of the detailed dataset, comprising EDA profiles annotated with qualitative data, the identification of five distinct EDA morphologies, and the proposition of the moving average crossover as an indicator of the beginning of engagement or disengagement in an individual.

Heart rate response to cognitive load as a marker of depression and increased anxiety

Introduction

Understanding the interplay between cardiovascular parameters, cognitive stress induced by increasing load, and mental well-being is vital for the development of integrated health strategies today. By monitoring physiological signals like electrocardiogram (ECG) and photoplethysmogram (PPG) in real time, researchers can discover how cognitive tasks influence both cardiovascular and mental health. Cardiac biomarkers resulting from cognitive strain act as indicators of autonomic nervous system function, potentially reflecting conditions related to heart and mental health, including depression and anxiety. The purpose of this study is to investigate how cognitive load affects ECG and PPG measurements and whether these can signal early cardiovascular changes during depression and anxiety disorders.

Methods

Ninety participants aged 18 to 45 years, ranging from symptom-free individuals to those with diverse psychological conditions, were assessed using psychological questionnaires and anamnesis. ECG and PPG monitoring were conducted as volunteers engaged in a cognitive 1-back task consisting of two separate blocks, each with six progressively challenging levels. The participants' responses were analyzed to correlate physiological and psychological data with cognitive stressors and outcomes.

Results

The study confirmed a notable interdependence between anxiety and depression, and cardiovascular responses. Task accuracy decreased with increased task difficulty. A strong relationship between PPG-measured heart rate and markers of depression and trait anxiety was observed. Increasing task difficulty corresponded to an increase in heart rate, linked with elevated levels of depression and trait anxiety. A strong relationship between ECG-measured heart rate and anxiety attacks was observed. Increasing task difficulty corresponded to an increase in heart rate, linked with elevated levels of anxiety attacks, although this association decreased under more challenging conditions.

Discussion

The findings underscore the predictive importance of ECG and PPG heart rate parameters in mental health assessment, particularly depression and anxiety under cognitive stress induced by increasing load. We discuss mechanisms of sympathetic activation explaining these differences. Our research outcomes have implications for clinical assessments and wearable device algorithms for more precise, personalized mental health diagnostics.

Speed behaviour and mental workload of small-spacing expressway interchanges based on field driving test

Many small-spacing interchanges (SSI) appear with the improvement of the expressway network. To investigate the speed and mental workload characteristics in the SSI and acquire the mechanism of the influence of speed on the drivers' workload, 37 participants were recruited to perform a field driving test. Each driver performed four driving conditions (i.e. ramp-mainline, mainline-ramp, mainline driving, and auxiliary lane driving). The speed and drivers' electrocardiogram (ECG) data were collected using SpeedBox speed acquisition equipment and PhysioLAB physiological instrument. The heart rate increase (HRI) index was used to analyse the drivers' mental workload regularity. The relationship model between speed and HRI was developed to examine the impact of speed on HRI. The results show that the speed variation in the SSI displayed two patterns: 'decrease - increase and continuous decrease.' The drivers' HRI variation presented four patterns: 'convex curve, continuously increasing, continuously decreasing and concave curve'. SSI's influenced area length is given based on the speed and HRI variation regularity. HRI is significantly higher when driving in the ramp-mainline condition in the SSI than when driving in other conditions, indicating that drivers are more nervous when merging with the mainline traffic. HRI increases significantly in the first 50% of the weaving area in four driving conditions, indicating that vehicle weaving greatly influences the drivers' mental workload. A positive correlation exists between vehicle speed and drivers' HRI without interference from other vehicles and road alignment.

Effects of hearing acuity on psychophysiological responses to effortful speech perception

In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37-72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.

The role of visual conditions and aircraft type on different aspects of pilot workload

OBJECTIVE

The objective of our work was to assess the impact of flight conditions by aircraft type on the workload estimated using NASA-Task Load Index (NASA-TLX).

BACKGROUND

Learning about subjective workload is important for assessing the impact of a pilot's work environment on their performance in the cockpit. This is an important element of flight safety and includes the prevention of aviation accidents.

METHODS

The study included 146 military pilots that fly the following aircrafts: flying fast-jet (21), fixed-wing (24), and rotary-wing (101). The NASA-TLX questionnaire was used to assess workload and pilots were asked to determine the level of workload resulting from flying under the following conditions: daytime flight (VFR), night-vision flight performed under Night Visual Flight Rules (NVFR), and night-vision flight using night-vision goggles (NVGs).

RESULTS

The highest level of workload was consistently attributed to flights performed under NVG conditions. NVFR conditions were rated as the most burdensome, while VFR conditions were rated as the least burdensome. Fast-jet pilots rated their mental performance and effort workload as significantly higher than pilots of other aircrafts.

CONCLUSION

Pilots' perceived workload is influenced by both flight conditions and the type of aircraft they fly. Workload knowledge is important for flight safety and should be taken into account during training and flight-task planning.

APPLICATION

The results of our study can be useful both in flight training and in work on the effectiveness of the human-machine interface. Awareness of one's own limitations due to the work environment can help improve flight safety.

Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review

Measuring pilot mental workload (MWL) is crucial for enhancing aviation safety. However, MWL is a multi-dimensional construct that could be affected by multiple factors. Particularly, in the context of a more automated cockpit setting, the traditional methods of assessing pilot MWL may face challenges. Heart rate variability (HRV) has emerged as a potential tool for detecting pilot MWL during real-flight operations. This review aims to investigate the relationship between HRV and pilot MWL and to assess the performance of machine-learning-based MWL detection systems using HRV parameters. A total of 29 relevant papers were extracted from three databases for review based on rigorous eligibility criteria. We observed significant variability across the reviewed studies, including study designs and measurement methods, as well as machine-learning techniques. Inconsistent results were observed regarding the differences in HRV measures between pilots under varying levels of MWL. Furthermore, for studies that developed HRV-based MWL detection systems, we examined the diverse model settings and discovered that several advanced techniques could be used to address specific challenges. This review serves as a practical guide for researchers and practitioners who are interested in employing HRV indicators for evaluating MWL and wish to incorporate cutting-edge techniques into their MWL measurement approaches.

Measuring cognitive workload in the nuclear control room: a review

Despite the substantial literature and human factors guidance, evaluators report challenges in selecting cognitive workload measures for the evaluation of complex human-technology systems. A review of 32 articles found that self-report measures and secondary tasks were systematically sensitive to human-system interface conditions and correlated with physiological measures. Therefore, including a self-report measure of cognitive workload is recommended when evaluating human-system interfaces. Physiological measures were mainly used in method studies, and future research must demonstrate the utility of these measures for human-system evaluation in complex work settings. However, indexes of physiological measures showed promise for cognitive workload assessment. The review revealed a limited focus on the measurement of excessive cognitive workload, although this is a key topic in nuclear process control. To support human-system evaluation of adequate cognitive workload, future research on behavioural measures may be useful in the identification and analysis of underload and overload.

An extension of the shifted Wald model of human response times: Capturing the time dynamic properties of human cognition : Trial-varying Wald model

Despite the ubiquitous nature of evidence accumulation models in cognitive and experimental psychology, there has been a comparatively limited uptake of such techniques in the applied literature. While quantifying latent cognitive processing properties has significant potential for applied domains such as adaptive work systems, accumulator models often fall short in practical applications. Two primary reasons for these shortcomings are the complexities and time needed for the application of cognitive models, and the failure of current models to capture systematic trial-to-trial variability in parameters. In this manuscript, we develop a novel, trial-varying extension of the shifted Wald model to address these concerns. By leveraging conjugate properties of the Wald distribution, we derive computationally efficient solutions for threshold and drift parameters which can be updated instantaneously with new data. The resulting model allows the quantification of systematic variation in latent cognitive parameters across trials and we demonstrate the utility of such analyses through simulations and an exemplar application to an existing data set. The analytic nature of our solutions opens the door for real-world applications, significantly extending the reach of computational models of behavioral responses.

Physiological measures of operators' mental state in supervisory process control tasks: a scoping review

Physiological measures are often used to assess the mental state of human operators in supervisory process control tasks. However, the diversity of research approaches creates a heterogeneous landscape of empirical evidence. To map existing evidence and provide guidance to researchers and practitioners, this paper systematically reviews 109 empirical studies that report relationships between peripheral nervous system measures and mental state dimensions (e.g. mental workload, mental fatigue, stress, and vigilance) of interest. Ocular and electrocardiac measures were the most prominent measures across application fields. Most studies sought to validate such measures for reliable assessments of cognitive task demands and time on task, with measures of pupil size receiving the most empirical support. In comparison, less research examined the utility of physiological measures in predicting human task performance. This approach is discussed as an opportunity to focus on operators' individual response to cognitive task demands and to advance the state of research.

Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room

OBJECTIVE

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance.

BACKGROUND

Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons.

METHODS

A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context.

RESULTS

From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making.

CONCLUSIONS

The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

Influence of the time-task constraint on ocular metrics of semi-elite soccer players

This study novelty aimed to investigate the influence of manipulating the available time to perform the training tasks on soccer players´ ocular metrics, following training. Specifically, pupillary response (pupil diameter) and saccadic features (latency, accuracy, velocity, and number) were measured with a portable eye tracker following training to reflect the mental load accumulated by players during the training sessions. Nine semi-elite soccer players performed two training sessions, based on large-sided games, on an artificial grass field. These two sessions were composed of the same tasks but varying the required time to complete the task goals (Session 1: No time limitations to perform the tasks; Session 2: Limited time to perform the tasks). The participants performed, before (pre-test) and after (post-test) each training session, a prosaccade task in a room near the playing field. Findings revealed a differentiated effect of the available time to complete the training tasks on ocular metrics because significant differences were found in all variables after training (p < .001 for pupil diameter; p < .01 for saccade accuracy and number of saccades; p < .05 for saccade velocity and latency). Ocular metrics could be a promising tool to evaluate mental load following practice because they were sensitive to the time-task constraint, providing researchers a valuable information for a better planning of the mental workload when designed training tasks.

Comparison of two pupillometric measures as indicators of cognitive strain and under the influence of screen luminance

The pupil diameter has been shown to provide insight to a person's experienced cognitive strain. Pupillary light responses, however, make this measure unreliable in uncontrolled settings. Two derived indicators-Index of Cognitive Activity (ICA) and Index of Pupillary Activity (IPA)-aim to 'eliminate' lighting influences, changing based only on the perceived cognitive strain. The IPA potentially offers a valuable alternative to the ICA through its fully transparent calculation, which lifts the restrictions to proprietary software and supported eye trackers. The measures are examined and compared based on two experimental studies; (i) as indicators of cognitive strain during mental arithmetic tasks and (ii) under different conditions of computer screen luminance. Results indicate that neither indicator differentiates between the increasing levels of cognitive strain. Differences in screen luminance are reflected in both indicators, although differently between the conditions. Both results contradict the claims of the indicators and further investigations are thus required.

Return-to-Manual Performance can be Predicted Before Automation Fails

OBJECTIVE

This study aimed to examine operator state variables (workload, fatigue, and trust in automation) that may predict return-to-manual (RTM) performance when automation fails in simulated air traffic control.

BACKGROUND

Prior research has largely focused on triggering adaptive automation based on reactive indicators of performance degradation or operator strain. A more direct and effective approach may be to proactively engage/disengage automation based on predicted operator RTM performance (conflict detection accuracy and response time), which requires analyses of within-person effects.

METHOD

Participants accepted and handed-off aircraft from their sector and were assisted by imperfect conflict detection/resolution automation. To avoid aircraft conflicts, participants were required to intervene when automation failed to detect a conflict. Participants periodically rated their workload, fatigue and trust in automation.

RESULTS

For participants with the same or higher average trust than the sample average, an increase in their trust (relative to their own average) slowed their subsequent RTM response time. For participants with lower average fatigue than the sample average, an increase in their fatigue (relative to own average) improved their subsequent RTM response time. There was no effect of workload on RTM performance.

CONCLUSIONS

RTM performance degraded as trust in automation increased relative to participants' own average, but only for individuals with average or high levels of trust.

APPLICATIONS

Study outcomes indicate a potential for future adaptive automation systems to detect vulnerable operator states in order to predict subsequent RTM performance decrements.

Evidence of compensatory neural hyperactivity in a subgroup of chemotherapy-treated breast cancer survivors and its association with brain aging

Chemotherapy-related cognitive impairment (CRCI) remains poorly understood in terms of the mechanisms of cognitive decline. Neural hyperactivity has been reported on average in cancer survivors, but it is unclear which patients demonstrate this neurophenotype, limiting precision medicine in this population. We evaluated a retrospective sample of 80 breast cancer survivors and 80 non-cancer controls, age 35-73, for which we had previously identified and validated three data-driven, biological subgroups (biotypes) of CRCI. We measured neural activity using the z-normalized percent amplitude of fluctuation from resting state functional magnetic resonance imaging (MRI). We tested established, quantitative criteria to determine if hyperactivity can accurately be considered compensatory. We also calculated brain age gap by applying a previously validated algorithm to anatomic MRI. We found that neural activity differed across the three CRCI biotypes and controls (F = 13.5, p < 0.001), with Biotype 2 demonstrating significant hyperactivity compared to the other groups (p < 0.004, corrected), primarily in prefrontal regions. Alternatively, Biotypes 1 and 3 demonstrated significant hypoactivity (p < 0.02, corrected). Hyperactivity in Biotype 2 met several of the criteria to be considered compensatory. However, we also found a positive relationship between neural activity and brain age gap in these patients (r = 0.45, p = 0.042). Our results indicated that neural hyperactivity is specific to a subgroup of breast cancer survivors and, while it seems to support preserved cognitive function, it could also increase the risk of accelerated brain aging. These findings could inform future neuromodulatory interventions with respect to the risks and benefits of up or downregulation of neural activity.

Mental workload assessment by monitoring brain, heart, and eye with six biomedical modalities during six cognitive tasks

Introduction

The efficiency and safety of complex high precision human-machine systems such as in aerospace and robotic surgery are closely related to the cognitive readiness, ability to manage workload, and situational awareness of their operators. Accurate assessment of mental workload could help in preventing operator error and allow for pertinent intervention by predicting performance declines that can arise from either work overload or under stimulation. Neuroergonomic approaches based on measures of human body and brain activity collectively can provide sensitive and reliable assessment of human mental workload in complex training and work environments.

Methods

In this study, we developed a new six-cognitive-domain task protocol, coupling it with six biomedical monitoring modalities to concurrently capture performance and cognitive workload correlates across a longitudinal multi-day investigation. Utilizing two distinct modalities for each aspect of cardiac activity (ECG and PPG), ocular activity (EOG and eye-tracking), and brain activity (EEG and fNIRS), 23 participants engaged in four sessions over 4 weeks, performing tasks associated with working memory, vigilance, risk assessment, shifting attention, situation awareness, and inhibitory control.

Results

The results revealed varying levels of sensitivity to workload within each modality. While certain measures exhibited consistency across tasks, neuroimaging modalities, in particular, unveiled meaningful differences between task conditions and cognitive domains.

Discussion

This is the first comprehensive comparison of these six brain-body measures across multiple days and cognitive domains. The findings underscore the potential of wearable brain and body sensing methods for evaluating mental workload. Such comprehensive neuroergonomic assessment can inform development of next generation neuroadaptive interfaces and training approaches for more efficient human-machine interaction and operator skill acquisition.