Measuring mental workload using physiological measures: A systematic review

Recognition and evaluation of mental workload in different stages of perceptual and cognitive information processing using a multimodal approach

This study explores the effects of different perceptual and cognitive information processing stages on mental workload by assessing multimodal indicators of mental workload such as the NASA-TLX, task performance, ERPs and eye movements. Repeated measures ANOVA of the data showed that among ERP indicators, P1, N1 and N2 amplitudes were sensitive to perceptual load (P-load), P3 amplitude was sensitive to P-load only in the prefrontal region during high cognitive load (C-load) states, and P3 amplitude in the occipital and parietal regions was sensitive to C-load. Among the eye movement indicators, blink frequency was sensitive to P-load in all C-load states, but to C-load in only low P-load states; pupil diameter and blink duration were sensitive to both P-load and C-load. Based on the above indicators, the k-nearest neighbours (KNN) algorithm was used to propose a classification method for the four different mental workload states with an accuracy of 97.89%.Practitioner summary: Based on the results of this study, it is possible to implement the monitoring of mental workload states and optimise brain task allocation in operations involving high mental workload, such as human-computer interaction.

Granular estimation of user cognitive workload using multi-modal physiological sensors

Mental workload (MWL) is a crucial area of study due to its significant influence on task performance and potential for significant operator error. However, measuring MWL presents challenges, as it is a multi-dimensional construct. Previous research on MWL models has focused on differentiating between two to three levels. Nonetheless, tasks can vary widely in their complexity, and little is known about how subtle variations in task difficulty influence workload indicators. To address this, we conducted an experiment inducing MWL in up to 5 levels, hypothesizing that our multi-modal metrics would be able to distinguish between each MWL stage. We measured the induced workload using task performance, subjective assessment, and physiological metrics. Our simulated task was designed to induce diverse MWL degrees, including five different math and three different verbal tiers. Our findings indicate that all investigated metrics successfully differentiated between various MWL levels induced by different tiers of math problems. Notably, performance metrics emerged as the most effective assessment, being the only metric capable of distinguishing all the levels. Some limitations were observed in the granularity of subjective and physiological metrics. Specifically, the subjective overall mental workload couldn't distinguish lower levels of workload, while all physiological metrics could detect a shift from lower to higher levels, but did not distinguish between workload tiers at the higher or lower ends of the scale (e.g., between the easy and the easy-medium tiers). Despite these limitations, each pair of levels was effectively differentiated by one or more metrics. This suggests a promising avenue for future research, exploring the integration or combination of multiple metrics. The findings suggest that subtle differences in workload levels may be distinguishable using combinations of subjective and physiological metrics.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments

BACKGROUND

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings.

METHODS

Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken.

RESULTS

Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05).

CONCLUSION

This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios

Assessing drivers' mental workload is crucial for reducing road accidents. This study examined drivers' mental workload in a simulated auditory-based dual-task driving scenario, with driving tasks as the main task, and auditory-based N-back tasks as the secondary task. A total of three levels of mental workload (i.e., low, medium, high) were manipulated by varying the difficulty levels of the secondary task (i.e., no presence of secondary task, 1-back, 2-back). Multimodal measures, including a set of subjective measures, physiological measures, and behavioral performance measures, were collected during the experiment. The results showed that an increase in task difficulty led to increased subjective ratings of mental workload and a decrease in task performance for the secondary N-back tasks. Significant differences were observed across the different levels of mental workload in multimodal physiological measures, such as delta waves in EEG signals, fixation distance in eye movement signals, time- and frequency-domain measures in ECG signals, and skin conductance in EDA signals. In addition, four driving performance measures related to vehicle velocity and the deviation of pedal input and vehicle position also showed sensitivity to the changes in drivers' mental workload. The findings from this study can contribute to a comprehensive understanding of effective measures for mental workload assessment in driving scenarios and to the development of smart driving systems for the accurate recognition of drivers' mental states.

The effects of increased mental workload of air traffic controllers on time perception: Behavioral and physiological evidence

Research has shown that timing is modulated by mental workload, making duration judgments a measure of cognitive demand, alongside subjective assessments, and physiological measurements. Yet, it is unclear whether such findings can be extended in less controlled setups. By employing air traffic controllers in a real aviation environment, we tested whether tasks with different levels of cognitive load can affect their timing behavior. Participants completed temporal production, verbal estimation, and passage of time judgments, while actively engaging in real flight control sessions. Subjective assessments of task demands, as well as physiological responses (cardiac and electrodermal activity) were also measured. Accuracy of the produced intervals was measured at two distinct phases of the flight (during low-load cruising vs. high-load landing) and under two different task load manipulations (controlling one vs. two helicopters and speaking in native vs. non-native language). Analysis of interval production accuracy showed that during the high-load landing phase significant overproductions were made, compared to the low-load cruising phase, and landing two helicopters led to greater overproductions compared to landing only one. The duration of the two-helicopter sessions was significantly overestimated compared to the single-helicopter ones, and the passage of time was felt significantly faster. Subjective assessments of workload were positively correlated with the temporal estimations and passage of time judgments, and skin responses were positively correlated with the produced intervals. Overall, our results are consistent with past research, suggesting that mental workload modulates time perception in complex, real-world environments, thus making timing behavior a reliable index of the workload changes.

Collaborative robots can augment human cognition in regret-sensitive tasks

Despite theoretical benefits of collaborative robots, disappointing outcomes are well documented by clinical studies, spanning rehabilitation, prostheses, and surgery. Cognitive load theory provides a possible explanation for why humans in the real world are not realizing the benefits of collaborative robots: high cognitive loads may be impeding human performance. Measuring cognitive availability using an electrocardiogram, we ask 25 participants to complete a virtual-reality task alongside an invisible agent that determines optimal performance by iteratively updating the Bellman equation. Three robots assist by providing environmental information relevant to task performance. By enabling the robots to act more autonomously-managing more of their own behavior with fewer instructions from the human-here we show that robots can augment participants' cognitive availability and decision-making. The way in which robots describe and achieve their objective can improve the human's cognitive ability to reason about the task and contribute to human-robot collaboration outcomes. Augmenting human cognition provides a path to improve the efficacy of collaborative robots. By demonstrating how robots can improve human cognition, this work paves the way for improving the cognitive capabilities of first responders, manufacturing workers, surgeons, and other future users of collaborative autonomy systems.

Home health care nurse routing and scheduling problem considering ergonomic risk factors

Home health care routing and scheduling is a complex problem that requires many aspects to be considered simultaneously. One of the important aspects is ergonomics. Home health care nurses are at higher risk of work-related health problems such as musculoskeletal disorders and burnout since they are frequently exposed to physical, mental, and environmental ergonomic risks in their jobs. Therefore, it is essential to integrate ergonomic considerations into the construction of daily schedules for home health care nurses to mitigate these health risks. The purpose of this study is to present a mathematical model that incorporate ergonomic risks. We introduce a set of constraints into our model to prevent nurses from encountering excessive workloads. To assess the workload, we propose a subjective assessment method and employ a fuzzy inference system to calculate nurses' perceived workload levels. We applied our model to a several numerical examples to investigate the impact of workload on the nurse daily schedules. We observed that, at a specified workload level, there may be alternative solutions where the number of patients visited is the same. Therefore, we defined an objective function to maximize patient visits while minimizing nurses' workload levels as much as possible. As a result, our model generates solutions that effectively reduce nurse workloads, leading to more balanced schedules. Thus, our study offers a comprehensive approach to home health care scheduling by incorporating ergonomic considerations, ultimately enhancing both patient care and nurse well-being.

Investigation of eye movement measures of mental workload in healthcare: Can pupil dilations reflect fatigue or overload when it comes to health information system use?

The use of health information systems (HIS) can result in high workloads and, consequently, poor performance characterized by e.g. increased occurrence of errors among clinicians. Pupillometry offers a good possibility to measure mental workload in a dynamic work setting. Currently, there is a lack of empirical research in the context of healthcare settings. Therefore, the aim of the present study was to examine whether specific eye movement measures are suitable for measuring mental workload in the healthcare setting, especially when working with HIS. 49 persons participated in our simulation-lab study. They had to complete a system-related task as well as an increasing n-back task. Both tasks were modified regarding task difficulty. Results show significant differences for objective and subjective workload measures between increasing task levels. There are also hints for an overload/fatigue indicator in pupil data. Our results are limited in terms of external validity, causality and effects. Future work should focus on high-fidelity simulations and less time-consuming analysis approaches.

Change in heart rate variability with increasing time-on-task as a marker for mental fatigue: A systematic review

Fatigue-specific changes in the autonomic nervous system are often assumed to underlie the development of mental fatigue caused by prolonged cognitive tasks (i.e. Time-on-Task). Therefore, several previous studies have chosen to investigate the Time-on-Task related changes in heart rate variability (HRV). However, previous studies have used many different HRV indices, and their results often show inconsistencies. The present study, therefore, systematically reviewed previous empirical HRV studies with healthy individuals and in which mental fatigue is induced by prolonged cognitive tasks. Articles relevant to the objectives were systematically searched and selected by applying the PRISMA guidelines. We screened 360 records found on 4 databases and found that 19 studies were eligible for full review in accordance with the inclusion criteria. In general, all studies reviewed (with the exception of two studies) found significant changes in HRV with increasing Time-on-Task, suggesting that HRV is a reliable autonomic marker for Time-on-Task induced fatigue. The most conclusive HRV indices that showed a consistent Time-on-Task effect were the low frequency component of HRV and the time domain indices, particularly the root mean square of successive differences. Time-on-Task typically induced an increasing trend in both type of measures.

Sharing imagery and analysis tools in a simulated submarine control room

We examined the impact of sharing periscope imagery and analysis tools in eighteen five-member teams (Sonar, Periscope, 2xTrack Motion Analysts, Track Manager) who undertook simulated submarine patrol tasks. Compared to a baseline condition, sharing imagery to team members increased perceived individual workload, with no improvement to team performance (tactical picture accuracy). When both imagery and analysis tools were shared, perceived workload increased and tactical picture compilation was more accurate. Despite this improved tactical picture for the imagery and analysis tools condition, there was no advantage to mission completion (rendezvous/close contact detection) or situation awareness. In contrast to the increased subjective workload, individuals in teams provided with shared imagery (with or without tools) had a lower physiological response (heart rate, electrodermal) to task demands compared to the baseline condition. Sharing imagery and analysis tools likely benefited tactical picture compilation by enabling dynamic task redistribution and multiple streams of concurrent data analysis.

Advanced workstations and collaborative robots: exploiting eye-tracking and cardiac activity indices to unveil senior workers' mental workload in assembly tasks

Introduction: As a result of Industry 5.0's technological advancements, collaborative robots (cobots) have emerged as pivotal enablers for refining manufacturing processes while re-focusing on humans. However, the successful integration of these cutting-edge tools hinges on a better understanding of human factors when interacting with such new technologies, eventually fostering workers' trust and acceptance and promoting low-fatigue work. This study thus delves into the intricate dynamics of human-cobot interactions by adopting a human-centric view. Methods: With this intent, we targeted senior workers, who often contend with diminishing work capabilities, and we explored the nexus between various human factors and task outcomes during a joint assembly operation with a cobot on an ergonomic workstation. Exploiting a dual-task manipulation to increase the task demand, we measured performance, subjective perceptions, eye-tracking indices and cardiac activity during the task. Firstly, we provided an overview of the senior workers' perceptions regarding their shared work with the cobot, by measuring technology acceptance, perceived wellbeing, work experience, and the estimated social impact of this technology in the industrial sector. Secondly, we asked whether the considered human factors varied significantly under dual-tasking, thus responding to a higher mental load while working alongside the cobot. Finally, we explored the predictive power of the collected measurements over the number of errors committed at the work task and the participants' perceived workload. Results: The present findings demonstrated how senior workers exhibited strong acceptance and positive experiences with our advanced workstation and the cobot, even under higher mental strain. Besides, their task performance suffered increased errors and duration during dual-tasking, while the eye behavior partially reflected the increased mental demand. Some interesting outcomes were also gained about the predictive power of some of the collected indices over the number of errors committed at the assembly task, even though the same did not apply to predicting perceived workload levels. Discussion: Overall, the paper discusses possible applications of these results in the 5.0 manufacturing sector, emphasizing the importance of adopting a holistic human-centered approach to understand the human-cobot complex better.

Emotional experiences of service robots' anthropomorphic appearance: a multimodal measurement method

Anthropomorphic appearance is a key factor to affect users' attitudes and emotions. This research aimed to measure emotional experience caused by robots' anthropomorphic appearance with three levels - high, moderate, and low - using multimodal measurement. Fifty participants' physiological and eye-tracker data were recorded synchronously while they observed robot images that were displayed in random order. Afterward, the participants reported subjective emotional experiences and attitudes towards those robots. The results showed that the images of the moderately anthropomorphic service robots induced higher pleasure and arousal ratings, and yielded significantly larger pupil diameter and faster saccade velocity, than did the low or high robots. Moreover, participants' facial electromyography, skin conductance, and heart-rate responses were higher when observing moderately anthropomorphic service robots. An implication of the research is that service robots' appearance should be designed to be moderately anthropomorphic; too many human-like features or machine-like features may disturb users' positive emotions and attitudes.Practitioner Summary: This research aimed to measure emotional experience caused by three types of anthropomorphic service robots using a multimodal measurement experiment. The results showed that moderately anthropomorphic service robots evoked more positive emotion than high and low anthropomorphic robots. Too many human-like features or machine-like features may disturb users' positive emotions.

Development and validation of a task load index for process control room operators (PCRO-TLX)

Process control room operators (PCRO) perform a range of complex cognitive safety-critical tasks. The aim of this exploratory sequential mixed methods study was to develop an occupation specific tool to measure the task load of PCRO using NASA Task Load Index (TLX) methodology. Participants were 30 human factors experts and 146 PCRO at two refinery complexes in Iran. Dimensions were developed via a cognitive task analysis, a research review, and three expert panels. Six dimensions were identified: perceptual demand, performance, mental demand, time pressure, effort, and stress. Data from 120 PCRO confirmed that the developed PCRO-TLX has acceptable psychometric properties, and a comparison with the NASA-TLX confirmed that perceptual, not physical, demand was relevant for measuring workload in PCRO. There was a positive convergence of scores of the Subjective Workload Assessment Technique and the PCRO-TLX. This reliable tool (α = 0.83) is recommended for risk assessing the task load of PCRO.Practitioner summary: There are benefits of having a specific tool to measure task load in safety critical roles. Thus, we developed and validated an easy-to-use targeted tool, the PCRO-TLX, for process control room operatives. Timely use and response will assure optimal production alongside health and safety in an organisation.Abbreviations: PCRO: process control room operator; TLX: task load index; PCRO-TLX: process control room operator task load index; NASA-TLX: National Aeronautics and Space Administration task load index; SWAT: subjective workload assessment technique; DALI: driving activity load index; SURG-TLX: surgery task load index; SIM-TLX: virtual reality simulation task load index; VACP: visual, auditory, cognitive and psychomotor; CVI: content validity index; CVR: content validity ratio; RMSEA: root mean square of error approximation; GFI: goodness of fit index; AGFI: adjusted goodness of fit index; CFI: comparative fit index; ANOVA: analysis of variance; CI: confidence interval.

Methods and evaluation of physiological measurements with acoustic stimuli-a systematic review

Objective. The detection of psychological loads, such as stress reactions, is receiving greater attention and social interest, as stress can have long-term effects on health O'Connor, Thayer and Vedhara (2021Ann. Rev. Psychol.72, 663-688). Acoustic stimuli, especially noise, are investigated as triggering factors. The application of physiological measurements in the detection of psychological loads enables the recording of a further quantitative dimension that goes beyond purely perceptive questionnaires. Thus, unconscious reactions to acoustic stimuli can also be captured. The numerous physiological signals and possible experimental designs with acoustic stimuli may quickly lead to a challenging implementation of the study and an increased difficulty in reproduction or comparison between studies. An unsuitable experimental design or processing of the physiological data may result in conclusions about psychological loads that are not valid anymore.Approach. The systematic review according to the preferred reporting items for systematic reviews and meta-analysis standard presented here is therefore intended to provide guidance and a basis for further studies in this field. For this purpose, studies were identified in which the participants' short-term physiological responses to acoustic stimuli were investigated in the context of a listening test in a laboratory study.Main Results. A total of 37 studies met these criteria and data items were analysed in terms of the experimental design (studied psychological load, independent variables/acoustic stimuli, participants, playback, scenario/context, duration of test phases, questionnaires for perceptual comparison) and the physiological signals (measures, calculated features, systems, data processing methods, data analysis methods, results). The overviews show that stress is the most studied psychological load in response to acoustic stimuli. An ECG/PPG system and the measurement of skin conductance were most frequently used for the detection of psychological loads. A critical aspect is the numerous different methods of experimental design, which prevent comparability of the results. In the future, more standardized methods are needed to achieve more valid analyses of the effects of acoustic stimuli.

The levels and related factors of mental workload among nurses: A systematic review and meta-analysis

AIM

The aim was to determine the overall levels and related factors of mental workload assessed using the NASA-TLX tool among nurses.

BACKGROUND

Mental workload is a key element that affects nursing performance. However, there exists no review regarding mental workload assessed using the NASA-TLX tool, focusing on nurses.

DESIGN

A systematic review and meta-analysis.

DATA SOURCES

PubMed, MEDLINE, Web of Science, EMBASE, PsycINFO, Scopus, CINAHL, CNKI, CBM, Weipu and WanFang databases were searched from 1 January 1998 to 30 February 2022.

REVIEW METHODS

Following the PRISMA statement recommendations, review methods resulted in 31 quantitative studies retained for inclusion which were evaluated with the evaluation criteria for observational studies as recommended by the Agency for Healthcare Research and Quality. The data were pooled and a random-effects meta-analysis conducted.

RESULTS

Findings showed the pooled mental workload score was 65.24, and the pooled prevalence of high mental workload was 54%. Subgroup analysis indicated nurses in developing countries and emergency departments experienced higher mental workloads, and the mental workloads of front-line nurses increased significantly during the COVID-19 pandemic.

CONCLUSION

These findings highlight that nurses experience high mental workloads as assessed using the NASA-TLX tool and there is an urgent need to explore interventions to decrease their mental workloads.

Emerging Applications for Computer Vision and Artificial Intelligence in Management of the Cardiovascular Patient

Artificial intelligence and telemedicine promise to reshape patient care to an unprecedented extent, leading to a safer and more sustainable work environment and improved patient care. In this article, we summarize how these emerging technologies can be used in the care of cardiovascular patients in such ways as fall detection and prevention, virtual nursing, remote case support, automation of instrument counts in the operating room, and efficiency optimization in the cardiovascular suite.

Physiological Measurements of Situation Awareness: A Systematic Review

OBJECTIVE

The goal of this systematic literature review is to investigate the relationship between indirect physiological measurements and direct measures of situation awareness (SA).

BACKGROUND

Across different environments and tasks, assessments of SA are often performed using techniques designed specifically to directly measure SA, such as SAGAT, SPAM, and/or SART. However, research suggests that indirect physiological sensing methods may also be capable of predicting SA. Currently, it is unclear which particular physiological approaches are sensitive to changes in SA.

METHOD

Seven databases were searched using the PRISMA reporting guidelines. Eligibility criteria included human-subject experiments that used at least one direct SA assessment technique, as well as at least one physiological measurement. Information extracted from each article was the physiological metric(s), the direct SA measurement(s), the correlation between these two metrics, and the experimental task(s). All studies underwent a quality assessment.

RESULTS

Twenty-five articles were included in this review. Eye tracking techniques were the most commonly used physiological measures, and correlations between conscious aspects of eye movement measures and direct SA scores were observed. Evidence for cardiovascular predictors of SA were mixed. EEG studies were too few to form strong conclusions, but were consistently positive.

CONCLUSION

Further investigation is needed to methodically collect more relevant data and comprehensively model the relationships between a wider range of physiological measurements and direct assessments of SA.

APPLICATION

This review will guide researchers and practitioners in methods to indirectly assess SA with sensors and highlight opportunities for future research on wearables and SA.

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

Humans' performance varies due to the mental resources that are available to successfully pursue a task. To monitor users' current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications.

Subjective mental workload in Spanish emergency nurses. A study on predictive factors

INTRODUCTION

Mental workload refers to the cognitive or intellectual requirements that a worker is subjected to in a workday. The objective of the present work was to discover the subjective mental workload of nursing staff at Hospital Emergency Units, and its relationship with sociodemographic, work, environmental factors at the workplace, and personality variables.

METHOD

A quantitative, descriptive, observational, and crosssectional study was conducted with 201 emergency nurses from 13 different provinces in Spain. Each participant completed 5 questionnaires (sociodemographic, work conditions, environmental conditions, personality, and subjective mental workload). Descriptive statistics were obtained, and Pearson's correlations and multivariate models (multiple linear regression) were performed.

RESULTS

The nurses had medium to high levels of mental workload. The environmental conditions had a direct relationship with the mental workload, especially with respect to noise and lighting. The participants obtained high scores in kindness, responsibility, openness/intellect, and extraversion. Positive and statistically significant relations were found between neuroticism and mental workload. Being female, older, and having stable employment or a permanent contract were associated with a greater mental workload of emergency nurses.

CONCLUSION

The domain of neuroticism personality, and the hygienic conditions in the workplace were the predictors with the most weight in the model. This study could be useful for defining aspects that need to be considered for the well-being of emergency nurses, such as lighting conditions or environmental noise in the workplace. It also invites reflection on the influence of personal factors (age, gender, personality) and work factors (type of contract, professional experience) on the mental workload of emergency nurses.

Cognitive tasks and combined statistical methods to evaluate, model, and predict mental workload

Mental workload (MWL) is a concept that is used as a reference for assessing the mental cost of activities. In recent times, challenges related to user experience are determining the expected MWL value for a given activity and real-time adaptation of task complexity level to achieve or maintain desired MWL. As a consequence, it is important to have at least one task that can reliably predict the MWL level associated with a given complexity level. In this study, we used several cognitive tasks to meet this need, including the N-Back task, the commonly used reference test in the MWL literature, and the Corsi test. Tasks were adapted to generate different MWL classes measured via NASA-TLX and Workload Profile questionnaires. Our first objective was to identify which tasks had the most distinct MWL classes based on combined statistical methods. Our results indicated that the Corsi test satisfied our first objective, obtaining three distinct MWL classes associated with three complexity levels offering therefore a reliable model (about 80% accuracy) to predicted MWL classes. Our second objective was to achieve or maintain the desired MWL, which entailed the use of an algorithm to adapt the MWL class based on an accurate prediction model. This model needed to be based on an objective and real-time indicator of MWL. For this purpose, we identified different performance criteria for each task. The classification models obtained indicated that only the Corsi test would be a good candidate for this aim (more than 50% accuracy compared to a chance level of 33%) but performances were not sufficient to consider identifying and adapting the MWL class online with sufficient accuracy during a task. Thus, performance indicators require to be complemented by other types of measures like physiological ones. Our study also highlights the limitations of the N-back task in favor of the Corsi test which turned out to be the best candidate to model and predict the MWL among several cognitive tasks.

Mental workload variations during different cognitive office tasks with social media interruptions

Interruption at work by social media (SM) is a pervasive phenomenon. This study investigated the impact of SM interruptions and task cognitive levels on mental workload (MWL) and physiological indexes. Each subject performed six simulated computer tasks differentiated by two factors: task cognitive level and performing condition. MWL was reflected through three categories of data: perceived mental workload, physiological indexes, and primary task performance. The results revealed significant effects of SM interruptions on heart rate, low-frequency/high-frequency (LF/HF) ratio, and skin conductance. ANOVA results showed there were main effects of task cognitive level on LF/HF and skin conductance. These effects during interrupted tasks were more profound. In addition, participants experienced higher MWL and recorded lower primary task performance in the knowledge-based task than the rule- and skill-based tasks. Our findings can guide managers and employees regarding appropriate use of SM in the workplace and better managing interruption and workload.Practitioner Summary: Office workers suffer from increased overall mental workload due to unpredictable interruptions while working. This study shows that participants' mental workload increased when receiving SM interruptions, which was more profound during complex tasks. This highlights the importance of SM interruptions management for employees' health, performance, and mobile application developers.Abbreviations: ANOVA: analysis of variance; DSSQ: dundee stress state questionnaire; ECG: electrocardiographic; EDA: electrodermal activity; EEG: electroencephalographic; HPA: hypothalamus-pituitaryadrenocortical; HR: heart rate; HRV: heart rate variability; LF/HF: low frequency/high frequency; MSDs: musculoskeletal disorders; MWL: mental workload; NN: normal to normal; RMS: root means square; RR: time duration between two successive R peaks; RT: response time; SC: skin conductance; SDNN: standard deviation of normal to normal; SM: social media; TCL: task cognitive level; TPC: task performing condition; WMC: working memory capacity.

The effect of flight phase on electrodermal activity and gaze behavior: A simulator study

Current advances in airplane cockpit design and layout are often driven by a need to improve the pilot's awareness of the aircraft's state. This involves an improvement in the flow of information from aircraft to pilot. However, providing the aircraft with information on the pilot's state remains an open challenge. This work takes a first step towards determining the pilot's state based on biosensor data. We conducted a simulator study to record participants' electrodermal activity and gaze behavior, indicating pilot state changes during three distinct flight phases in an instrument failure scenario. The results show a significant difference in these psychophysiological measures between a phase of regular flight, the incident phase, and a phase with an additional troubleshooting task after the failure. The differences in the observed measures suggest great potential for a pilot-aware cockpit that can provide assistance based on the sensed pilot state.

Mathematical-heuristic modelling for human performance envelope

BACKGROUND: Using the theory of complex systems, some human functions (thinking, memory, language) and human relationships have been analyzed and explained. In order to study the limits of human performance (in Air Traffic Controllers and pilots) a new concept was created, called the Human Performance Envelope (HPE). OBJECTIVE: The aim of this paper is to apply the principles of the complex system to the analysis of the human factors of the HPE concept. Moreover, this paper’s objective is to create a mathematical model that will give the opportunity to study all the physiological ergonomic factors, not only the ones that are most commonly studied. The most studied factors are mental workload, stress and situation awareness (SA). By applying the mathematical model, it is possible to analyze all the physiological factors (stress, mental workload, fatigue, attention, vigilance and SA). METHODS: In the present paper the theory of complex systems (hybrid modelling) was applied to the Human Performance Envelope concept. A mathematical model was created, then it was validated and solved based on previous researches. RESULTS: Firstly, a literature analysis was performed on the complex systems application by the present researchers concerning pilots’ HPE. The proportional and inverse proportional relationships between the nine human factors were visually illustrated. Finally, a mathematical model was proposed, consisting of a set of equations, which were partially solved and validated by the experiments on pilots done by other researchers. CONCLUSIONS: Further research is required to validate the whole mathematical model, including physiological measurements (experiments) for the six ergonomic factors and the applied heuristic psychosocial methods for the others.

Characteristics and influencing factors of early childhood teachers’ work stress and burnout: A comparative study between China, Ghana, and Pakistan

Introduction

Many studies have documented the stress and burnout experienced by early childhood teachers. However, few have focused on comparisons among countries, particularly developing ones. Meanwhile, female teachers, who are more sensitive and tend to provide emotional responses, are often overlooked as a major force of emotional involvement. This study examined the similarities and differences of early childhood teachers’ stress, burnout, and gender in China, Ghana, and Pakistan.

Methods

This study adopted a cross-sectional design. The participants included 945 preschool and lower primary school teachers recruited from Zhejiang Province in China, the Ashanti Region in Ghana, and Punjab, Pakistan. The analyses were conducted using structural equation modeling. First, the study estimated all parameters separately and without constraints between the groups for all models. Second, the study compared the latent mean difference and of stressors and burnout between teachers’ personal and job characteristics. Third, a structural equation model was used to assess the relationship between teachers’ stressors and burnout.

Results

Across the three countries, female teachers are more stressed out, with higher emotional demands and work-family conflicts, and are more prone to burnout with a greater level of emotional exhaustion and a lower level of personal accomplishments than their male counterparts are. Moreover, Chinese teachers were found to be the most stressed-out group with the highest level of burnout. In comparison to teachers in China and Pakistan, early childhood teachers in Ghana possess the lowest level of emotional demands. With the lowest level of emotional exhaustion and the highest level of personal accomplishments, Pakistani teachers were unlikely to experience burnout.

Discussion

This study comparatively analyzed the characteristics of stress and burnout among ECTs in different cultural settings and educational systems in three developing countries (China, Ghana, and Pakistan), and revealed workplace characteristics and circumstances for ECTs. In addition, this study takes gender as the main influencing factor and explores its effect on ECTs’ stress and burnout, and it highlights and verifies "emotionality" in ECTs' profession. As a result, policymakers and stakeholders in different countries may be encouraged to improve ECE quality and the well-being of ECTs.

Workers' whole day workload and next day cognitive performance

Workload experienced over the whole day, not just work periods, may impact worker cognitive performance. We hypothesized that experiencing greater than typical whole day workload would be associated with lower visual processing speed and lower sustained attention ability, on the next day. To test this, we used dynamic structural equation modeling to analyze data from 56 workers with type 1 diabetes. For a two-week period, on smartphones they answered questions about whole day workload at the end of each day, and completed cognitive tests 5 or 6 times throughout each day. Repeated smartphone cognitive tests were used, instead of traditional one- time cognitive assessment in the laboratory, to increase the ecological validity of the cognitive tests. Examples of reported occupations in our sample included housekeeper, teacher, physician, and cashier. On workdays, the mean number of work hours reported was 6.58 (SD 3.5). At the within-person level, greater whole day workload predicted decreased mean processing speed the next day (standardized estimate=-0.10, 95% CI -0.18 to -0.01) using a random intercept model; the relationship was not significant and only demonstrated a tendency toward the expected effect (standardized estimate= -0.07, 95% CI -0.15 to 0.01) in a model with a random intercept and a random regression slope. Whole day workload was not found to be associated with next-day mean sustained attention ability. Study results suggested that just one day of greater than average workload could impact next day processing speed, but future studies with larger sample sizes are needed to corroborate this finding.

The physiology of intraoperative error: using electrokardiograms to understand operator performance during robot-assisted surgery simulations

BACKGROUND

No platform for objective, synchronous and on-line evaluation of both intraoperative error and surgeon physiology yet exists. Electrokardiogram (EKG) metrics have been associated with cognitive and affective features that are known to impact surgical performance but have not yet been analyzed in conjunction with real-time error signals using objective, real-time methods.

METHODS

EKGs and operating console point-of-views (POVs) for fifteen general surgery residents and five non-medically trained participants were captured during three simulated robotic-assisted surgery (RAS) procedures. Time and frequency-domain EKG statistics were extracted from recorded EKGs. Intraoperative errors were detected from operating console POV videos. EKG statistics were synchronized with intraoperative error signals.

RESULTS

Relative to personalized baselines, IBI, SDNN and RMSSD decreased 0.15% (S.E. 3.603e-04; P = 3.25e-05), 3.08% (S.E. 1.603e-03; P < 2e-16) and 1.19% (S.E. 2.631e-03; P = 5.66e-06), respectively, during error. Relative LF RMS power decreased 1.44% (S.E. 2.337e-03; P = 8.38e-10), and relative HF RMS power increased 5.51% (S.E. 1.945e-03; P < 2e-16).

CONCLUSIONS

Use of a novel, on-line biometric and operating room data capture and analysis platform enabled detection of distinct operator physiological changes during intraoperative errors. Monitoring operator EKG metrics during surgery may help improve patient outcomes through real-time assessments of intraoperative surgical proficiency and perceived difficulty as well as inform personalized surgical skills development.

Cognitive Load Management: An Invaluable Tool for Safe and Effective Surgical Training

This article highlights the importance of considering Cognitive Load (CL) and Cognitive Load Theory (CLT) during surgical training, focusing on the acquisition of intra-operative skills. It describes the basis of CLT with the overarching aim of describing CLT-based techniques to enhance current training strategies and surgical performance, many of which are instinctively already employed in surgical practice. Currently, methods of feedback and assessment are imperfect - typically subjective, unsystematic, opportunistic, or retrospective, and at risk of human bias. Surgical Sabermetrics, the advanced analytics of surgical and audio-visual data, aims to enhance this feedback by providing objective, real-time, digital-based feedback. This article introduces the benefit of real-time measurement of CL to enhance feedback and its applications to surgical performance that follow the ethos of Surgical Sabermetrics.¹ The 2022 theme for ICOSET was "Making it Better." Cognitive Load and Surgical Sabermetrics principles provide tools to make Surgical training better, with the goal of higher quality care for patients.

Cross-task cognitive workload recognition using a dynamic residual network with attention mechanism based on neurophysiological signals

BACKGROUND AND OBJECTIVE

Evaluation of human cognitive workload (CW) helps improve the user experience of human-centered systems. To provide a continuous estimation of the CW, we built a CW recognizer that maps human electroencephalograms (EEGs) to discrete CW levels with deep learning tools. However, the EEG distribution varies when humans perform different cognitive tasks. There is thus a question on the capacity for generalizing the CW recognizer across tasks. In this study, we examined the CW's performance when it was trained and tested on two EEG databases corresponding to different human-machine tasks.

METHODS

A novel deep neural network-based EEG recognizer, dynamic residual network with attention mechanism (DRNA-Net), is proposed in the present study. By taking advantage of recurrent networks, the DRNA-Net further incorporates a self-attention mechanism in discovering robust EEG patterns across different cognitive tasks.

RESULTS

We designed an experiment that applied a multidimensional N-back task to induce the CW that consists of visual and auditory memory tasks. We validated the cross-task generalization capability of the DRNA-Net based on the EEG features extracted from the N-back task and a public database. The results show that the DRNA-Net achieves classification accuracy and Macro-F1 values are 0.6055 and 0.6067, respectively.

CONCLUSIONS

The performance of the DRNA-Net indicates that it has a certain ability of cross-task cognitive workload classification, which outperforms several shallow learners and deep convolutional neural networks under various conditions of the feature subsets.

Using machine learning methods and EEG to discriminate aircraft pilot cognitive workload during flight

Pilots of aircraft face varying degrees of cognitive workload even during normal flight operations. Periods of low cognitive workload may be followed by periods of high cognitive workload and vice versa. During such changing demands, there exists potential for increased error on behalf of the pilots due to periods of boredom or excessive cognitive task demand. To further understand cognitive workload in aviation, the present study involved collection of electroencephalogram (EEG) data from ten (10) collegiate aviation students in a live-flight environment in a single-engine aircraft. Each pilot possessed a Federal Aviation Administration (FAA) commercial pilot certificate and either FAA class I or class II medical certificate. Each pilot flew a standardized flight profile representing an average instrument flight training sequence. For data analysis, we used four main sub-bands of the recorded EEG signals: delta, theta, alpha, and beta. Power spectral density (PSD) and log energy entropy of each sub-band across 20 electrodes were computed and subjected to two feature selection algorithms (recursive feature elimination (RFE) and lasso cross-validation (LassoCV), and a stacking ensemble machine learning algorithm composed of support vector machine, random forest, and logistic regression. Also, hyperparameter optimization and tenfold cross-validation were used to improve the model performance, reliability, and generalization. The feature selection step resulted in 15 features that can be considered an indicator of pilots' cognitive workload states. Then these features were applied to the stacking ensemble algorithm, and the highest results were achieved using the selected features by the RFE algorithm with an accuracy of 91.67% (± 0.11), a precision of 93.89% (± 0.09), recall of 91.67% (± 0.11), F-score of 91.22% (± 0.12), and the mean ROC-AUC of 0.93 (± 0.06). The achieved results indicated that the combination of PSD and log energy entropy, along with well-designed machine learning algorithms, suggest the potential for the use of EEG to discriminate periods of the low, medium, and high workload to augment aircraft system design, including flight automation features to improve aviation safety.

Protocol for a scoping review on 'surgical sabermetrics:' technology-enhanced measurement of operative non-technical skills

INTRODUCTION

Surgeons need high fidelity, high quality, objective, non-judgemental and quantitative feedback to measure their performance in order to optimise their performance and improve patient safety. This can be provided through surgical sabermetrics, defined as 'advanced analytics of digitally recorded surgical training and operative procedures to enhance insight, support professional development and optimise clinical and safety outcomes'. The aim of this scoping review is to investigate the assessment of surgeon's non-technical skills using sabermetrics principles, focusing on digital, automated measurements that do not require a human observer.

METHODS AND ANALYSIS

To investigate the current methods of digital, automated measurements of surgeons' non-technical skills, a systematic scoping review will be conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines, using databases from medicine and other fields. Covidence software is used for screening of potential studies. A data extraction tool will be developed specifically for this study to evaluate the methods of measurement. Quality assurance will be assessed using Quality Assessment Tool for Diverse Designs. Multiple reviewers will be responsible for screening of studies and data extraction.

ETHICS AND DISSEMINATION

This is a review study, not using primary data, and therefore, ethical approval is not required. A range of methods will be employed for dissemination of the results of this study, including publication in journals and conference presentations.

Domain- and task-analytic workload (DTAW) method: a methodology for predicting mental workload during severe accidents in nuclear power plants

Excessive mental workload reduces operators' performance and threatens the safety of nuclear power plants (NPPs) in severe accident management (SAM). Given the lack of suitable mental workload measurement methods for SAM tasks, we proposed a Domain- and Task-Analytic Workload (DTAW) method to predict SAM workload. The DTAW method is developed in three stages: scenario construction based on work domain analysis, task analysis, and workload estimation with eight workload components scored through task-analytic and projective methods. To demonstrate its utility, we applied the method to construct two SAM scenarios and predict the mental workload demand of operators in these scenarios as compared to two design basis accident scenarios. With statistical analysis, the DTAW method can predict the overall subjective workload rated by NPP operators, be used to identify high-load tasks, cluster tasks with similar workload patterns, and provide direct implications for improving SAM strategies and supporting systems.Practitioner summary: To predict mental workload in severe accident management (SAM) scenarios in nuclear power plants, we proposed an analytic method and applied it to estimate mental workload in two SAM scenarios and two design basis accident (DBA) scenarios. We found that the workload pattern in SAM scenarios is different from that in DBA scenarios.

Effect of time-delay on lunar sampling tele-operations: Evidences from cardiac, ocular and behavioral measures

The purpose of this study is to quantify performance in human-robot interaction under time-delay conditions in a lunar tele-operations sampling task, by testing the hypothesis that an increase of time-delay would lead to higher perceived workload and lower human performance in human-robotic integrated operations. Tele-operation is key in the exploration of the Moon, and allows for robotic elements to be controlled from orbital infrastructure and other planetary bodies such as the Earth. Considering that future missions aim to control rovers (amongst others for sampling tasks) from Earth (delay: 3s), the Gateway (delay: 0.5s) and the Moon (delay: 0s), control under the time-delay conditions for these locations must be studied. Time-delay can affect performance, and understanding the performance means that mission operations can be planned bottom-up, which benefits both the preparation of the crew and the design of rovers. An experiment was conducted with 18 engineers who were assigned to control a robotic arm under three time-delay conditions, representing the three control locations. Several metrics were derived from cardiac, ocular, subjective and behavioral measures. The analyses disclosed that the large time-delay condition statistically increased the perceived workload, the time to complete the mission and decreased heart rate variability compared to the other conditions. However, no effect of time-delay was found on attentional and executive abilities. The metrics proved to be effective in the study of performance quantification in human-robot interaction for tele-operations in lunar control scenarios. This approach can be implemented for a larger range of robotic activities, such as tele-operated driving.

Reliability of Mental Workload Index Assessed by EEG with Different Electrode Configurations and Signal Pre-Processing Pipelines

BACKGROUND AND OBJECTIVE

Mental workload (MWL) is a relevant construct involved in all cognitively demanding activities, and its assessment is an important goal in many research fields. This paper aims at evaluating the reproducibility and sensitivity of MWL assessment from EEG signals considering the effects of different electrode configurations and pre-processing pipelines (PPPs).

METHODS

Thirteen young healthy adults were enrolled and were asked to perform 45 min of Simon's task to elicit a cognitive demand. EEG data were collected using a 32-channel system with different electrode configurations (fronto-parietal; Fz and Pz; Cz) and analyzed using different PPPs, from the simplest bandpass filtering to the combination of filtering, Artifact Subspace Reconstruction (ASR) and Independent Component Analysis (ICA). The reproducibility of MWL indexes estimation and the sensitivity of their changes were assessed using Intraclass Correlation Coefficient and statistical analysis.

RESULTS

MWL assessed with different PPPs showed reliability ranging from good to very good in most of the electrode configurations (average consistency > 0.87 and average absolute agreement > 0.92). Larger fronto-parietal electrode configurations, albeit being more affected by the choice of PPPs, provide better sensitivity in the detection of MWL changes if compared to a single-electrode configuration (18 vs. 10 statistically significant differences detected, respectively).

CONCLUSIONS

The most complex PPPs have been proven to ensure good reliability (>0.90) and sensitivity in all experimental conditions. In conclusion, we propose to use at least a two-electrode configuration (Fz and Pz) and complex PPPs including at least the ICA algorithm (even better including ASR) to mitigate artifacts and obtain reliable and sensitive MWL assessment during cognitive tasks.

Towards a versatile mental workload modeling using neurometric indices

Researchers have been working to magnify mental workload (MWL) modeling for a long time. An important aspect of its modeling is feature selection as it interprets bulky and high-dimensional EEG data and enhances the accuracy of the classification model. In this study, a feature selection technique is proposed to obtain an optimized feature set with multiple domain features that can contribute to classifying the MWL at three distinct levels. The brain signals from thirteen healthy subjects were examined while they attended an intrinsic MWL of spotting differences in a set of similar pictures. The Recursive Feature Elimination (RFE) technique selects the robust features from the feature matrix by eliminating all the least contributing features. Along with the Support Vector Machine (SVM), the overall classification accuracy with the proposed RFE reached 0.913 from 0.791 surpassing the other techniques mentioned. The results of the study also significantly display the variation in the mean values of the selected features at the three workload levels (p<0.05). This model can become the principle for defining the workload level quantification applicable to diverse fields like neuroergonomics study, intelligent assistive devices (ADs) development, blue-chip technology exploration, cognitive evaluation of students, power plant operators, traffic operators, etc.

Age Differences in Learning-Related Neurophysiological Changes

Abstract. Research in young adults has demonstrated that neurophysiological measures are able to provide insight into learning processes. However, to date, it remains unclear whether neurophysiological changes during learning in older adults are comparable to those in younger adults. The current study addressed this issue by exploring age differences in changes over time in a range of neurophysiological outcome measures collected during visuomotor sequence learning. Specifically, measures of electroencephalography (EEG), skin conductance, heart rate, heart rate variability, respiration rate, and eye-related measures, in addition to behavioral performance measures, were collected in younger ( Mage = 27.24 years) and older adults ( Mage = 58.06 years) during learning. Behavioral responses became more accurate over time in both age groups during visuomotor sequence learning. Yet, older adults needed more time in each trial to enhance the precision of their movement. Changes in EEG during learning demonstrated a stronger increase in theta power in older compared to younger adults and a decrease in gamma power in older adults while increasing slightly in younger adults. No such differences between the two age groups were found on other neurophysiological outcome measures, suggesting changes in brain activity during learning to be more sensitive to age differences than changes in peripheral physiology. Additionally, differences in which neurophysiological outcomes were associated with behavioral performance on the learning task were found between younger and older adults. This indicates that the neurophysiological underpinnings of learning may differ between younger and older adults. Therefore, the current findings highlight the importance of taking age into account when aiming to gain insight into behavioral performance through neurophysiology during learning.

Staffs' physiological responses to irrelevant background speech and mental workload in open-plan bank office workspaces

BACKGROUND

Acoustic comfort is one of the most critical challenges in the open-plan workspace.

OBJECTIVE

This study was aimed to assess the effect of irrelevant background speech (IBS) and mental workload (MWL) on staffs' physiological parameters in open-plan bank office workspaces.

METHODS

In this study, 109 male cashier staff of the banks were randomly selected. The 30-minute equivalent noise level (LAeq) of the participants was measured in three intervals at the beginning (section A), middle (section B), and end of working hours (section C). The heart rate (HR) and heart rate variability (HRV): low frequency (LF), high frequency (HF), and LF/HF of the staff were also recorded in sections A, B, and C. Moreover, staff was asked to rate the MWL using the NASA-Task load.

RESULTS

The dominant frequency of the LAeq was 500 Hz, and the LAeq in the frequency range of 250 to 2000 was higher than other frequencies. The LAeq (500 Hz) was 55.82, 69.35, and 69.64 dB(A) in sections A, B, and C, respectively. The results show that the IBS affects staffs' physiological responses so that with increasing in IBS, the HF power decreases. Moreover, with higher MWL, increasing noise exposure, especially IBS, causes more increases in LF power and LF/HF ratio.

CONCLUSION

It seems that the IBS can affect physiological responses and increase staff stress in open-plan bank office workspaces. Moreover, the mental workload can intensify these consequences in these working settings.

Wearable electroencephalography and multi-modal mental state classification: A systematic literature review

BACKGROUND

Wearable multi-modal time-series classification applications outperform their best uni-modal counterparts and hold great promise. A modality that directly measures electrical correlates from the brain is electroencephalography. Due to varying noise sources, different key brain regions, key frequency bands, and signal characteristics like non-stationarity, techniques for data pre-processing and classification algorithms are task-dependent.

METHOD

Here, a systematic literature review on mental state classification for wearable electroencephalography is presented. Four search terms in different combinations were used for an in-title search. The search was executed on the 29th of June 2022, across Google Scholar, PubMed, IEEEXplore, and ScienceDirect. 76 most relevant publications were set into context as the current state-of-the-art in mental state time-series classification.

RESULTS

Pre-processing techniques, features, and time-series classification models were analyzed. Across publications, a window length of one second was mainly chosen for classification and spectral features were utilized the most. The achieved performance per time-series classification model is analyzed, finding linear discriminant analysis, decision trees, and k-nearest neighbors models outperform support-vector machines by a factor of up to 1.5. A historical analysis depicts future trends while under-reported aspects relevant to practical applications are discussed.

CONCLUSIONS

Five main conclusions are given, covering utilization of available area for electrode placement on the head, most often or scarcely utilized features and time-series classification model architectures, baseline reporting practices, as well as explainability and interpretability of Deep Learning. The importance of a 'test battery' assessing the influence of data pre-processing and multi-modality on time-series classification performance is emphasized.

Intraocular pressure responses to a virtual reality shooting simulation in active-duty members of the Spanish Army: The influence of task complexity

Ocular physiology is sensitive to cognitively demanding tasks. However, it is unknown whether the intraocular pressure is also affected by the cognitive demands of military operations. The main objective was to determine the impact of a virtual reality shooting simulation with two levels of complexity on intraocular pressure levels in military personnel. Eighteen active-duty members of the Spanish Army and eighteen civilians performed two 4 min simulated shooting tasks with two levels of complexity using a virtual reality. In the "easy" task participants performed a simulated shoot when the stimulus (military with a rifle) appeared, while in the "difficult" task the stimulus randomly was a military with a rifle or with his hands on the air and participants were instructed to respond only when the military with a rifle appeared. Intraocular pressure was measured with a rebound tonometer before and immediately after each task. Complementarily, perceived levels of mental load and shooting performance (reaction time) were assessed. Intraocular pressure was greater after completing the more complex task in both military personnel (p-value < 0.01, Cohen´s d = 1.19) and civilians (p-value < 0.01, Cohen´s d = 1.16). Also, perceived levels of task load and reaction time were higher in the difficult compared to the easy shooting tasks (both p < 0.001). The rise in intraocular pressure is positively associated with the cognitive demands of simulated military operations. The potential application of this finding is the development of objective tools based on intraocular pressure for the evaluation of the mental state in real-world contexts, permitting to improve soldiers´ safety and performance.

The "Effort Elephant" in the Room: What Is Effort, Anyway?

Despite decades of research in the fields of judgment and decision-making, social psychology, cognitive psychology, human-machine interaction, behavioral economics, and neuroscience, we still do not know what "cognitive effort" is. The definitions in use are often imprecise and sometimes diametrically opposed. Researchers with different assumptions talk past each other, and many aspects of effort conservation remain untested and difficult to measure. In this article, we explain why effort is so difficult to pin down and why it is important that researchers develop consensus on precise definitions. Next, we describe major "hidden" sources of miscommunication: areas in which researchers disagree in their underlying assumptions about the nature of effort without realizing it. We briefly review a number of methods used to both measure and manipulate the effortfulness of thinking and highlight why they often produce contradictory findings. We conclude by reviewing existing perspectives on cognitive effort and integrating them to suggest a common framework for communicating about effort as a limited cognitive resource.

Assessing mental workload with wearable devices - Reliability and applicability of heart rate and motion measurements

Wearable devices are increasingly used for assessing physiological data. Industry 4.0 aims to achieve the real-time assessment of the workers' condition to adapt processes including the current mental workload. Mental workload can be assessed via physiological data. This paper researches the potential of wearable devices for mental workload assessment by utilizing heart rate and motion data collected with a smartwatch. A laboratory study was conducted with four levels of mental workload, ranging from none to high and during sitting and stepping activities. When sitting, a difference in the heart rate and motion data from the smartwatch was only found between no mental workload and any mental workload task. For the stepping condition, differences were found for the movement data. Based on these results, wearable devices could be useful in the future for detecting whether a mental demanding task is currently performed during low levels of physical activity.

Modeling Cognitive Load as a Self-Supervised Brain Rate with Electroencephalography and Deep Learning

The principal reason for measuring mental workload is to quantify the cognitive cost of performing tasks to predict human performance. Unfortunately, a method for assessing mental workload that has general applicability does not exist yet. This is due to the abundance of intuitions and several operational definitions from various fields that disagree about the sources or workload, its attributes, the mechanisms to aggregate these into a general model and their impact on human performance. This research built upon these issues and presents a novel method for mental workload modelling from EEG data employing deep learning. This method is self-supervised, employing a continuous brain rate, an index of cognitive activation, and does not require human declarative knowledge. The aim is to induce models automatically from data, supporting replicability, generalisability and applicability across fields and contexts. This specific method is a convolutional recurrent neural network trainable with spatially preserving spectral topographic head-maps from EEG data, aimed at fitting a novel brain rate variable. Findings demonstrate the capacity of the convolutional layers to learn meaningful high-level representations from EEG data since within-subject models had, on average, a test Mean Absolute Percentage Error of around 11%. The addition of a Long-Short Term Memory layer for handling sequences of high-level representations was not significant, although it did improve their accuracy. These findings point to the existence of quasi-stable blocks of automatically learnt high-level representations of cognitive activation because they can be induced through convolution and seem not to be dependent on each other over time, intuitively matching the non-stationary nature of brain responses. Additionally, across-subject models, induced with data from an increasing number of participants, thus trained with data containing more variability, obtained a similar accuracy to the within-subject models. This highlights the potential generalisability of the induced high-level representations across people, suggesting the existence of subject-independent cognitive activation patterns. This research contributes to the body of knowledge by providing scholars with a novel computational method for mental workload modelling that aims to be generally applicable and does not rely on ad hoc human crafted models.

User Experience and Physiological Response in Human-Robot Collaboration: A Preliminary Investigation

Within the context of Industry 4.0 and of the new emerging Industry 5.0, human factors are becoming increasingly important, especially in Human-Robot Collaboration (HRC). This paper provides a novel study focused on the human aspects involved in industrial HRC by exploring the effects of various HRC setting factors. In particular, this paper aims at investigating the impact of industrial HRC on user experience, affective state, and stress, assessed through both subjective measures (i.e., questionnaires) and objective ones (i.e., physiological signals). A collaborative assembly task was implemented with different configurations, in which the robot movement speed, the distance between the operator and the robot workspace, and the control of the task execution time were varied. Forty-two participants were involved in the study and provided feedbacks on interaction quality and their affective state. Participants’ physiological responses (i.e., electrodermal activity and heart rate) were also collected non-invasively to monitor the amount of stress generated by the interaction. Analysis of both subjective and objective responses revealed how the configuration factors considered influence them. Robot movement speed and control of the task execution time resulted to be the most influential factors. The results also showed the need for customization of HRC to improve ergonomics, both psychological and physical, and the well-being of the operator.

Effects of Noise Exposure and Mental Workload on Physiological Responses during Task Execution

Twelve healthy male students were recruited to investigate the physiological response to different noise exposure and mental workload (MW) conditions, while performing multi-attribute task battery (MATB) tasks. The experiments were conducted under three noise exposure conditions, with different sound pressure levels and sharpness. After adaptation to each noise condition, the participants were required to perform the resting test and the MATB task tests with low, medium, and high MW. The electroencephalogram (EEG), electrocardiogram (ECG), and eye movement data were obtained, during the periods when participants were in the resting and task taking state. The results showed that subjects' physiological responses at rest were unaffected by noise exposure conditions. However, during the execution of MATB tasks, the elevated sound pressure level and increased sharpness were significantly correlated with increased mean pupil diameter and heart rate variability (HRV). These responses suggested that the human body defends itself through physiological regulation when noise causes adverse effects. If the negative effects of noise were more severe, this could damage the body's health and result in a significant drop in task performance. The elevated mental demands led to increased stress on the subjects, which was reflected in a considerable increase in theta relative power. Either high or low MW was related with reduced saccade amplitude and a decrease in weighted task performance, indicating an inverted U-shaped relationship between workload level and work performance.

Toward Mental Effort Measurement Using Electrodermal Activity Features

The ability to monitor mental effort during a task using a wearable sensor may improve productivity for both work and study. The use of the electrodermal activity (EDA) signal for tracking mental effort is an emerging area of research. Through analysis of over 92 h of data collected with the Empatica E4 on a single participant across 91 different activities, we report on the efficacy of using EDA features getting at signal intensity, signal dispersion, and peak intensity for prediction of the participant's self-reported mental effort. We implemented the logistic regression algorithm as an interpretable machine learning approach and found that features related to signal intensity and peak intensity were most useful for the prediction of whether the participant was in a self-reported high mental effort state; increased signal and peak intensity were indicative of high mental effort. When cross-validated by activity moderate predictive efficacy was achieved (AUC = 0.63, F1 = 0.63, precision = 0.64, recall = 0.63) which was significantly stronger than using the model bias alone. Predicting mental effort using physiological data is a complex problem, and our findings add to research from other contexts showing that EDA may be a promising physiological indicator to use for sensor-based self-monitoring of mental effort throughout the day. Integration of other physiological features related to heart rate, respiration, and circulation may be necessary to obtain more accurate predictions.

Classification of Drivers' Mental Workload Levels: Comparison of Machine Learning Methods Based on ECG and Infrared Thermal Signals

Mental workload (MW) represents the amount of brain resources required to perform concurrent tasks. The evaluation of MW is of paramount importance for Advanced Driver-Assistance Systems, given its correlation with traffic accidents risk. In the present research, two cognitive tests (Digit Span Test-DST and Ray Auditory Verbal Learning Test-RAVLT) were administered to participants while driving in a simulated environment. The tests were chosen to investigate the drivers' response to predefined levels of cognitive load to categorize the classes of MW. Infrared (IR) thermal imaging concurrently with heart rate variability (HRV) were used to obtain features related to the psychophysiology of the subjects, in order to feed machine learning (ML) classifiers. Six categories of models have been compared basing on unimodal IR/unimodal HRV/multimodal IR + HRV features. The best classifier performances were reached by the multimodal IR + HRV features-based classifiers (DST: accuracy = 73.1%, sensitivity = 0.71, specificity = 0.69; RAVLT: accuracy = 75.0%, average sensitivity = 0.75, average specificity = 0.87). The unimodal IR features based classifiers revealed high performances as well (DST: accuracy = 73.1%, sensitivity = 0.73, specificity = 0.73; RAVLT: accuracy = 71.1%, average sensitivity = 0.71, average specificity = 0.85). These results demonstrated the possibility to assess drivers' MW levels with high accuracy, also using a completely non-contact and non-invasive technique alone, representing a key advancement with respect to the state of the art in traffic accident prevention.

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Augmented reality on industrial assembly line: Impact on effectiveness and mental workload

Studies examining the potential of augmented reality (AR) to improve assembly tasks are often unrepresentative of real assembly line conditions and assess mental workload only through subjective measurements and leads to conflicting results. We proposed a study directly carried out in industrial settings, to compare the impact of AR-based instructions to computerized instructions, on assembly effectiveness (completion time and errors) and mental workload using objective (eye tracking), subjective (NASA-TLX) and behavioral measurements (dual task paradigm). According to our results, AR did not improve effectiveness (increased assembly times and no decrease in assembly errors). Two out of three measurements indicated that AR led to more mental workload for simple assembly workstation, but equated computer instructions for complex workstation. Our data also suggest that, AR users were less able to detect external events (danger, alert), which may play an important role in the occurrence of work accidents.

Case Study of the Psychophysical State of Student-Operators During UAVO Training, Based on Heart Rate Parameter

This article focuses on the human factor in UAV operations. In the manuscript, research on the psychophysical state of student-operators under the license of UAVO VLOS <4 kg. For the analysis of the psychophysical state, the pulse parameter was used, which is one of the values that describe the work of the cardiovascular system and is one for the objective methods of assessing the psychophysical state of a human being. The data collected were analyzed using the STATISTICA software. The article focuses on the above aspect and analyzes the psychophysical state of the student-operator during flight training. The obtained results were also related to research on similar topics in the chapter discussion section.