What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures

The role of visual conditions and aircraft type on different aspects of pilot workload

OBJECTIVE

The objective of our work was to assess the impact of flight conditions by aircraft type on the workload estimated using NASA-Task Load Index (NASA-TLX).

BACKGROUND

Learning about subjective workload is important for assessing the impact of a pilot's work environment on their performance in the cockpit. This is an important element of flight safety and includes the prevention of aviation accidents.

METHODS

The study included 146 military pilots that fly the following aircrafts: flying fast-jet (21), fixed-wing (24), and rotary-wing (101). The NASA-TLX questionnaire was used to assess workload and pilots were asked to determine the level of workload resulting from flying under the following conditions: daytime flight (VFR), night-vision flight performed under Night Visual Flight Rules (NVFR), and night-vision flight using night-vision goggles (NVGs).

RESULTS

The highest level of workload was consistently attributed to flights performed under NVG conditions. NVFR conditions were rated as the most burdensome, while VFR conditions were rated as the least burdensome. Fast-jet pilots rated their mental performance and effort workload as significantly higher than pilots of other aircrafts.

CONCLUSION

Pilots' perceived workload is influenced by both flight conditions and the type of aircraft they fly. Workload knowledge is important for flight safety and should be taken into account during training and flight-task planning.

APPLICATION

The results of our study can be useful both in flight training and in work on the effectiveness of the human-machine interface. Awareness of one's own limitations due to the work environment can help improve flight safety.

One down, fifty to go: managers' perceptions of their workload and how they cope with it to maintain their psychological health

Background

Like many other countries, healthcare services in Canada face numerous organizational changes with the main objective of doing more with less. The approach taken within different healthcare networks has brought about a reform in healthcare facilities in Quebec, leading to several mergers and eliminating over 1,000 managerial positions. As a result, this has placed a progressively heavier workload on the shoulders of the remaining managers. Research on mental health in the workplace has mainly focused with the workforce and generally neglects managers. However, studies have shown that workload is a risk factor for managers. Therefore, the objectives of our study are to (1) better understand the elements that make up a manager's workload and the factors that influence it and (2) identify the coping strategies used by managers to deal with their workloads.

Methods

Employing a qualitative approach, we analyzed 61 semistructured interviews through an abductive method, utilizing diverse frameworks for data analysis. The participants came from the same Quebec healthcare establishment.

Results

Our findings align with the notion that workload is a multifaceted phenomenon that warrants a holistic analysis. The workload mapping framework we propose for healthcare network managers enables pinpointing those factors that contribute to the burden of their workload. Ultimately, this workload can detrimentally impact the psychological wellbeing of employees.

Conclusion

In conclusion, this study takes a comprehensive look at workload by using a holistic approach, enabling a more comprehensive understanding of this phenomenon. It also allows for the identification of coping strategies used by managers to deal with their workloads. Finally, our results can provide valuable guidance for the interventions aimed at addressing workload issues among healthcare network managers in Quebec by utilizing the specific elements we have identified.

Sharing imagery and analysis tools in a simulated submarine control room

We examined the impact of sharing periscope imagery and analysis tools in eighteen five-member teams (Sonar, Periscope, 2xTrack Motion Analysts, Track Manager) who undertook simulated submarine patrol tasks. Compared to a baseline condition, sharing imagery to team members increased perceived individual workload, with no improvement to team performance (tactical picture accuracy). When both imagery and analysis tools were shared, perceived workload increased and tactical picture compilation was more accurate. Despite this improved tactical picture for the imagery and analysis tools condition, there was no advantage to mission completion (rendezvous/close contact detection) or situation awareness. In contrast to the increased subjective workload, individuals in teams provided with shared imagery (with or without tools) had a lower physiological response (heart rate, electrodermal) to task demands compared to the baseline condition. Sharing imagery and analysis tools likely benefited tactical picture compilation by enabling dynamic task redistribution and multiple streams of concurrent data analysis.

Development and validation of a task load index for process control room operators (PCRO-TLX)

Process control room operators (PCRO) perform a range of complex cognitive safety-critical tasks. The aim of this exploratory sequential mixed methods study was to develop an occupation specific tool to measure the task load of PCRO using NASA Task Load Index (TLX) methodology. Participants were 30 human factors experts and 146 PCRO at two refinery complexes in Iran. Dimensions were developed via a cognitive task analysis, a research review, and three expert panels. Six dimensions were identified: perceptual demand, performance, mental demand, time pressure, effort, and stress. Data from 120 PCRO confirmed that the developed PCRO-TLX has acceptable psychometric properties, and a comparison with the NASA-TLX confirmed that perceptual, not physical, demand was relevant for measuring workload in PCRO. There was a positive convergence of scores of the Subjective Workload Assessment Technique and the PCRO-TLX. This reliable tool (α = 0.83) is recommended for risk assessing the task load of PCRO.Practitioner summary: There are benefits of having a specific tool to measure task load in safety critical roles. Thus, we developed and validated an easy-to-use targeted tool, the PCRO-TLX, for process control room operatives. Timely use and response will assure optimal production alongside health and safety in an organisation.Abbreviations: PCRO: process control room operator; TLX: task load index; PCRO-TLX: process control room operator task load index; NASA-TLX: National Aeronautics and Space Administration task load index; SWAT: subjective workload assessment technique; DALI: driving activity load index; SURG-TLX: surgery task load index; SIM-TLX: virtual reality simulation task load index; VACP: visual, auditory, cognitive and psychomotor; CVI: content validity index; CVR: content validity ratio; RMSEA: root mean square of error approximation; GFI: goodness of fit index; AGFI: adjusted goodness of fit index; CFI: comparative fit index; ANOVA: analysis of variance; CI: confidence interval.

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

Humans' performance varies due to the mental resources that are available to successfully pursue a task. To monitor users' current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications.

Validation of Endurance Model for Manual Tasks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-5. [PMID: 38083441 DOI: 10.1109/embc40787.2023.10341139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Physical fatigue in the workplace can lead to work-related musculoskeletal disorders (WMSDs), especially in occupations that require repetitive, mid-air movements, such as manufacturing and assembly tasks in industry settings. The current paper endeavors to validate an existing torque-based fatigue prediction model for lifting tasks. The model uses anthropometrics and the maximum torque of the individual to predict the time to fatigue. Twelve participants took part in the study which measured body composition parameters and the maximum force produced by the shoulder joint in flexion, followed by three lifting tasks for the shoulder in flexion, including isometric and dynamic tasks with one and two hands. Inertial measurements units (IMUs) were worn by participants to determine the torque at each instant to calculate the endurance time and CE, while a self-subjective questionnaire was utilized to assess physical exertion, the Borg Rate of Perceived Exertion (RPE) scale. The model was effective for static and two-handed tasks and produced errors in the range of [28.62 49.21] for the last task completed, indicating the previous workloads affect the endurance time, even though the individual perceives they are fully rested. The model was not effective for the one-handed dynamic task and differences were observed between males and females, which will be the focus of future work.An individualized, torque-based fatigue prediction model, such as the model presented, can be used to design worker-specific target levels and workloads, take inter and intra individual differences into account, and put fatigue mitigating interventions into place before fatigue occurs; resulting in potentially preventing WMSDs, aiding in worker wellbeing and benefitting the quality and efficiency of the work output.Clinical Relevance- This research provides the basis for an individualized, torque-based approach to the prediction of fatigue at the shoulder joint which can be used to assign worker tasks and rest breaks, design worker specific targets and reduce the prevalence of work-related musculoskeletal disorders in occupational settings.

A Systematic Review of Cost, Effort, and Load Research in Information Search and Retrieval, 1972-2020

During the Information Search and Retrieval (ISR) process, user-system interactions such as submitting queries, examining results, and engaging with information, impose some degree of demand on the user’s resources. Within ISR, these demands are well recognised, and numerous studies have demonstrated that the Cost, Effort, and Load (CEL) experienced during the search process are affected by a variety of factors. Despite this recognition, there is no universally accepted definition of the constructs of CEL within the field of ISR. Ultimately this has led to problems with how these constructs have been interpreted and subsequently measured. This systematic review contributes a synthesis of literature, summarising key findings relating to how researchers have been defining and measuring CEL within ISR over the past 50 years. After manually screening 1,109 articles, we detail and analyse 91 articles which examine CEL within ISR. The discussion focuses on comparing the similarities and differences between CEL definitions and measures before identifying the limitations of the current state of the nomenclature. Opportunities for future research are also identified. Going forward, we propose a CEL taxonomy that integrates the relationships between CEL and their related constructs, which will help focus and disambiguate future research in this important area.

Distracted worker: Using pupil size and blink rate to detect cognitive load during manufacturing tasks

This study sets out to extend the use of blink rate and pupil size to the assessment of cognitive load of completing common automotive manufacturing tasks. Nonoptimal cognitive load is detrimental to safety. Existing occupational ergonomics approaches come short of measuring dynamic changes in cognitive load during complex assembling tasks. Cognitive demand was manipulated by having participants complete two versions of the n-back task (easy, hard). Two durations of the physical task were also considered (short, long). Pupil size and blink rate increased under greater cognitive task demand. High cognitive load also resulted in longer task completion times, and higher ratings of mental and temporal demand, and effort. This exploratory study offers relevant insights on the use of ocular metrics for cognitive load assessment in occupational ergonomics. While the existing eye-tracking technology may yet limit their adoption in the field, they offer advantages over the more popular expert-based and self-reported techniques in measuring changes in cognitive load during dynamic tasks.

Trust is Not a Virtue: Why We Should Not Trust Trust

There is currently significant research and industry interest in engineering machines and algorithms that humans will trust. This is justified as a means for facilitating the adoption of developing technology. However, there are many problems with trust that directly relate to its epistemological validity, usefulness, ethical implications, and potential for human disempowerment. This article explores trust from this perspective in the hopes of encouraging the human factors engineering community to de-emphasize trust as an end goal and replace it with more objective measures and good human factors engineering practices.

Human Mental Workload: A Survey and a Novel Inclusive Definition

Human mental workload is arguably the most invoked multidimensional construct in Human Factors and Ergonomics, getting momentum also in Neuroscience and Neuroergonomics. Uncertainties exist in its characterization, motivating the design and development of computational models, thus recently and actively receiving support from the discipline of Computer Science. However, its role in human performance prediction is assured. This work is aimed at providing a synthesis of the current state of the art in human mental workload assessment through considerations, definitions, measurement techniques as well as applications, Findings suggest that, despite an increasing number of associated research works, a single, reliable and generally applicable framework for mental workload research does not yet appear fully established. One reason for this gap is the existence of a wide swath of operational definitions, built upon different theoretical assumptions which are rarely examined collectively. A second reason is that the three main classes of measures, which are self-report, task performance, and physiological indices, have been used in isolation or in pairs, but more rarely in conjunction all together. Multiple definitions complement each another and we propose a novel inclusive definition of mental workload to support the next generation of empirical-based research. Similarly, by comprehensively employing physiological, task-performance, and self-report measures, more robust assessments of mental workload can be achieved.

Computer Vision and Human Behaviour, Emotion and Cognition Detection: A Use Case on Student Engagement

Computer vision has shown great accomplishments in a wide variety of classification, segmentation and object recognition tasks, but tends to encounter more difficulties when tasks require more contextual assessment. Measuring the engagement of students is an example of such a complex task, as it requires a strong interpretative component. This research describes a methodology to measure students’ engagement, taking both an individual (student-level) and a collective (classroom) approach. Results show that students’ individual behaviour, such as note-taking or hand-raising, is challenging to recognise, and does not correlate with students’ self-reported engagement. Interestingly, students’ collective behaviour can be quantified in a more generic way using measures for students’ symmetry, reaction times and eye-gaze intersections. Nonetheless, the evidence for a connection between these collective measures and engagement is rather weak. Although this study does not succeed in providing a proxy of students’ self-reported engagement, our approach sheds light on the needs for future research. More concretely, we suggest that not only the behavioural, but also the emotional and cognitive component of engagement should be captured.

Development and validation of a behavioural video coding scheme for detecting mental workload in manual assembly

Manual assembly in the future Industry 4.0 workplace will put high demands on operators' cognitive processing. The development of mental workload (MWL) measures therefore looms large. Physiological gauges such as electroencephalography (EEG) show promising possibilities, but still lack sufficient reliability when applied in the field. This study presents an alternative measure with a substantial ecological validity. First, we developed a behavioural video coding scheme identifying 11 assembly behaviours potentially revealing MWL being too high. Subsequently, we explored its validity by analysing videos of 24 participants performing a high and a low complexity assembly. Results showed that five of the behaviours identified, such as freezing and the amount of part rotations, significantly differed in occurrence and/or duration between the two conditions. The study hereby proposes a novel and naturalistic method that could help practitioners to map and redesign critical assembly phases, and researchers to enrich validation of MWL-measures through measurement triangulation. Practitioner summary: Current physiological mental workload (MWL) measures still lack sufficient reliability when applied in the field. Therefore, we identified several observable assembly behaviours that could reveal MWL being too high. The results propose a method to map MWL by observing specific assembly behaviours such as freezing and rotating parts. Abbreviations: MWL: mental workload; EEG: electroencephalography; fNIRS: functional near infrared spectroscopy; AOI: area of interest; SMI: SensoMotoric Instruments, ETG: Eye-Tracking Glasses; FPS: frames per second; BORIS: Behavioral Observation Research Interactive Software; IRR: inter-rater reliability; SWAT: Subjective Workload Assessment Technique; NASA-TLX: National Aeronautics and Space Administration Task Load Index; EL: emotional load; DSSQ: Dundee Stress State Questionnaire; PHL: physical load; SBO: Strategisch Basis Onderzoek.

Understanding individualistic response patterns when assessing expert operators on nuclear power plant control tasks

We evaluated the performance of three highly practiced participants on three task types that comprised a simulated nuclear power plant control operation. Multiple subjective, physiological, and objective performance measures were collected on these three highly-practiced individuals. Results indicated ceiling effects in terms of performance accuracy, yet each individual adopted a unique response strategy across the respective sub-tasks. Their maximised accuracy was achieved at the expense of longer response times across differing sub-tasks. The measures which proved diagnostic and predictive of performance capacity were explored. The current conclusion presents us with an invidious problem in that performance and workload associations, insensitivities, and dissociations may be unique to each individual operator, and may well depend also upon the overall task in context. Such findings push our science away from seeking nomothetic assertions and toward individuated concerns. In consequence, the age of the idiographic may well be upon us. Practitioner summary: The importance and relevance of nuclear power control is self-evident. Concerns here have centred around the safety of the technology and its operators. Our work informs practitioners in this industry, and in Ergonomics in general, of the response of highly trained individuals in these safety-critical, operational domains. We show that even experts engage in personal and individual strategies, an observation critical to the assessment of this specific workplace, and potentially all others. Abbreviations: NPP: nuclear power plant; ROs: reactor operators; MCR: main control room; LOA: levels of automation; EOP: emergency operating procedure; OP: operating procedures; ISA: instantaneous self-assessment; DSSQ: Dundee stres state questionnaire.