Auditory probe sensitivity to mental workload changes - an event-related potential study

Evaluation of a headphones-fitted EEG system for the recording of auditory evoked potentials and mental workload assessment

Advancements in portable neuroimaging technologies open up new opportunities to gain insight into the neural dynamics and cognitive processes underlying day-to-day behaviors. In this study, we evaluated the relevance of a headphone- mounted electroencephalogram (EEG) system for monitoring mental workload. The participants (N = 12) were instructed to pay attention to auditory alarms presented sporadically while performing the Multi-Attribute Task Battery (MATB) whose difficulty was staged across three conditions to manipulate mental workload. The P300 Event-Related Potentials (ERP) elicited by the presentation of auditory alarms were used as probes of attentional resources available. The amplitude and latency of P300 ERPs were compared across experimental conditions. Our findings indicate that the P300 ERP component can be captured using a headphone-mounted EEG system. Moreover, neural responses to alarm could be used to classify mental workload with high accuracy (over 80%) at a single-trial level. Our analyses indicated that the signal-to-noise ratio acquired by the sponge-based sensors remained stable throughout the recordings. These results highlight the potential of portable neuroimaging technology for the development of neuroassistive applications while underscoring the current limitations and challenges associated with the integration of EEG sensors in everyday-life wearable technologies. Overall, our study contributes to the growing body of research exploring the feasibility and validity of wearable neuroimaging technologies for the study of human cognition and behavior in real-world settings.

Task-independent auditory probes reveal changes in mental workload during simulated quadrotor UAV training

Objective

The event-related potential (ERP) methods based on laboratory control scenes have been widely used to measure the level of mental workload during operational tasks. In this study, both task difficulty and test time were considered. Auditory probes (ignored task-irrelevant background sounds) were used to explore the changes in mental workload of unmanned aerial vehicle (UAV) operators during task execution and their ERP representations.

Approach

51 students participated in a 10-day training and test of simulated quadrotor UAV. During the experiment, background sound was played to induce ERP according to the requirements of oddball paradigm, and the relationship between mental workload and the amplitudes of N200 and P300 in ERP was explored.

Main results

Our study shows that the mental workload during operational task training is multi-dimensional, and its changes are affected by bottom-up perception and top-down cognition. The N200 component of the ERP evoked by the auditory probe corresponds to the bottom-up perceptual part; while the P300 component corresponds to the top-down cognitive part, which is positively correlated with the improvement of skill level.

Significance

This paper describes the relationship between ERP induced by auditory probes and mental workload from the perspective of multi-resource theory and human information processing. This suggests that the auditory probe can be used to reveal the mental workload during the training of operational tasks, which not only provides a possible reference for measuring the mental workload, but also provides a possibility for identifying the development of the operator's skill level and evaluating the training effect.

Measuring cognitive load of digital interface combining event-related potential and BubbleView

Helmet mounted display systems (HMDs) are high-performance display devices for modern aircraft. We propose a novel method combining event-related potentials (ERPs) and BubbleView to measure cognitive load under different HMD interfaces. The distribution of the subjects' attention resources is reflected by analyzing the BubbleView, and the input of the subjects' attention resources on the interface is reflected by analyzing the ERP's P3b and P2 components. The results showed that the HMD interface with more symmetry and a simple layout had less cognitive load, and subjects paid more attention to the upper portion of the interface. Combining the experimental data of ERP and BubbleView, we can obtain a more comprehensive, objective, and reliable HMD interface evaluation result. This approach has significant implications for the design of digital interfaces and can be utilized for the iterative evaluation of HMD interfaces.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

Neurocognitive Dynamics of Prosodic Salience over Semantics during Explicit and Implicit Processing of Basic Emotions in Spoken Words

How language mediates emotional perception and experience is poorly understood. The present event-related potential (ERP) study examined the explicit and implicit processing of emotional speech to differentiate the relative influences of communication channel, emotion category and task type in the prosodic salience effect. Thirty participants (15 women) were presented with spoken words denoting happiness, sadness and neutrality in either the prosodic or semantic channel. They were asked to judge the emotional content (explicit task) and speakers' gender (implicit task) of the stimuli. Results indicated that emotional prosody (relative to semantics) triggered larger N100, P200 and N400 amplitudes with greater delta, theta and alpha inter-trial phase coherence (ITPC) and event-related spectral perturbation (ERSP) values in the corresponding early time windows, and continued to produce larger LPC amplitudes and faster responses during late stages of higher-order cognitive processing. The relative salience of prosodic and semantics was modulated by emotion and task, though such modulatory effects varied across different processing stages. The prosodic salience effect was reduced for sadness processing and in the implicit task during early auditory processing and decision-making but reduced for happiness processing in the explicit task during conscious emotion processing. Additionally, across-trial synchronization of delta, theta and alpha bands predicted the ERP components with higher ITPC and ERSP values significantly associated with stronger N100, P200, N400 and LPC enhancement. These findings reveal the neurocognitive dynamics of emotional speech processing with prosodic salience tied to stage-dependent emotion- and task-specific effects, which can reveal insights into understanding language and emotion processing from cross-linguistic/cultural and clinical perspectives.

The Effect of Cognitive Load on Auditory Susceptibility During Automated Driving

OBJECTIVE

We experimentally test the effect of cognitive load on auditory susceptibility during automated driving.

BACKGROUND

In automated vehicles, auditory alerts are frequently used to request human intervention. To ensure safe operation, human drivers need to be susceptible to auditory information. Previous work found reduced susceptibility during manual driving and in a lesser amount during automated driving. However, in practice, drivers also perform nondriving tasks during automated driving, of which the associated cognitive load may further reduce susceptibility to auditory information. We therefore study the effect of cognitive load during automated driving on auditory susceptibility.

METHOD

Twenty-four participants were driven in a simulated automated car. Concurrently, they performed a task with two levels of cognitive load: repeat a noun or generate a verb that expresses the use of this noun. Every noun was followed by a probe stimulus to elicit a neurophysiological response: the frontal P3 (fP3), which is a known indicator for the level of auditory susceptibility.

RESULTS

The fP3 was significantly lower during automated driving with cognitive load compared with without. The difficulty level of the cognitive task (repeat or generate) showed no effect.

CONCLUSION

Engaging in other tasks during automated driving decreases auditory susceptibility as indicated by a reduced fP3.

APPLICATION

Nondriving task can create additional cognitive load. Our study shows that performing such tasks during automated driving reduces the susceptibility for auditory alerts. This can inform designers of semi-automated vehicles (SAE levels 3 and 4), where human intervention might be needed.

EEG reveals deficits in sensory gating and cognitive processing in asymptomatic adults with a history of concussion

OBJECTIVE

Individuals with a concussion history tend to perform worse on dual-tasks compared controls but the underlying neural mechanisms contributing to these deficits are not understood. This study used event-related potentials (ERPs) to investigate sensory gating and cognitive processing in athletes with and without a history of concussion while they performed a challenging dual-task.

METHODS

We recorded sensory (P50, N100) and cognitive (P300) ERPs in 30 athletes (18 no previous concussion; 12 history of concussion) while they simultaneously performed an auditory oddball task and a working memory task that progressively increased in difficulty.

RESULTS

The concussion group had reduced auditory performance as workload increased compared to the no-concussion group. Sensory gating and cognitive processing were reduced in the concussion group indicating problems with filtering relevant from irrelevant information and appropriately allocating resources. Sensory gating (N100) was positively correlated with cognitive processing (P300) at the hardest workload in the no-concussion group but negatively correlated in the concussion group.

CONCLUSION

Concussions result in long-term problems in behavioral performance, which may be due to poorer sensory gating that impacts cognitive processing.

SIGNIFICANCE

Problems effectively gating sensory information may influence the availability or allocation of attention at the cognitive stage in those with a concussion.

Investigation of the optimal time interval between task-irrelevant auditory probes for evaluating mental workload in the shortest possible time

Event-related brain potentials (ERPs) elicited by auditory stimuli unrelated to a current visual-cognitive task (i.e., task-irrelevant auditory probes) can be used to evaluate the level of mental workload. Towards the evaluation of workload in the shortest possible time, the present study with a multiple-stimulus paradigm (Takeda and Kimura, 2014, Int. J. Psychophysiol.) examined whether manipulating time intervals between probes could improve the temporal resolution in evaluating workload. Probes were presented in four interval conditions as a combination of two mean interval lengths [long (600 ms) vs. short (300 ms)] and two interval variabilities [variable (five levels) vs. fixed], while participants were performing a driving game at slow and fast speeds (i.e., imposing low and high workload, respectively). For each interval condition, the minimum data length required to obtain a significant difference in the amplitude of ERPs (i.e., auditory N1 and P2) between the slow and fast driving tasks was estimated. The N1 difference was significant in all four interval conditions but the required minimum data lengths to observe this difference did not greatly differ across the interval conditions (about 60-90 s). The P2 difference was significant only in the long-variable condition and the required minimum data length was about 120 s. These results suggest that, at least with a multiple-stimulus paradigm, manipulations of time intervals between probes did not greatly improve the temporal resolution in evaluating mental workload; at present, long-variable intervals would be optimal for evaluating mental workload in detail.

Cross-Task Consistency of Electroencephalography-Based Mental Workload Indicators: Comparisons Between Power Spectral Density and Task-Irrelevant Auditory Event-Related Potentials

Mental workload (MWL) estimators based on ongoing electroencephalography (EEG) and event-related potentials (ERPs) have shown great potentials to build adaptive aiding systems for human-machine systems by estimating MWL in real time. However, extracting EEG features which are consistent in indicating MWL across different tasks is still one of the critical challenges. This study attempts to compare the cross-task consistency in indexing MWL variations between two commonly used EEG-based MWL indicators, power spectral density (PSD) of ongoing EEG and task-irrelevant auditory ERPs (tir-aERPs). The verbal N-back and the multi-attribute task battery (MATB), both with two difficulty levels, were employed in the experiment, along with task-irrelevant auditory probes. EEG was recorded from 17 subjects when they were performing the tasks. The tir-aERPs elicited by the auditory probes and the relative PSDs of ongoing EEG between two consecutive auditory probes were extracted and statistically analyzed to reveal the effects of MWL and task type. Discriminant analysis and support vector machine were employed to examine the generalization of tir-aERP and PSD features in indexing MWL variations across different tasks. The results showed that the amplitudes of tir-aERP components, N1, early P3a, late P3a, and the reorienting negativity, significantly decreased with the increasing MWL in both N-back and MATB. Task type had no obvious influence on the amplitudes and topological layout of the MWL-sensitive tir-aERP features. The relative PSDs in θ, α, and low β bands were also sensitive to MWL variations. However, the MWL-sensitive PSD features and their topological patterns were significantly affected by task type. The cross-task classification results based on tir-aERP features also significantly outperformed the PSD features. These results suggest that the tir-aERPs should be potentially more consistent MWL indicators across very different task types when compared to PSD. The current study may provide new insights to our understanding of the common and distinctive neuropsychological essences of MWL across different tasks.

Working With Environmental Noise and Noise-Cancelation: A Workload Assessment With EEG and Subjective Measures

As working and learning environments become open and flexible, people are also potentially surrounded by ambient noise, which causes an increase in mental workload. The present study uses electroencephalogram (EEG) and subjective measures to investigate if noise-canceling technologies can fade out external distractions and free up mental resources. Therefore, participants had to solve spoken arithmetic tasks that were read out via headphones in three sound environments: a quiet environment (no noise), a noisy environment (noise), and a noisy environment but with active noise-canceling headphones (noise-canceling). Our results of brain activity partially confirm an assumed lower mental load in no noise and noise-canceling compared to noise test condition. The mean P300 activation at Cz resulted in a significant differentiation between the no noise and the other two test conditions. Subjective data indicate an improved situation for the participants when using the noise-canceling technology compared to “normal” headphones but shows no significant discrimination. The present results provide a foundation for further investigations into the relationship between noise-canceling technology and mental workload. Additionally, we give recommendations for an adaptation of the test design for future studies.

A novel approach to validate the efficacy of single task ERP paradigms to measure cognitive workload

The present study examined the utility of a single-task paradigm to evaluate cognitive workload. The cognitive workload from twenty-five healthy participants was measured during a tilt-ball game while tones were presented in the background to generate event-related potentials (ERPs) in electroencephalographic (EEG) data. In the game, participants were instructed to move the ball to highlighted targets and avoid moving obstacles. The game's difficulty level was manipulated (easy, medium, hard) by adjusting the number and speed of the moving obstacles. The difficulty levels were presented in a random order during multiple short runs to minimize the effects of habituation, fatigue, and boredom. The behavioral results showed that greater task difficulty resulted in a significant decrease (p < 0.001) in game performance, i.e., participants achieved few targets with a high collision rate. To evaluate cognitive workload, we measured the amplitude of early ERP components (N1, P1, and P2) corresponding to the involuntary attention orienting response. The amplitude of the N1 component decreased significantly (p = 0.029) with an increase in cognitive workload. These findings suggest that the early ERP component, specifically the N1, corresponds to attention orienting response, and that the task difficulty modulates it. This study provided evidence that the inverse relationship between ERP components and cognitive workload can be reliably assessed by controlling for other factors such as habituation or boredom during a single task paradigm.

Task-irrelevant Auditory Event-related Potentials as Mental Workload Indicators: A Between-task Comparison Study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020;2020:3216-3219. [PMID: 33018689 DOI: 10.1109/embc44109.2020.9175957]

Real-time monitoring of mental workload (MWL) is a crucial step to build closed-loop adaptive aiding systems for human-machine systems. MWL estimators based on spontaneous electroencephalography (EEG) and event-related potentials (ERPs) have shown great potentials to achieve this goal. However, the previous studies show that the between-task robustness of these EEG/ERP-based MWL estimators is still an unsolved intractable question. This study attempts to examine the task-irrelevant auditory event-related potentials (tir-aERPs) as MWL indicators. A working memory task (verbal n-back) and a visuo-motor task (multi-attribute task battery, MATB), both with two difficulty levels (easy and hard), were used in the experiment, along with task-irrelevant auditory probes that did not need any response from the participants. EEG was recorded from ten participants when they were performing the tasks. The tir-aERPs elicited by the auditory probes were extracted and analyzed. The results show that the amplitudes of N1, early P3a (eP3a) and the late reorienting negativity (RON) significantly decreased with the increasing MWL in both n-back and MATB. Task type has no obvious influence on the amplitudes and topological layout of the MWL-sensitive tir-aERPs features. These results suggest that the tir-aERPs are potentially more constant MWL indicators across very different task types. Therefore, the tir-aERPs should be taken into consideration in future task-independent MWL monitoring studies.

The influence of cognitive load on susceptibility to audio

In this study we evaluate how cognitive load affects susceptibility to auditory signals. Previous research has used the frontal P3 (fP3) event related potential response to auditory novel stimuli as an index for susceptibility to auditory signals. This work demonstrated that tasks that induce cognitive load such as visual and manual tasks, reduced susceptibility. It is however unknown whether cognitive load without visual or manual components also reduces susceptibility. To investigate this, we induced cognitive load by means of the verb generation task, in which participants need to think about a verb that matches a noun. The susceptibility to auditory signals was measured by recording the event related potential in response to a successively presented oddball probe stimulus at 3 different inter-stimulus intervals, 0 ms, 200 ms or 400 ms after the offset of the noun from the verb generation task. An additional control baseline condition, in which oddball response was probed without a verb generation task, was also included. Results show that the cognitive load associated with the verb task reduces fP3 response (and associated auditory signal susceptibility) compared to baseline, independent of presentation interval. This suggests that not only visual and motor processing, but also cognitive load without visual or manual components, can reduce susceptibility to auditory signals and alerts.

Background Music Dependent Reduction of Aversive Perception and Its Relation to P3 Amplitude Reduction and Increased Heart Rate

Music is commonly used to modify mood and has attracted attention as a potential therapeutic intervention. Despite the well-recognized effects of music on mood, changes in affective perception due to music remain majorly unknown. Here, we examined if the perception of aversive stimuli could be altered by mood-changing background music. Using subjective scoring data from 17 healthy volunteers, we assessed the effect of relaxing background music (RelaxBGM), busy background music (BusyBGM), or no background music (NoBGM) conditions on response to aversive white noise stimulation. Interestingly, affective response to the white noise was selectively alleviated, and white noise-related P3 component amplitude was reduced in BusyBGM. However, affective responses as well as P3 amplitude to reference pure tone stimuli were similar regardless of background music conditions. Interestingly, heart rate (HR) increased in BusyBGM, whereas no increase in HR was found in similar distress, NoBGM condition. These findings suggest that increase in HR, which happens during BusyBGM exposure, can be a reflecting feature of music that ameliorates the affective response to aversive stimuli, possibly through selective reduction in neurophysiological responses.

Measurement of cognitive dynamics during video watching through event-related potentials (ERPs) and oscillations (EROs)

Event-related potentials (ERPs) and oscillations (EROs) are reliable measures of cognition, but they require time-locked electroencephalographic (EEG) data to repetitive triggers that are not available in continuous sensory input streams. However, such real-life-like stimulation by videos or virtual-reality environments may serve as powerful means of creating specific cognitive or affective states and help to investigate dysfunctions in psychiatric and neurological disorders more efficiently. This study aims to develop a method to generate ERPs and EROs during watching videos. Repeated luminance changes were introduced on short video segments, while EEGs of 10 subjects were recorded. The ERP/EROs time-locked to these distortions were analyzed in time and time-frequency domains and tested for their cognitive significance through a long term memory test that included frames from the watched videos. For each subject, ERPs and EROs corresponding to video segments of recalled images with 25% shortest and 25% longest reaction times were compared. ERPs produced by transient luminance changes displayed statistically significant fluctuations both in time and time-frequency domains. Statistical analyses showed that a positivity around 450 ms, a negativity around 500 ms and delta and theta EROs correlated with memory performance. Few studies mixed video streams with simultaneous ERP/ERO experiments with discrete task-relevant or passively presented auditory or somatosensory stimuli, while the present study, by obtaining ERPs and EROs to task-irrelevant events in the same sensory modality as that of the continuous sensory input, produces minimal interference with the main focus of attention on the video stream.

Effortful Processing Reduces the Attraction Effect in Multi-Alternative Decision Making: An Electrophysiological Study Using a Task-Irrelevant Probe Technique

The attraction effect in multi-alternative decision making reflects the context-dependent violation of axioms that are considered fundamental to rational choice. This effect is believed to depend on relatively effortless and intuitive processing (System 1) rather than on effortful and elaborative processing (System 2). To investigate the relationship between cognitive resources and the attraction effect in detail, we used a task-irrelevant probe technique, wherein task-irrelevant auditory probes were presented while participants viewed each alternative in a decision-making task, and measured the electroencephalographic responses to the probes. Thirty participants solved 48 hypothetical purchase problems with three alternatives that differed in terms of two attributes. We found that, in the second epoch of the experimental trials (possibly corresponding to the evaluation and comparison stage), the mean N1 amplitudes of the event-related potentials elicited by the auditory probes were significantly smaller when participants chose the competitor (i.e., trials in which no attraction effect occurred) than when participants chose the target (i.e., trials in which an attraction effect may have occurred). This result suggests that the allocation of more cognitive resources to the alternatives disrupts the attraction effect. This finding supports the assumption that intuitive comparisons among alternatives executed by System 1 are critical for the occurrence of the attraction effect.

Effects of Task Difficulty on Neural Processes Underlying Semantics: An Event-Related Potentials Study

Purpose Improving the ability to listen efficiently in noisy environments is a critical goal for hearing rehabilitation. However, understanding of the impact of difficult listening conditions on language processing is limited. The current study evaluated the neural processes underlying semantics in challenging listening conditions. Method Thirty adults with normal hearing completed an auditory sentence processing task in 4-talker babble. Event-related brain potentials were elicited by the final word in high- or low-context sentences, where the final word was either highly expected or not expected, followed by a 4-alternative forced-choice response with either longer (1,000 ms), middle (700 ms), or shorter (400 ms) response time deadlines (RTDs). Results Behavioral accuracy was reduced, and reactions times were faster for shorter RTDs. N400 amplitudes, reflecting ease of lexical access, were larger when elicited by target words in low-context sentences followed by shorter compared with longer RTDs. Conclusions These results reveal that more neural resources are allocated for semantic processing/lexical access when listening difficulty increases. Differences between RTDs may reflect increased attentional allocation for shorter RTDs. These findings suggest that situational listening demands can impact the demands for cognitive resources engaged in language processing, which could significantly impact listener experiences across environments.

Potential Mechanisms for the Ketamine-Induced Reduction of P3b Amplitudes

In specific dosages, the N-methyl-D-aspartate receptor (NMDA) antagonist ketamine can be used to model transient psychotic symptoms in healthy individuals that resemble those of schizophrenia. Ketamine administration also temporarily impairs cognitive functions, which can be studied by event-related potentials (ERPs). ERPs also allow dissecting what stages of information processing are affected by ketamine and what stages remain functional. For tasks requiring the differentiation of targets and non-targets, it has repeatedly been shown that ketamine administration in healthy individuals leads to decreased amplitudes of the ERP component P3b in response to target stimuli. However, it could be argued that this ketamine-induced P3b reduction is the consequence of an increased difficulty to differentiate targets from non-targets, primarily mediated by ketamine's psychotomimetic rather than pharmacological effects. The current review of ERP studies seeks to clarify the issue whether P3b effects of ketamine may indeed be explained as the consequence of an experienced increase in task difficulty or whether alternative mechanisms are perhaps more plausible. The review first summarizes the effects of task difficulty on ERP components related to intentional stimulus categorization (P3b), involuntary attention switches to distractors (P3a), as well as sensory processing (P1, N1). Secondly, the ERP effects of task difficulty are contrasted with those observed in ketamine studies in healthy individuals. Findings show that P3b amplitudes are consistently diminished by an increased task difficulty, as well as after ketamine administration. In contrast and as most important difference, increased task difficulty leads to increased P3a amplitudes to distractors presented in same modality as targets, whereas ketamine leads to reduced P3a amplitudes for such distractors. This dissociation indicates that the decreased P3b amplitudes after ketamine cannot be explained by a drug-induced increase in task difficulty. The conjoint reductions of P3a and P3b amplitudes instead suggest that working memory operations, in particular working memory updating are impaired after ketamine, which is in line with previous behavioral findings.

Measuring working memory load effects on electrophysiological markers of attention orienting during a simulated drive

Intersection accidents result in a significant proportion of road fatalities, and attention allocation likely plays a role. Attention allocation may depend on (limited) working memory (WM) capacity. Driving is often combined with tasks increasing WM load, consequently impairing attention orienting. This study (n = 22) investigated WM load effects on event-related potentials (ERPs) related to attention orienting. A simulated driving environment allowed continuous lane-keeping measurement. Participants were asked to orient attention covertly towards the side indicated by an arrow, and to respond only to moving cars appearing on the attended side by pressing a button. WM load was manipulated using a concurrent memory task. ERPs showed typical attentional modulation (cue: contralateral negativity, LDAP; car: N1, P1, SN and P3) under low and high load conditions. With increased WM load, lane-keeping performance improved, while dual task performance degraded (memory task: increased error rate; orienting task: increased false alarms, smaller P3). Practitioner Summary: Intersection driver-support systems aim to improve traffic safety and flow. However, in-vehicle systems induce WM load, increasing the tendency to yield. Traffic flow reduces if drivers stop at inappropriate times, reducing the effectiveness of systems. Consequently, driver-support systems could include WM load measurement during driving in the development phase.

Effects of Distracting Task with Different Mental Workload on Steady-State Visual Evoked Potential Based Brain Computer Interfaces-an Offline Study

Brain-computer interfaces (BCIs), independent of the brain's normal output pathways, are attracting an increasing amount of attention as devices that extract neural information. As a typical type of BCI system, the steady-state visual evoked potential (SSVEP)-based BCIs possess a high signal-to-noise ratio and information transfer rate. However, the current high speed SSVEP-BCIs were implemented with subjects concentrating on stimuli, and intentionally avoided additional tasks as distractors. This paper aimed to investigate how a distracting simultaneous task, a verbal n-back task with different mental workload, would affect the performance of SSVEP-BCI. The results from fifteen subjects revealed that the recognition accuracy of SSVEP-BCI was significantly impaired by the distracting task, especially under a high mental workload. The average classification accuracy across all subjects dropped by 8.67% at most from 1- to 4-back, and there was a significant negative correlation (maximum r = −0.48, p < 0.001) between accuracy and subjective mental workload evaluation of the distracting task. This study suggests a potential hindrance for the SSVEP-BCI daily use, and then improvements should be investigated in the future studies.

Enhancing performance of P300-Speller under mental workload by incorporating dual-task data during classifier training

As one of the most important brain-computer interface (BCI) paradigms, P300-Speller was shown to be significantly impaired once applied in practical situations due to effects of mental workload. This study aims to provide a new method of building training models to enhance performance of P300-Speller under mental workload. Three experiment conditions based on row-column P300-Speller paradigm were performed including speller-only, 3-back-speller and mental-arithmetic-speller. Data under dual-task conditions were introduced to speller-only data respectively to build new training models. Then performance of classifiers with different models was compared under the same testing condition. The results showed that when tasks of imported training data and testing data were the same, character recognition accuracies and round accuracies of P300-Speller with mixed-data training models significantly improved (FDR, p < 0.005). When they were different, performance significantly improved when tested on mental-arithmetic-speller (FDR, p < 0.05) while the improvement was modest when tested on n-back-speller (FDR, p < 0.1). The analysis of ERPs revealed that ERP difference between training data and testing data was significantly diminished when the dual-task data was introduced to training data (FDR, p < 0.05). The new method of training classifier on mixed data proved to be effective in enhancing performance of P300-Speller under mental workload, confirmed the feasibility to build a universal training model and overcome the effects of mental workload in its practical applications.

Reduced Attention Allocation during Short Periods of Partially Automated Driving: An Event-Related Potentials Study

Research on partially automated driving has revealed relevant problems with driving performance, particularly when drivers' intervention is required (e.g., take-over when automation fails). Mental fatigue has commonly been proposed to explain these effects after prolonged automated drives. However, performance problems have also been reported after just a few minutes of automated driving, indicating that other factors may also be involved. We hypothesize that, besides mental fatigue, an underload effect of partial automation may also affect driver attention. In this study, such potential effect was investigated during short periods of partially automated and manual driving and at different speeds. Subjective measures of mental demand and vigilance and performance to a secondary task (an auditory oddball task) were used to assess driver attention. Additionally, modulations of some specific attention-related event-related potentials (ERPs, N1 and P3 components) were investigated. The mental fatigue effects associated with the time on task were also evaluated by using the same measurements. Twenty participants drove in a fixed-base simulator while performing an auditory oddball task that elicited the ERPs. Six conditions were presented (5-6 min each) combining three speed levels (low, comfortable and high) and two automation levels (manual and partially automated). The results showed that, when driving partially automated, scores in subjective mental demand and P3 amplitudes were lower than in the manual conditions. Similarly, P3 amplitude and self-reported vigilance levels decreased with the time on task. Based on previous studies, these findings might reflect a reduction in drivers' attention resource allocation, presumably due to the underload effects of partial automation and to the mental fatigue associated with the time on task. Particularly, such underload effects on attention could explain the performance decrements after short periods of automated driving reported in other studies. However, further studies are needed to investigate this relationship in partial automation and in other automation levels.

Can an aircraft be piloted via sonification with an acceptable attentional cost? A comparison of blind and sighted pilots

In the aeronautics field, some authors have suggested that an aircraft's attitude sonification could be used by pilots to cope with spatial disorientation situations. Such a system is currently used by blind pilots to control the attitude of their aircraft. However, given the suspected higher auditory attentional capacities of blind people, the possibility for sighted individuals to use this system remains an open question. For example, its introduction may overload the auditory channel, which may in turn alter the responsiveness of pilots to infrequent but critical auditory warnings. In this study, two groups of pilots (blind versus sighted) performed a simulated flight experiment consisting of successive aircraft maneuvers, on the sole basis of an aircraft sonification. Maneuver difficulty was varied while we assessed flight performance along with subjective and electroencephalographic (EEG) measures of workload. The results showed that both groups of participants reached target-attitudes with a good accuracy. However, more complex maneuvers increased subjective workload and impaired brain responsiveness toward unexpected auditory stimuli as demonstrated by lower N1 and P3 amplitudes. Despite that the EEG signal showed a clear reorganization of the brain in the blind participants (higher alpha power), the brain responsiveness to unexpected auditory stimuli was not significantly different between the two groups. The results suggest that an auditory display might provide useful additional information to spatially disoriented pilots with normal vision. However, its use should be restricted to critical situations and simple recovery or guidance maneuvers.

Online EEG-Based Workload Adaptation of an Arithmetic Learning Environment

In this paper, we demonstrate a closed-loop EEG-based learning environment, that adapts instructional learning material online, to improve learning success in students during arithmetic learning. The amount of cognitive workload during learning is crucial for successful learning and should be held in the optimal range for each learner. Based on EEG data from 10 subjects, we created a prediction model that estimates the learner's workload to obtain an unobtrusive workload measure. Furthermore, we developed an interactive learning environment that uses the prediction model to estimate the learner's workload online based on the EEG data and adapt the difficulty of the learning material to keep the learner's workload in an optimal range. The EEG-based learning environment was used by 13 subjects to learn arithmetic addition in the octal number system, leading to a significant learning effect. The results suggest that it is feasible to use EEG as an unobtrusive measure of cognitive workload to adapt the learning content. Further it demonstrates that a promptly workload prediction is possible using a generalized prediction model without the need for a user-specific calibration.

Efficient Workload Classification based on Ignored Auditory Probes: A Proof of Concept

Mental workload is a mental state that is currently one of the main research focuses in neuroergonomics. It can notably be estimated using measurements in electroencephalography (EEG), a method that allows for direct mental state assessment. Auditory probes can be used to elicit event-related potentials (ERPs) that are modulated by workload. Although, some papers do report ERP modulations due to workload using attended or ignored probes, to our knowledge there is no literature regarding effective workload classification based on ignored auditory probes. In this paper, in order to efficiently estimate workload, we advocate for the use of such ignored auditory probes in a single-stimulus paradigm and a signal processing chain that includes a spatial filtering step. The effectiveness of this approach is demonstrated on data acquired from participants that performed the Multi-Attribute Task Battery - II. They carried out this task during two 10-min blocks. Each block corresponded to a workload condition that was pseudorandomly assigned. The easy condition consisted of two monitoring tasks performed in parallel, and the difficult one consisted of those two tasks with an additional plane driving task. Infrequent auditory probes were presented during the tasks and the participants were asked to ignore them. The EEG data were denoised and the probes' ERPs were extracted and spatially filtered using a canonical correlation analysis. Next, binary classification was performed using a Fisher LDA and a fivefold cross-validation procedure. Our method allowed for a very high estimation performance with a classification accuracy above 80% for every participant, and minimal intrusiveness thanks to the use of a single-stimulus paradigm. Therefore, this study paves the way to the efficient use of ERPs for mental state monitoring in close to real-life settings and contributes toward the development of adaptive user interfaces.

Steering Demands Diminish the Early-P3, Late-P3 and RON Components of the Event-Related Potential of Task-Irrelevant Environmental Sounds

The current study investigates the demands that steering places on mental resources. Instead of a conventional dual-task paradigm, participants of this study were only required to perform a steering task while task-irrelevant auditory distractor probes (environmental sounds and beep tones) were intermittently presented. The event-related potentials (ERPs), which were generated by these probes, were analyzed for their sensitivity to the steering task's demands. The steering task required participants to counteract unpredictable roll disturbances and difficulty was manipulated either by adjusting the bandwidth of the roll disturbance or by varying the complexity of the control dynamics. A mass univariate analysis revealed that steering selectively diminishes the amplitudes of early P3, late P3, and the re-orientation negativity (RON) to task-irrelevant environmental sounds but not to beep tones. Our findings are in line with a three-stage distraction model, which interprets these ERPs to reflect the post-sensory detection of the task-irrelevant stimulus, engagement, and re-orientation back to the steering task. This interpretation is consistent with our manipulations for steering difficulty. More participants showed diminished amplitudes for these ERPs in the "hard" steering condition relative to the "easy" condition. To sum up, the current work identifies the spatiotemporal ERP components of task-irrelevant auditory probes that are sensitive to steering demands on mental resources. This provides a non-intrusive method for evaluating mental workload in novel steering environments.

Assessment of mental workload: A new electrophysiological method based on intra-block averaging of ERP amplitudes

The present study contributes to the current debate about electrophysiological measurements of mental workload. Specifically, the allocation of attentional resources during different complexity levels of tasks and its changes over time are of great interest. Therefore, we investigated mental workload using tasks varying in difficulty during an auditory oddball target paradigm. For data analysis, we applied a novel method to compute event-related potentials (ERPs) by intra-block epoch averaging of P2, P3a and P3b amplitude components for the infrequent target stimuli. We obtained eight consecutive blocks of 5 epochs each, which allowed us to develop an electrophysiological parameter to measure mental workload. In both the easy and the more constraining tasks, the amplitude of P2 decreased beginning with the second block of the sequence. In contrast, the amplitudes of P3a and P3b components linearly decreased following the repetition of the target in the more constraining task, but not in the easy task. Statistical analysis revealed intra-block differences on amplitudes of ERPs of interest between the easy and the more constraining tasks, confirming this method as a measure to assess mental workload. Since a subject is his own control, the present method represents an electrophysiological parameter for individual measurement of mental workload and may therefore be applicable in clinical routine.

Training and testing ERP-BCIs under different mental workload conditions

OBJECTIVE

As one of the most popular and extensively studied paradigms of brain-computer interfaces (BCIs), event-related potential-based BCIs (ERP-BCIs) are usually built and tested in ideal laboratory settings in most existing studies, with subjects concentrating on stimuli and intentionally avoiding possible distractors. This study is aimed at examining the effect of simultaneous mental activities on ERP-BCIs by manipulating various levels of mental workload during the training and/or testing of an ERP-BCI.

APPROACH

Mental workload was manipulated during the training or testing of a row-column P300-speller to investigate how and to what extent the spelling performance and the ERPs evoked by the oddball stimuli are affected by simultaneous mental workload.

MAIN RESULTS

Responses of certain ERP components, temporal-occipital N200 and the late reorienting negativity evoked by the oddball stimuli and the classifiability of ERP features between targets and non-targets decreased with the increase of mental workload encountered by the subject. However, the effect of mental workload on the performance of ERP-BCI was not always negative but depended on the conditions where the ERP-BCI was built and applied. The performance of ERP-BCI built under an ideal lab setting without any irrelevant mental activities declined with the increasing mental workload of the testing data. However, the performance was significantly improved when an ERP-BCI was built under an appropriate mental workload level, compared to that built under speller-only conditions.

SIGNIFICANCE

The adverse effect of concurrent mental activities may present a challenge for ERP-BCIs trained in ideal lab settings but which are to be used in daily work, especially when users are performing demanding mental processing. On the other hand, the positive effects of the mental workload of the training data suggest that introducing appropriate mental workload during training ERP-BCIs is of potential benefit to the performance in practical applications.

Cognitive workload modulation through degraded visual stimuli: a single-trial EEG study

OBJECTIVE

Our experiments explored the effect of visual stimuli degradation on cognitive workload.

APPROACH

We investigated the subjective assessment, event-related potentials (ERPs) as well as electroencephalogram (EEG) as measures of cognitive workload.

MAIN RESULTS

These experiments confirm that degradation of visual stimuli increases cognitive workload as assessed by subjective NASA task load index and confirmed by the observed P300 amplitude attenuation. Furthermore, the single-trial multi-level classification using features extracted from ERPs and EEG is found to be promising. Specifically, the adopted single-trial oscillatory EEG/ERP detection method achieved an average accuracy of 85% for discriminating 4 workload levels. Additionally, we found from the spatial patterns obtained from EEG signals that the frontal parts carry information that can be used for differentiating workload levels.

SIGNIFICANCE

Our results show that visual stimuli can modulate cognitive workload, and the modulation can be measured by the single trial EEG/ERP detection method.

A simple ERP method for quantitative analysis of cognitive workload in myoelectric prosthesis control and human-machine interaction

Common goals in the development of human-machine interface (HMI) technology are to reduce cognitive workload and increase function. However, objective and quantitative outcome measures assessing cognitive workload have not been standardized for HMI research. The present study examines the efficacy of a simple event-related potential (ERP) measure of cortical effort during myoelectric control of a virtual limb for use as an outcome tool. Participants trained and tested on two methods of control, direct control (DC) and pattern recognition control (PRC), while electroencephalographic (EEG) activity was recorded. Eighteen healthy participants with intact limbs were tested using DC and PRC under three conditions: passive viewing, easy, and hard. Novel auditory probes were presented at random intervals during testing, and significant task-difficulty effects were observed in the P200, P300, and a late positive potential (LPP), supporting the efficacy of ERPs as a cognitive workload measure in HMI tasks. LPP amplitude distinguished DC from PRC in the hard condition with higher amplitude in PRC, consistent with lower cognitive workload in PRC relative to DC for complex movements. Participants completed trials faster in the easy condition using DC relative to PRC, but completed trials more slowly using DC relative to PRC in the hard condition. The results provide promising support for ERPs as an outcome measure for cognitive workload in HMI research such as prosthetics, exoskeletons, and other assistive devices, and can be used to evaluate and guide new technologies for more intuitive HMI control.

Reduced neural error signaling in left inferior prefrontal cortex in young adults with ADHD

OBJECTIVE

The neural network involved in inhibition of inappropriate response tendencies shares commonalities with the error-processing network, signaling failure of inhibition. Most studies on error processing in ADHD have been conducted in children using electrophysiological methods.

METHOD

Using event-related functional magnetic resonance imaging, the authors studied 14 adults with ADHD and 12 group-matched healthy control participants while performing a modified version of a combined Eriksen Flanker-Go/NoGo-task.

RESULTS

Patients with ADHD demonstrated significantly reduced error signaling in the left inferior frontal gyrus bordering the anterior insular cortex (BA 47), computed from the contrast of unsuccessful minus successful inhibition trials.

CONCLUSION

Hypoactivation of the left inferior frontal cortex during error signaling might represent a neurofunctional marker of a crucial prerequisite for error processing in adults with ADHD. This possibly indicates a dysfunction of the neural system that operates task-set related representations and monitoring of erroneous performances in service of ensuing posterror processing.

Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload

While studies exist that compare different physiological variables with respect to their association with mental workload, it is still largely unclear which variables supply the best information about momentary workload of an individual and what is the benefit of combining them. We investigated workload using the n-back task, controlling for body movements and visual input. We recorded EEG, skin conductance, respiration, ECG, pupil size and eye blinks of 14 subjects. Various variables were extracted from these recordings and used as features in individually tuned classification models. Online classification was simulated by using the first part of the data as training set and the last part of the data for testing the models. The results indicate that EEG performs best, followed by eye related measures and peripheral physiology. Combining variables from different sensors did not significantly improve workload assessment over the best performing sensor alone. Best classification accuracy, a little over 90%, was reached for distinguishing between high and low workload on the basis of 2 min segments of EEG and eye related variables. A similar and not significantly different performance of 86% was reached using only EEG from single electrode location Pz.

The auditory N1 amplitude for task-irrelevant probes reflects visual interest

The present study examined the relationship between the amplitude of N1 component of event-related potentials (ERPs) elicited by task-irrelevant auditory probes and the observer's level of interest in co-occurring visual stimuli. Participants watched short animated video clips (about 400 s) played either forward (interesting) or backward (boring) accompanied by task-irrelevant sequence of auditory probes. The tone frequency of probes was fixed in a monotonous sequence condition but randomly varied in a variable sequence condition. The mean stimulus onset asynchrony of probes was 600 ms in both sequence conditions. Results showed that the N1 amplitude for probes in the variable sequence condition became smaller when participants watched interesting animated videos compared with their watching boring ones; a parallel effect was not observed in the monotonous sequence condition. Furthermore, analysis of sub-blocks (i.e., 360 s of the analysis time window for each animated video was divided into 20 s × 18 sub-blocks) showed a significant correlation between the forward-minus-backward differences in scored interest levels with the N1 amplitude in the variable sequence condition. This finding points to the possibility that the observer's interest can be estimated by neurophysiological data just for 20 s. The present study should remarkably extend the usability of the task-irrelevant probe technique.

Electrophysiological measurement of interest during walking in a simulated environment

A reliable neuroscientific technique for objectively estimating the degree of interest in a real environment is currently required in the research fields of neuroergonomics and neuroeconomics. Toward the development of such a technique, the present study explored electrophysiological measures that reflect an observer's interest in a nearly-real visual environment. Participants were asked to walk through a simulated shopping mall and the attractiveness of the shopping mall was manipulated by opening and closing the shutters of stores. During the walking task, participants were exposed to task-irrelevant auditory probes (two-stimulus oddball sequence). The results showed a smaller P2/early P3a component of task-irrelevant auditory event-related potentials and a larger lambda response of eye-fixation-related potentials in an interesting environment (i.e., open-shutter condition) than in a boring environment (i.e., closed-shutter condition); these findings can be reasonably explained by supposing that participants allocated more attentional resources to visual information in an interesting environment than in a boring environment, and thus residual attentional resources that could be allocated to task-irrelevant auditory probes were reduced. The P2/early P3a component and the lambda response may be useful measures of interest in a real visual environment.

Novel sounds as a psychophysiological measure of listening effort in older listeners with and without hearing loss

OBJECTIVE

To investigate whether task-irrelevant novel sounds presented during an auditory task can provide information about the level of listening effort.

METHODS

Event-related potentials (ERPs) were recorded for novel sounds presented during two Experiments, a frequency discrimination task and a speech-perception-in-noise (SPIN) test, each with varying degrees of task difficulty (easy, medium, hard). Difficulty was adjusted to the individual frequency discrimination threshold and 50% speech recognition signal-to-noise ratio (SNR), respectively. Older listeners (age range 60-86 years) with either normal hearing for their age or a mild-to-moderate hearing loss participated.

RESULTS

Amplitudes of Novelty P3 and late positive potential (LPP) increased with increasing task difficulty, whereas amplitudes of N1 and N2 decreased. Participants with hearing loss had significantly larger LPP amplitudes in the easy condition of the SPIN test than did normal-hearing listeners. Most correlations between ERP amplitudes and behavioral data were not significant suggesting that listening effort is not a simple equivalent of behavioral performance.

CONCLUSIONS

LPP amplitude appeared to be the most sensitive component for capturing listening effort reflecting the arousal level of the listener.

SIGNIFICANCE

ERPs to novel sounds could be easily recorded during hearing tests and provide an objective physiological measure of listening effort, thus supplementing behavioral performance data.

The pattern of the electromagnetic field emitted by mobile phones in motor vehicle driving simulators

INTRODUCTION

The paper reports the results of the determinations of UMTS EMF distributions in the driver’s cab of motor vehicle simulators. The results will serve as the basis for future research on the influence of EMF emitted by mobile phones on driver physiology.

MATERIALS AND METHODS

Two motor vehicle driving simulators were monitored, while an EMF source was placed at the driver's head or on the dashboard of the motor vehicle driving simulator. For every applied configuration, the maximal electric field strength was measured, as were the values at 16 points corresponding to chosen locations on a driver's or passenger's body.

RESULTS

When the power was set for the maximum (49 mW), a value of 27 V/m was measured in the vicinity of the driver's head when the phone was close to the head. With the same power, when the phone was placed on the dashboard, the measured maximum was 15.2 V/m in the vicinity of the driver's foot. Similar results were obtained for the passenger. Significant perturbations in EMF distribution and an increase in electric field strength values in the mo-tor vehicle driving simulator were also observed in comparison to free space measurements, and the electric field strength was up to 3 times higher inside the simulator.

CONCLUSIONS

This study can act as the basis of future studies concerning the influence of the EMF emitted by mobile phones on the physiology of the driver. Additionally, the authors postulate that it is advisable to keep mobile phones at a distance from the head, i.e. use, whenever possible, hands-free kits to reduce EMF exposure, both for drivers and passengers.

Task difficulty affects the predictive process indexed by visual mismatch negativity

Visual mismatch negativity (MMN) is an event-related brain potential (ERP) component that is elicited by prediction-incongruent events in successive visual stimulation. Previous oddball studies have shown that visual MMN in response to task-irrelevant deviant stimuli is insensitive to the manipulation of task difficulty, which supports the notion that visual MMN reflects attention-independent predictive processes. In these studies, however, visual MMN was evaluated in deviant-minus-standard difference waves, which may lead to an underestimation of the effects of task difficulty due to the possible superposition of N1-difference reflecting refractory effects. In the present study, we investigated the effects of task difficulty on visual MMN, less contaminated by N1-difference. While the participant performed a size-change detection task regarding a continuously-presented central fixation circle, we presented oddball sequences consisting of deviant and standard bar stimuli with different orientations (9.1 and 90.9%) and equiprobable sequences consisting of 11 types of control bar stimuli with different orientations (9.1% each) at the surrounding visual fields. Task difficulty was manipulated by varying the magnitude of the size-change. We found that the peak latencies of visual MMN evaluated in the deviant-minus-control difference waves were delayed as a function of task difficulty. Therefore, in contrast to the previous understanding, the present findings support the notion that visual MMN is associated with attention-demanding predictive processes.