Effects of Neural Mechanisms of Pretask Resting EEG Alpha Information on Situational Awareness: A Functional Connectivity Approach

An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems

OBJECTIVE

This study developed a fixation-related electroencephalography band power (FRBP) approach for situation awareness (SA) assessment in automated driving.

BACKGROUND

Maintaining good SA in Level 3 automated vehicles is crucial to drivers' takeover performance when the automated system fails. A multimodal fusion approach that enables the analysis of the visual behavioral and cognitive processes of SA can facilitate real-time assessment of SA in future driver state monitoring systems.

METHOD

Thirty participants performed three simulated automated driving tasks. After each task, the Situation Awareness Global Assessment Technique (SAGAT) was deployed to capture their SA about key elements that could affect their takeover task performance. Participants eye movements and brain activities were recorded. Data on their brain activity after each eye fixation on the key elements were extracted and labeled according to the correctness of the SAGAT. Mixed-effects models were used to identify brain regions that were indicative of SA, and machine learning models for SA assessment were developed based on the identified brain regions.

RESULTS

Participants' alpha and theta oscillation at frontal and temporal areas are indicative of SA. In addition, the FRBP technique can be used to predict drivers' SA with an accuracy of 88% using a neural network model.

CONCLUSION

The FRBP technique, which incorporates eye movements and brain activities, can provide more comprehensive evaluation of SA. Findings highlight the potential of utilizing FRBP to monitor drivers' SA in real-time.

APPLICATION

The proposed framework can be expanded and applied to driver state monitoring systems to measure human SA in real-world driving.

The Effects of Auditory Working Memory Task on Situation Awareness in Complex Dynamic Environments: An Eye-movement Study

OBJECTIVE

This study investigated the effect of auditory working memory task on situation awareness (SA) and eye-movement patterns in complex dynamic environments.

BACKGROUND

Many human errors in aviation are caused by a lack of SA, and distraction from auditory secondary tasks is a serious threat to SA. However, it remains unclear how auditory working memory tasks affect SA and eye-movement patterns.

METHOD

Participants (n = 28) were randomly allocated to two groups and received different periods of visual search training (short versus long). They subsequently completed a situation awareness measurement task in three auditory secondary task conditions (without secondary task, auditory calculation task, and auditory 2-back task). Eye-movement data were collected during the situation awareness measurement task.

RESULTS

The auditory 2-back task significantly reduced overall SA, Level 1 SA, dwell times, and total percentage of fixation time on task-related areas of interests in the SA measurement task. Overall SA and Level 3 SA were not reduced by the auditory 2-back task in individuals in the longer visual search training time condition.

CONCLUSION

Auditory working memory load impairs SA in the perception and projection stage; however, greater experience can overcome impairment of SA in the projection stage.

APPLICATION

This study provided possible approaches to preventing loss of SA: (1) improving crew members' communication skills to ensure the accurate and clear transmission of information, reducing the difficulty of processing information, and (2) providing targeted cognitive training tailored to each pilot's level of experience.

Evidence of elevated situational awareness for active duty soldiers during navigation of a virtual environment

U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians' physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups' physiological or cognitive markers and performance for these groups.

Recognizing the situation awareness of forklift operators based on EEG techniques in a field experiment

Lack of situation awareness (SA) is the primary cause of human errors when operating forklifts, so determining the SA level of the forklift operator is crucial to the safety of forklift operations. An EEG recognition approach of forklift operator SA in actual settings was presented in order to address the issues with invasiveness, subjectivity, and intermittency of existing measuring methods. In this paper, we conducted a field experiment that mimicked a typical forklift operation scenario to verify the differences in EEG states of forklift operators with different SA levels and investigate the correlation of multi-band combination features of each brain region of forklift operators with SA. Based on the sensitive EEG combination indexes, Support Vector Mechanism was used to construct a forklift operator SA recognition model. The results revealed that there were differences between forklift operators with high and low SA in the θ, α, and β frequency bands in zones F, C, P, and O; combined EEG indicators θ/β, (α + θ)/(α + β), and θ/(α + β) in zones F, P, and C were significantly correlated with SA; the recognition accuracy of the model reached 88.64% in the case of combined EEG indicators of zones C & F & P as input. It could provide a reference for SA measurement, contributing to the improvement of SA.

Physiological Measurements of Situation Awareness: A Systematic Review

OBJECTIVE

The goal of this systematic literature review is to investigate the relationship between indirect physiological measurements and direct measures of situation awareness (SA).

BACKGROUND

Across different environments and tasks, assessments of SA are often performed using techniques designed specifically to directly measure SA, such as SAGAT, SPAM, and/or SART. However, research suggests that indirect physiological sensing methods may also be capable of predicting SA. Currently, it is unclear which particular physiological approaches are sensitive to changes in SA.

METHOD

Seven databases were searched using the PRISMA reporting guidelines. Eligibility criteria included human-subject experiments that used at least one direct SA assessment technique, as well as at least one physiological measurement. Information extracted from each article was the physiological metric(s), the direct SA measurement(s), the correlation between these two metrics, and the experimental task(s). All studies underwent a quality assessment.

RESULTS

Twenty-five articles were included in this review. Eye tracking techniques were the most commonly used physiological measures, and correlations between conscious aspects of eye movement measures and direct SA scores were observed. Evidence for cardiovascular predictors of SA were mixed. EEG studies were too few to form strong conclusions, but were consistently positive.

CONCLUSION

Further investigation is needed to methodically collect more relevant data and comprehensively model the relationships between a wider range of physiological measurements and direct assessments of SA.

APPLICATION

This review will guide researchers and practitioners in methods to indirectly assess SA with sensors and highlight opportunities for future research on wearables and SA.

Hemodynamic functional connectivity optimization of frequency EEG microstates enables attention LSTM framework to classify distinct temporal cortical communications of different cognitive tasks

Temporal analysis of global cortical communication of cognitive tasks in coarse EEG information is still challenging due to the underlying complex neural mechanisms. This study proposes an attention-based time-series deep learning framework that processes fMRI functional connectivity optimized quasi-stable frequency microstates for classifying distinct temporal cortical communications of the cognitive task. Seventy volunteers were subjected to visual target detection tasks, and their electroencephalogram (EEG) and functional MRI (fMRI) were acquired simultaneously. At first, the acquired EEG information was preprocessed and bandpass to delta, theta, alpha, beta, and gamma bands and then subjected to quasi-stable frequency-microstate estimation. Subsequently, time-series elicitation of each frequency microstates is optimized with graph theory measures of simultaneously eliciting fMRI functional connectivity between frontal, parietal, and temporal cortices. The distinct neural mechanisms associated with each optimized frequency-microstate were analyzed using microstate-informed fMRI. Finally, these optimized, quasi-stable frequency microstates were employed to train and validate the attention-based Long Short-Term Memory (LSTM) time-series architecture for classifying distinct temporal cortical communications of the target from other cognitive tasks. The temporal, sliding input sampling windows were chosen between 180 to 750 ms/segment based on the stability of transition probabilities of the optimized microstates. The results revealed 12 distinct frequency microstates capable of deciphering target detections' temporal cortical communications from other task engagements. Particularly, fMRI functional connectivity measures of target engagement were observed significantly correlated with the right-diagonal delta (r = 0.31), anterior-posterior theta (r = 0.35), left-right theta (r = - 0.32), alpha (r = - 0.31) microstates. Further, neuro-vascular information of microstate-informed fMRI analysis revealed the association of delta/theta and alpha/beta microstates with cortical communications and local neural processing, respectively. The classification accuracies of the attention-based LSTM were higher than the traditional LSTM architectures, particularly the frameworks that sampled the EEG data with a temporal width of 300 ms/segment. In conclusion, the study demonstrates reliable temporal classifications of global cortical communication of distinct tasks using an attention-based LSTM utilizing fMRI functional connectivity optimized quasi-stable frequency microstates.