Passive BCI beyond the lab: current trends and future directions

Evaluating robotic actions: spatiotemporal brain dynamics of performance assessment in robot-assisted laparoscopic training

Introduction

Enhancing medical robot training traditionally relies on explicit feedback from physicians to identify optimal and suboptimal robotic actions during surgery. Passive brain-computer interfaces (BCIs) offer an emerging alternative by enabling implicit brain-based performance evaluations. However, effectively decoding these evaluations of robot performance requires a comprehensive understanding of the spatiotemporal brain dynamics identifying optimal and suboptimal robot actions within realistic settings.

Methods

We conducted an electroencephalographic study with 16 participants who mentally assessed the quality of robotic actions while observing simulated robot-assisted laparoscopic surgery scenarios designed to approximate real-world conditions. We aimed to identify key spatiotemporal dynamics using the surface Laplacian technique and two complementary data-driven methods: a mass-univariate permutation-based clustering and multivariate pattern analysis (MVPA)-based temporal decoding. A second goal was to identify the optimal time interval of evoked brain signatures for single-trial classification.

Results

Our analyses revealed three distinct spatiotemporal brain dynamics differentiating the quality assessment of optimal vs. suboptimal robotic actions during video-based laparoscopic training observations. Specifically, an enhanced left fronto-temporal current source, consistent with P300, LPP, and P600 components, indicated heightened attentional allocation and sustained evaluation processes during suboptimal robot actions. Additionally, amplified current sinks in right frontal and mid-occipito-parietal regions suggested prediction-based processing and conflict detection, consistent with the oERN and interaction-based ERN/N400. Both mass-univariate clustering and MVPA provided convergent evidence supporting these neural distinctions.

Discussion

The identified neural signatures propose that suboptimal robotic actions elicit enhanced, sustained brain dynamics linked to continuous attention allocation, action monitoring, conflict detection, and ongoing evaluative processing. The findings highlight the importance of prioritizing late evaluative brain signatures in BCIs to classify robotic actions reliably. These insights have significant implications for advancing machine-learning-based training paradigms.

Systematic review of experimental paradigms and deep neural networks for electroencephalography-based cognitive workload detection

This article summarizes a systematic literature review of deep neural network-based cognitive workload (CWL) estimation from electroencephalographic (EEG) signals. The focus of this article can be delineated into two main elements: first is the identification of experimental paradigms prevalently employed for CWL induction, and second, is an inquiry about the data structure and input formulations commonly utilized in deep neural networks (DNN)-based CWL detection. The survey revealed several experimental paradigms that can reliably induce either graded levels of CWL or a desired cognitive state due to sustained induction of CWL. This article has characterized them with respect to the number of distinct CWL levels, cognitive states, experimental environment, and agents in focus. Further, this literature analysis found that DNNs can successfully detect distinct levels of CWL despite the inter-subject and inter-session variability typically observed in EEG signals. Several methodologies were found using EEG signals in its native representation of a two-dimensional matrix as input to the classification algorithm, bypassing traditional feature selection steps. More often than not, researchers used DNNs as black-box type models, and only a few studies employed interpretable or explainable DNNs for CWL detection. However, these algorithms were mostly post hoc data analysis and classification schemes, and only a few studies adopted real-time CWL estimation methodologies. Further, it has been suggested that using interpretable deep learning methodologies may shed light on EEG correlates of CWL, but this remains mostly an unexplored area. This systematic review suggests using networks sensitive to temporal dependencies and appropriate input formulations for each type of DNN architecture to achieve robust classification performance. An additional suggestion is to utilize transfer learning methods to achieve high generalizability across tasks (task-independent classifiers), while simple cross-subject data pooling may achieve the same for subject-independent classifiers.

Optimizing EEG Signal Integrity: A Comprehensive Guide to Ocular Artifact Correction

Ocular artifacts, including blinks and saccades, pose significant challenges in the analysis of electroencephalographic (EEG) data, often obscuring crucial neural signals. This tutorial provides a comprehensive guide to the most effective methods for correcting these artifacts, with a focus on algorithms designed for both laboratory and real-world settings. We review traditional approaches, such as regression-based techniques and Independent Component Analysis (ICA), alongside more advanced methods like Artifact Subspace Reconstruction (ASR) and deep learning-based algorithms. Through detailed step-by-step instructions and comparative analysis, this tutorial equips researchers with the tools necessary to maintain the integrity of EEG data, ensuring accurate and reliable results in neurophysiological studies. The strategies discussed are particularly relevant for wearable EEG systems and real-time applications, reflecting the growing demand for robust and adaptable solutions in applied neuroscience.

o-CLEAN: a novel multi-stage algorithm for the ocular artifacts' correction from EEG data in out-of-the-lab applications

In the context of electroencephalographic (EEG) signal processing, artifacts generated by ocular movements, such as blinks, are significant confounding factors. These artifacts overwhelm informative EEG features and may occur too frequently to simply remove affected epochs without losing valuable data. Correcting these artifacts remains a challenge, particularly in out-of-lab and online applications using wearable EEG systems (i.e. with low number of EEG channels, without any additional channels to track EOG).Objective.The main objective of the present work consisted in validating a novel ocular blinks artefacts correction method, named multi-stage OCuLar artEfActs deNoising algorithm (o-CLEAN), suitable for online processing with minimal EEG channels.Approach.The research was conducted considering one EEG dataset collected in highly controlled environment, and a second one collected in real environment. The analysis was performed by comparing the o-CLEAN method with previously validated state-of-art techniques, and by evaluating its performance along two dimensions: (a) the ocular artefacts correction performance (IN-Blink), and (b) the EEG signal preservation when the method was applied without any ocular artefacts occurrence (OUT-Blink).Main results.Results highlighted that (i) o-CLEAN algorithm resulted to be, at least, significantly reliable as the most validated approaches identified in scientific literature in terms of ocular blink artifacts correction, (ii) o-CLEAN showed the best performances in terms of EEG signal preservation especially with a low number of EEG channels.Significance.The testing and validation of the o-CLEAN addresses a relevant open issue in bioengineering EEG processing, especially within out-of-the-lab application. In fact, the method offers an effective solution for correcting ocular artifacts in EEG signals with a low number of available channels, for online processing, and without any specific template of the EOG. It was demonstrated to be particularly effective for EEG data gathered in real environments using wearable systems, a rapidly expanding area within applied neuroscience.

The architecture of functional brain network modulated by driving under train running noise exposure

A noisy environment can considerably impact drivers' attention and fatigue, endangering driving safety. Consequently, this study designed a simulated driving experimental scenario to analyse the effects of noise generated during urban rail transit train operation on drivers' functional brain networks. The experiment recruited 16 participants, and the simulated driving scenario was conducted at noise levels of 50, 60, 70, and 80 dB. Functional connectivity between all electrode pairs across various frequency bands was evaluated using the weighted phase lag index (WPLI), and a brain network based on this was constructed. Graph theoretic analysis employed network global efficiency, degree, and clustering coefficient as metrics. Significant increases in the WPLI values of theta and alpha frequency bands were observed in high noise environments (70 dB, 80 dB), as well as enhanced brain synchronisation. Furthermore, concerning the topological metrics of brain networks, it was observed that the global efficiency of brain networks in theta and alpha frequency ranges, as well as the node degree and clustering coefficients, experienced substantial growth in high noise environments (70 dB, 80 dB) as opposed to 50 dB and 60 dB. This finding indicates that high-noise environments impact the reorganisation of functional brain networks, leading to a preference for network structures with improved global efficiency. Such findings may improve our understanding of the neural mechanisms of driving under noise exposure, and thus potentially reduce road accidents to some extent.

MOVING: A Multi-Modal Dataset of EEG Signals and Virtual Glove Hand Tracking

Brain-computer interfaces (BCIs) are pivotal in translating neural activities into control commands for external assistive devices. Non-invasive techniques like electroencephalography (EEG) offer a balance of sensitivity and spatial-temporal resolution for capturing brain signals associated with motor activities. This work introduces MOVING, a Multi-Modal dataset of EEG signals and Virtual Glove Hand Tracking. This dataset comprises neural EEG signals and kinematic data associated with three hand movements-open/close, finger tapping, and wrist rotation-along with a rest period. The dataset, obtained from 11 subjects using a 32-channel dry wireless EEG system, also includes synchronized kinematic data captured by a Virtual Glove (VG) system equipped with two orthogonal Leap Motion Controllers. The use of these two devices allows for fast assembly (∼1 min), although introducing more noise than the gold standard devices for data acquisition. The study investigates which frequency bands in EEG signals are the most informative for motor task classification and the impact of baseline reduction on gesture recognition. Deep learning techniques, particularly EEGnetV4, are applied to analyze and classify movements based on the EEG data. This dataset aims to facilitate advances in BCI research and in the development of assistive devices for people with impaired hand mobility. This study contributes to the repository of EEG datasets, which is continuously increasing with data from other subjects, which is hoped to serve as benchmarks for new BCI approaches and applications.

Several inaccurate or erroneous conceptions and misleading propaganda about brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction, which has potential applications for specific individuals or groups in specific scenarios. Extensive research has been conducted on the principles and implementation methods of BCI, and efforts are currently being made to bridge the gap from research to real-world applications. However, there are inaccurate or erroneous conceptions about BCI among some members of the public, and certain media outlets, as well as some BCI researchers, developers, manufacturers, and regulators, propagate misleading or overhyped claims about BCI technology. Therefore, this article summarizes the several misconceptions and misleading propaganda about BCI, including BCI being capable of "mind-controlled," "controlling brain," "mind reading," and the ability to "download" or "upload" information from or to the brain using BCI, among others. Finally, the limitations (shortcomings) and limits (boundaries) of BCI, as well as the necessity of conducting research aimed at countering BCI systems are discussed, and several suggestions are offered to reduce misconceptions and misleading claims about BCI.

EEG-based finger movement classification with intrinsic time-scale decomposition

Introduction

Brain-computer interfaces (BCIs) are systems that acquire the brain's electrical activity and provide control of external devices. Since electroencephalography (EEG) is the simplest non-invasive method to capture the brain's electrical activity, EEG-based BCIs are very popular designs. Aside from classifying the extremity movements, recent BCI studies have focused on the accurate coding of the finger movements on the same hand through their classification by employing machine learning techniques. State-of-the-art studies were interested in coding five finger movements by neglecting the brain's idle case (i.e., the state that brain is not performing any mental tasks). This may easily cause more false positives and degrade the classification performances dramatically, thus, the performance of BCIs. This study aims to propose a more realistic system to decode the movements of five fingers and the no mental task (NoMT) case from EEG signals.

Methods

In this study, a novel praxis for feature extraction is utilized. Using Proper Rotational Components (PRCs) computed through Intrinsic Time Scale Decomposition (ITD), which has been successfully applied in different biomedical signals recently, features for classification are extracted. Subsequently, these features were applied to the inputs of well-known classifiers and their different implementations to discriminate between these six classes. The highest classifier performances obtained in both subject-independent and subject-dependent cases were reported. In addition, the ANOVA-based feature selection was examined to determine whether statistically significant features have an impact on the classifier performances or not.

Results

As a result, the Ensemble Learning classifier achieved the highest accuracy of 55.0% among the tested classifiers, and ANOVA-based feature selection increases the performance of classifiers on five-finger movement determination in EEG-based BCI systems.

Discussion

When compared with similar studies, proposed praxis achieved a modest yet significant improvement in classification performance although the number of classes was incremented by one (i.e., NoMT).

A Neuroergonomic Approach Fostered by Wearable EEG for the Multimodal Assessment of Drivers Trainees

When assessing trainees' progresses during a driving training program, instructors can only rely on the evaluation of a trainee's explicit behavior and their performance, without having any insight about the training effects at a cognitive level. However, being able to drive does not imply knowing how to drive safely in a complex scenario such as the road traffic. Indeed, the latter point involves mental aspects, such as the ability to manage and allocate one's mental effort appropriately, which are difficult to assess objectively. In this scenario, this study investigates the validity of deploying an electroencephalographic neurometric of mental effort, obtained through a wearable electroencephalographic device, to improve the assessment of the trainee. The study engaged 22 young people, without or with limited driving experience. They were asked to drive along five different but similar urban routes, while their brain activity was recorded through electroencephalography. Moreover, driving performance, subjective and reaction times measures were collected for a multimodal analysis. In terms of subjective and performance measures, no driving improvement could be detected either through the driver's subjective measures or through their driving performance. On the other side, through the electroencephalographic neurometric of mental effort, it was possible to catch their improvement in terms of mental performance, with a decrease in experienced mental demand after three repetitions of the driving training tasks. These results were confirmed by the analysis of reaction times, that significantly improved from the third repetition as well. Therefore, being able to measure when a task is less mentally demanding, and so more automatic, allows to deduce the degree of users training, becoming capable of handling additional tasks and reacting to unexpected events.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

Neurophysiological Assessment of An Innovative Maritime Safety System in Terms of Ship Operators' Mental Workload, Stress, and Attention in the Full Mission Bridge Simulator

The current industrial environment relies heavily on maritime transportation. Despite the continuous technological advances for the development of innovative safety software and hardware systems, there is a consistent gap in the scientific literature regarding the objective evaluation of the performance of maritime operators. The human factor is profoundly affected by changes in human performance or psychological state. The difficulty lies in the fact that the technology, tools, and protocols for investigating human performance are not fully mature or suitable for experimental investigation. The present research aims to integrate these two concepts by (i) objectively characterizing the psychological state of mariners, i.e., mental workload, stress, and attention, through their electroencephalographic (EEG) signal analysis, and (ii) validating an innovative safety framework countermeasure, defined as Human Risk-Informed Design (HURID), through the aforementioned neurophysiological approach. The proposed study involved 26 mariners within a high-fidelity bridge simulator while encountering collision risk in congested waters with and without the HURID. Subjective, behavioral, and neurophysiological data, i.e., EEG, were collected throughout the experimental activities. The results showed that the participants experienced a statistically significant higher mental workload and stress while performing the maritime activities without the HURID, while their attention level was statistically lower compared to the condition in which they performed the experiments with the HURID (all p < 0.05). Therefore, the presented study confirmed the effectiveness of the HURID during maritime operations in critical scenarios and led the way to extend the neurophysiological evaluation of the HFs of maritime operators during the performance of critical and/or standard shipboard tasks.

Temporal-spatial convolutional residual network for decoding attempted movement related EEG signals of subjects with spinal cord injury

Brain Computer Interface (BCI) offers a promising approach to restoring hand functionality for people with cervical spinal cord injury (SCI). A reliable classification of brain activities based on appropriate flexibility in feature extraction could enhance BCI systems performance. In the present study, based on convolutional layers with temporal-spatial, Separable and Depthwise structures, we develop Temporal-Spatial Convolutional Residual Network)TSCR-Net(and Temporal-Spatial Convolutional Iterative Residual Network)TSCIR-Net(structures to classify electroencephalogram (EEG) signals. Using EEG signals in five different hand movement classes of SCI people, we compare the effectiveness of TSCIR-Net and TSCR-Net models with some competitive methods. We use the bayesian hyperparameter optimization algorithm to tune the hyperparameters of compact convolutional neural networks. In order to show the high generalizability of the proposed models, we compare the results of the models in different frequency ranges. Our proposed models decoded distinctive characteristics of different movement efforts and obtained higher classification accuracy than previous deep neural networks. Our findings indicate that TSCIR-Net and TSCR-Net models fulfills a better classification accuracy of 71.11%, and 64.55% for EEG_All and 57.74%, and 67.87% for EEG_Low frequency data sets than the compared methods in the literature.

Diversity and Suitability of the State-of-the-Art Wearable and Wireless EEG Systems Review

Wireless electroencephalography (EEG) systems have been attracting increasing attention in recent times. Both the number of articles discussing wireless EEG and their proportion relative to general EEG publications have increased over years. These trends indicate that wireless EEG systems could be more accessible to researchers and the research community has recognized the potential of wireless EEG systems. To explore the development and diverse applications of wireless EEG systems, this review highlights the trends in wearable and wireless EEG systems over the past decade and compares the specifications and research applications of the major wireless systems marketed by 16 companies. For each product, five parameters (number of channels, sampling rate, cost, battery life, and resolution) were assessed for comparison. Currently, these wearable and portable wireless EEG systems have three main application areas: consumer, clinical, and research. To address this multitude of options, the article also discussed the thought process to find a suitable device that meets personalization and use cases specificities. These investigations suggest that low-price and convenience are key factors for consumer applications, wireless EEG systems with FDA or CE-certification may be more suitable for clinical settings, and devices that provide raw EEG data with high-density channels are important for laboratory research. This article presents an overview of the current state of the wireless EEG systems specifications and possible applications and serves as a guide point as it is expected that more influential and novel research will cyclically promote the development of such EEG systems.

Spatial-temporal features-based EEG emotion recognition using graph convolution network and long short-term memory

Objective. Emotion recognition on the basis of electroencephalography (EEG) signals has received a significant amount of attention in the areas of cognitive science and human-computer interaction (HCI). However, most existing studies either focus on one-dimensional EEG data, ignoring the relationship between channels, or only extract time-frequency features while not involving spatial features.Approach. We develop spatial-temporal features-based EEG emotion recognition using a graph convolution network (GCN) and long short-term memory (LSTM), named ERGL. First, the one-dimensional EEG vector is converted into a two-dimensional mesh matrix, so that the matrix configuration corresponds to the distribution of brain regions at EEG electrode locations, thus to represent the spatial correlation between multiple adjacent channels in a better way. Second, the GCN and LSTM are employed together to extract spatial-temporal features; the GCN is used to extract spatial features, while LSTM units are applied to extract temporal features. Finally, a softmax layer is applied to emotion classification.Main results. Extensive experiments are conducted on the A Dataset for Emotion Analysis using Physiological Signals (DEAP) and the SJTU Emotion EEG Dataset (SEED). The classification results of accuracy, precision, and F-score for valence and arousal dimensions on DEAP achieved 90.67% and 90.33%, 92.38% and 91.72%, and 91.34% and 90.86%, respectively. The accuracy, precision, and F-score of positive, neutral, and negative classifications reached 94.92%, 95.34%, and 94.17%, respectively, on the SEED dataset.Significance. The above results demonstrate that the proposed ERGL method is encouraging in comparison to state-of-the-art recognition research.

Mind wandering state detection during video-based learning via EEG

The aim of this study is to explore the potential of technology for detecting mind wandering, particularly during video-based distance learning, with the ultimate benefit of improving learning outcomes. To overcome the challenges of previous mind wandering research in ecological validity, sample balance, and dataset size, this study utilized practical electroencephalography (EEG) recording hardware and designed a paradigm consisting of viewing short-duration video lectures under a focused learning condition and a future planning condition. Participants estimated statistics of their attentional state at the end of each video, and we combined this rating scale feedback with self-caught key press responses during video watching to obtain binary labels for classifier training. EEG was recorded using an 8-channel system, and spatial covariance features processed by Riemannian geometry were employed. The results demonstrate that a radial basis function kernel support vector machine classifier, using Riemannian-processed covariance features from delta, theta, alpha, and beta bands, can detect mind wandering with a mean area under the receiver operating characteristic curve (AUC) of 0.876 for within-participant classification and AUC of 0.703 for cross-lecture classification. Furthermore, our results suggest that a short duration of training data is sufficient to train a classifier for online decoding, as cross-lecture classification remained at an average AUC of 0.689 when using 70% of the training set (about 9 min). The findings highlight the potential for practical EEG hardware in detecting mind wandering with high accuracy, which has potential application to improving learning outcomes during video-based distance learning.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Consciousness: Matter or EMF?

Conventional theories of consciousness (ToCs) that assume that the substrate of consciousness is the brain's neuronal matter fail to account for fundamental features of consciousness, such as the binding problem. Field ToC's propose that the substrate of consciousness is the brain's best accounted by some kind of field in the brain. Electromagnetic (EM) ToCs propose that the conscious field is the brain's well-known EM field. EM-ToCs were first proposed only around 20 years ago primarily to account for the experimental discovery that synchronous neuronal firing was the strongest neural correlate of consciousness (NCC). Although EM-ToCs are gaining increasing support, they remain controversial and are often ignored by neurobiologists and philosophers and passed over in most published reviews of consciousness. In this review I examine EM-ToCs against established criteria for distinguishing between ToCs and demonstrate that they outperform all conventional ToCs and provide novel insights into the nature of consciousness as well as a feasible route toward building artificial consciousnesses.

Neurophysiological Evaluation of Students' Experience during Remote and Face-to-Face Lessons: A Case Study at Driving School

Nowadays, fostered by technological progress and contextual circumstances such as the economic crisis and pandemic restrictions, remote education is experiencing growing deployment. However, this growth has generated widespread doubts about the actual effectiveness of remote/online learning compared to face-to-face education. The present study was aimed at comparing face-to-face and remote education through a multimodal neurophysiological approach. It involved forty students at a driving school, in a real classroom, experiencing both modalities. Wearable devices to measure brain, ocular, heart and sweating activities were employed in order to analyse the students' neurophysiological signals to obtain insights into the cognitive dimension. In particular, four parameters were considered: the Eye Blink Rate, the Heart Rate and its Variability and the Skin Conductance Level. In addition, the students filled out a questionnaire at the end to obtain an explicit measure of their learning performance. Data analysis showed higher cognitive activity, in terms of attention and mental engagement, in the in-presence setting compared to the remote modality. On the other hand, students in the remote class felt more stressed, particularly during the first part of the lesson. The analysis of questionnaires demonstrated worse performance for the remote group, thus suggesting a common "disengaging" behaviour when attending remote courses, thus undermining their effectiveness. In conclusion, neuroscientific tools could help to obtain insights into mental concerns, often "blind", such as decreasing attention and increasing stress, as well as their dynamics during the lesson itself, thus allowing the definition of proper countermeasures to emerging issues when introducing new practices into daily life.

Estimating distribution shifts for predicting cross-subject generalization in electroencephalography-based mental workload assessment

Assessment of mental workload in real-world conditions is key to ensuring the performance of workers executing tasks that demand sustained attention. Previous literature has employed electroencephalography (EEG) to this end despite having observed that EEG correlates of mental workload vary across subjects and physical strain, thus making it difficult to devise models capable of simultaneously presenting reliable performance across users. Domain adaptation consists of a set of strategies that aim at allowing for improving machine learning systems performance on unseen data at training time. Such methods, however, might rely on assumptions over the considered data distributions, which typically do not hold for applications of EEG data. Motivated by this observation, in this work we propose a strategy to estimate two types of discrepancies between multiple data distributions, namely marginal and conditional shifts, observed on data collected from different subjects. Besides shedding light on the assumptions that hold for a particular dataset, the estimates of statistical shifts obtained with the proposed approach can be used for investigating other aspects of a machine learning pipeline, such as quantitatively assessing the effectiveness of domain adaptation strategies. In particular, we consider EEG data collected from individuals performing mental tasks while running on a treadmill and pedaling on a stationary bike and explore the effects of different normalization strategies commonly used to mitigate cross-subject variability. We show the effects that different normalization schemes have on statistical shifts and their relationship with the accuracy of mental workload prediction as assessed on unseen participants at training time.

RNeuMark: A Riemannian EEG Analysis Framework for Neuromarketing

Neuromarketing exploits neuroimaging techniques so as to reinforce the predictive power of conventional marketing tools, like questionnaires and focus groups. Electroencephalography (EEG) is the most commonly encountered neuroimaging technique due to its non-invasiveness, low-cost, and its very recent embedding in wearable devices. The transcription of brainwave patterns to consumer attitude is supported by various signal descriptors, while the quest for profitable novel ways is still an open research question. Here, we suggest the use of sample covariance matrices as alternative descriptors, that encapsulate the coordinated neural activity from distinct brain areas, and the adoption of Riemannian geometry for their handling. We first establish the suitability of Riemannian approach for neuromarketing-related problems and then suggest a relevant decoding scheme for predicting consumers' choices (e.g., willing to buy or not a specific product). Since the decision-making process involves the concurrent interaction of various cognitive processes and consequently of distinct brain rhythms, the proposed decoder takes the form of an ensemble classifier that builds upon a multi-view perspective, with each view dedicated to a specific frequency band. Adopting a standard machine learning procedure, and using a set of trials (training data) in conjunction with the associated behavior labels ("buy"/ "not buy"), we train a battery of classifiers accordingly. Each classifier is designed to operate in the space recovered from the inter-trial distances of SCMs and to cast a rhythm-depended decision that is eventually combined with the predictions of the rest ones. The demonstration and evaluation of the proposed approach are performed in 2 neuromarketing-related datasets of different nature. The first is employed to showcase the potential of the suggested descriptor, while the second to showcase the decoder's superiority against popular alternatives in the field.

AtAwAR Translate: Attention-Aware Language Translation Application in Augmented Reality for Mobile Phones

As lightweight, low-cost EEG headsets emerge, the feasibility of consumer-oriented brain-computer interfaces (BCI) increases. The combination of portable smartphones and easy-to-use EEG dry electrode headbands offers intriguing new applications and methods of human-computer interaction. In previous research, augmented reality (AR) scenarios have been identified to profit from additional user state information-such as that provided by a BCI. In this work, we implemented a system that integrates user attentional state awareness into a smartphone application for an AR written language translator. The attentional state of the user is classified in terms of internally and externally directed attention by using the Muse 2 electroencephalography headband with four frontal electrodes. The classification results are used to adapt the behavior of the translation app, which uses the smartphone's camera to display translated text as augmented reality elements. We present the first mobile BCI system that uses a smartphone and a low-cost EEG device with few electrodes to provide attention awareness to an AR application. Our case study with 12 participants did not fully support the assumption that the BCI improves usability. However, we are able to show that the classification accuracy and ease of setup are promising paths toward mobile consumer-oriented BCI usage. For future studies, other use cases, applications, and adaptations will be tested for this setup to explore the usability.

Evaluation of a New Lightweight EEG Technology for Translational Applications of Passive Brain-Computer Interfaces

Technologies like passive brain-computer interfaces (BCI) can enhance human-machine interaction. Anyhow, there are still shortcomings in terms of easiness of use, reliability, and generalizability that prevent passive-BCI from entering real-life situations. The current work aimed to technologically and methodologically design a new gel-free passive-BCI system for out-of-the-lab employment. The choice of the water-based electrodes and the design of a new lightweight headset met the need for easy-to-wear, comfortable, and highly acceptable technology. The proposed system showed high reliability in both laboratory and realistic settings, performing not significantly different from the gold standard based on gel electrodes. In both cases, the proposed system allowed effective discrimination (AUC > 0.9) between low and high levels of workload, vigilance, and stress even for high temporal resolution (<10 s). Finally, the generalizability of the proposed system has been tested through a cross-task calibration. The system calibrated with the data recorded during the laboratory tasks was able to discriminate the targeted human factors during the realistic task reaching AUC values higher than 0.8 at 40 s of temporal resolution in case of vigilance and workload, and 20 s of temporal resolution for the stress monitoring. These results pave the way for ecologic use of the system, where calibration data of the realistic task are difficult to obtain.

A Hybrid Hand-Crafted and Deep Neural Spatio-Temporal EEG Features Clustering Framework for Precise Emotional Status Recognition

Human emotions are variant with time, non-stationary, complex in nature, and are invoked as a result of human reactions during our daily lives. Continuously detecting human emotions from one-dimensional EEG signals is an arduous task. This paper proposes an advanced signal processing mechanism for emotion detection from EEG signals using continuous wavelet transform. The space and time components of the raw EEG signals are converted into 2D spectrograms followed by feature extraction. A hybrid spatio-temporal deep neural network is implemented to extract rich features. A differential-based entropy feature selection technique adaptively differentiates features based on entropy, based on low and high information regions. Bag of Deep Features (BoDF) is applied to create clusters of similar features and computes the features vocabularies for reduction of feature dimensionality. Extensive experiments are performed on the SEED dataset, which shows the significance of the proposed method compared to state-of-the-art methods. Specifically, the proposed model achieved 96.7%, 96.2%, 95.8%, and 95.3% accuracy with the SJTU SEED dataset, for SVM, ensemble, tree, and KNN classifiers, respectively.

A Review on the Role of Affective Stimuli in Event-Related Frontal Alpha Asymmetry

Frontal alpha asymmetry refers to the difference between the right and left alpha activity over the frontal brain region. Increased activity in the left hemisphere has been linked to approach motivation and increased activity in the right hemisphere has been linked to avoidance or withdrawal. However, research on alpha asymmetry is diverse and has shown mixed results, which may partly be explained by the potency of the used stimuli to emotionally and motivationally engage participants. This review gives an overview of the types of affective stimuli utilized with the aim to identify which stimuli elicit a strong approach-avoidance effect in an affective context. We hope this contributes to better understanding of what is reflected by alpha asymmetry, and in what circumstances it may be an informative marker of emotional state. We systematically searched the literature for studies exploring event-related frontal alpha asymmetry in affective contexts. The search resulted in 61 papers, which were categorized in five stimulus categories that were expected to differ in their potency to engage participants: images &amp; sounds, videos, real cues, games and other tasks. Studies were viewed with respect to the potency of the stimuli to evoke significant approach-avoidance effects on their own and in interaction with participant characteristics or condition. As expected, passively perceived stimuli that are multimodal or realistic, seem more potent to elicit alpha asymmetry than unimodal stimuli. Games, and other stimuli with a strong task-based component were expected to be relatively engaging but approach-avoidance effects did not seem to be much clearer than the studies using perception of videos and real cues. While multiple factors besides stimulus characteristics determine alpha asymmetry, and we did not identify a type of affective stimulus that induces alpha asymmetry highly consistently, our results indicate that strongly engaging, salient and/or personally relevant stimuli are important to induce an approach-avoidance effect.

Human Mental Workload: A Survey and a Novel Inclusive Definition

Human mental workload is arguably the most invoked multidimensional construct in Human Factors and Ergonomics, getting momentum also in Neuroscience and Neuroergonomics. Uncertainties exist in its characterization, motivating the design and development of computational models, thus recently and actively receiving support from the discipline of Computer Science. However, its role in human performance prediction is assured. This work is aimed at providing a synthesis of the current state of the art in human mental workload assessment through considerations, definitions, measurement techniques as well as applications, Findings suggest that, despite an increasing number of associated research works, a single, reliable and generally applicable framework for mental workload research does not yet appear fully established. One reason for this gap is the existence of a wide swath of operational definitions, built upon different theoretical assumptions which are rarely examined collectively. A second reason is that the three main classes of measures, which are self-report, task performance, and physiological indices, have been used in isolation or in pairs, but more rarely in conjunction all together. Multiple definitions complement each another and we propose a novel inclusive definition of mental workload to support the next generation of empirical-based research. Similarly, by comprehensively employing physiological, task-performance, and self-report measures, more robust assessments of mental workload can be achieved.

SRI-EEG: State-Based Recurrent Imputation for EEG Artifact Correction

Electroencephalogram (EEG) signals are often used as an input modality for Brain Computer Interfaces (BCIs). While EEG signals can be beneficial for numerous types of interaction scenarios in the real world, high levels of noise limits their usage to strictly noise-controlled environments such as a research laboratory. Even in a controlled environment, EEG is susceptible to noise, particularly from user motion, making it highly challenging to use EEG, and consequently BCI, as a ubiquitous user interaction modality. In this work, we address the EEG noise/artifact correction problem. Our goal is to detect physiological artifacts in EEG signal and automatically replace the detected artifacts with imputed values to enable robust EEG sensing overall requiring significantly reduced manual effort than is usual. We present a novel EEG state-based imputation model built upon a recurrent neural network, which we call SRI-EEG, and evaluate the proposed method on three publicly available EEG datasets. From quantitative and qualitative comparisons with six conventional and neural network based approaches, we demonstrate that our method achieves comparable performance to the state-of-the-art methods on the EEG artifact correction task.

A State-of-the-Art Review of EEG-Based Imagined Speech Decoding

Currently, the most used method to measure brain activity under a non-invasive procedure is the electroencephalogram (EEG). This is because of its high temporal resolution, ease of use, and safety. These signals can be used under a Brain Computer Interface (BCI) framework, which can be implemented to provide a new communication channel to people that are unable to speak due to motor disabilities or other neurological diseases. Nevertheless, EEG-based BCI systems have presented challenges to be implemented in real life situations for imagined speech recognition due to the difficulty to interpret EEG signals because of their low signal-to-noise ratio (SNR). As consequence, in order to help the researcher make a wise decision when approaching this problem, we offer a review article that sums the main findings of the most relevant studies on this subject since 2009. This review focuses mainly on the pre-processing, feature extraction, and classification techniques used by several authors, as well as the target vocabulary. Furthermore, we propose ideas that may be useful for future work in order to achieve a practical application of EEG-based BCI systems toward imagined speech decoding.

Unsupervised Clustering of Individuals Sharing Selective Attentional Focus Using Physiological Synchrony

Research on brain signals as indicators of a certain attentional state is moving from laboratory environments to everyday settings. Uncovering the attentional focus of individuals in such settings is challenging because there is usually limited information about real-world events, as well as a lack of data from the real-world context at hand that is correctly labeled with respect to individuals' attentional state. In most approaches, such data is needed to train attention monitoring models. We here investigate whether unsupervised clustering can be combined with physiological synchrony in the electroencephalogram (EEG), electrodermal activity (EDA), and heart rate to automatically identify groups of individuals sharing attentional focus without using knowledge of the sensory stimuli or attentional focus of any of the individuals. We used data from an experiment in which 26 participants listened to an audiobook interspersed with emotional sounds and beeps. Thirteen participants were instructed to focus on the narrative of the audiobook and 13 participants were instructed to focus on the interspersed emotional sounds and beeps. We used a broad range of commonly applied dimensionality reduction ordination techniques-further referred to as mappings-in combination with unsupervised clustering algorithms to identify the two groups of individuals sharing attentional focus based on physiological synchrony. Analyses were performed using the three modalities EEG, EDA, and heart rate separately, and using all possible combinations of these modalities. The best unimodal results were obtained when applying clustering algorithms on physiological synchrony data in EEG, yielding a maximum clustering accuracy of 85%. Even though the use of EDA or heart rate by itself did not lead to accuracies significantly higher than chance level, combining EEG with these measures in a multimodal approach generally resulted in higher classification accuracies than when using only EEG. Additionally, classification results of multimodal data were found to be more consistent across algorithms than unimodal data, making algorithm choice less important. Our finding that unsupervised classification into attentional groups is possible is important to support studies on attentional engagement in everyday settings.

A brief review of motor imagery and bimanual coordination

Motor imagery is increasingly being used in clinical settings, such as in neurorehabilitation and brain computer interface (BCI). In stroke, patients lose upper limb function and must re-learn bimanual coordination skills necessary for the activities of daily living. Physiotherapists integrate motor imagery with physical rehabilitation to accelerate recovery. In BCIs, users are often asked to imagine a movement, often with sparse instructions. The EEG pattern that coincides with this cognitive task is captured, then used to execute an external command, such as operating a neuroprosthetic device. As such, BCIs are dependent on the efficient and reliable interpretation of motor imagery. While motor imagery improves patient outcome and informs BCI research, the cognitive and neurophysiological mechanisms which underlie it are not clear. Certain types of motor imagery techniques are more effective than others. For instance, focusing on kinesthetic cues and adopting a first-person perspective are more effective than focusing on visual cues and adopting a third-person perspective. As motor imagery becomes more dominant in neurorehabilitation and BCIs, it is important to elucidate what makes these techniques effective. The purpose of this review is to examine the research to date that focuses on both motor imagery and bimanual coordination. An assessment of current research on these two themes may serve as a useful platform for scientists and clinicians seeking to use motor imagery to help improve bimanual coordination, either through augmenting physical therapy or developing more effective BCIs.

Recent Advances and Current Trends in Brain-Computer Interface (BCI) Research and Their Applications

Brain-Computer Interface (BCI) provides direct communication between the brain and an external device. BCI systems have become a trendy field of research in recent years. These systems can be used in a variety of applications to help both disabled and healthy people. Concerning significant BCI progress, we may assume that these systems are not very far from real-world applications. This review has taken into account current trends in BCI research. In this survey, one hundred most cited articles from the WOS database were selected over the last four years. This survey is divided into several sectors. These sectors are Medicine, Communication and Control, Entertainment, and Other BCI applications. The application area, recording method, signal acquisition types, and countries of origin have been identified in each article. This survey provides an overview of the BCI articles published from 2016 to 2020 and their current trends and advances in different application areas.

Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review

Classification of electroencephalogram (EEG) is a key approach to measure the rhythmic oscillations of neural activity, which is one of the core technologies of brain-computer interface systems (BCIs). However, extraction of the features from non-linear and non-stationary EEG signals is still a challenging task in current algorithms. With the development of artificial intelligence, various advanced algorithms have been proposed for signal classification in recent years. Among them, deep neural networks (DNNs) have become the most attractive type of method due to their end-to-end structure and powerful ability of automatic feature extraction. However, it is difficult to collect large-scale datasets in practical applications of BCIs, which may lead to overfitting or weak generalizability of the classifier. To address these issues, a promising technique has been proposed to improve the performance of the decoding model based on data augmentation (DA). In this article, we investigate recent studies and development of various DA strategies for EEG classification based on DNNs. The review consists of three parts: what kind of paradigms of EEG-based on BCIs are used, what types of DA methods are adopted to improve the DNN models, and what kind of accuracy can be obtained. Our survey summarizes the current practices and performance outcomes that aim to promote or guide the deployment of DA to EEG classification in future research and development.

Pocketable Labs for Everyone: Synchronized Multi-Sensor Data Streaming and Recording on Smartphones with the Lab Streaming Layer

The streaming and recording of smartphone sensor signals is desirable for mHealth, telemedicine, environmental monitoring and other applications. Time series data gathered in these fields typically benefit from the time-synchronized integration of different sensor signals. However, solutions required for this synchronization are mostly available for stationary setups. We hope to contribute to the important emerging field of portable data acquisition by presenting open-source Android applications both for the synchronized streaming (Send-a) and recording (Record-a) of multiple sensor data streams. We validate the applications in terms of functionality, flexibility and precision in fully mobile setups and in hybrid setups combining mobile and desktop hardware. Our results show that the fully mobile solution is equivalent to well-established desktop versions. With the streaming application Send-a and the recording application Record-a, purely smartphone-based setups for mobile research and personal health settings can be realized on off-the-shelf Android devices.

Brain-computer interface-based assessment of color vision

Objective.Present methods for assessing color vision require the person's active participation. Here we describe a brain-computer interface-based method for assessing color vision that does not require the person's participation.Approach.This method uses steady-state visual evoked potentials to identify metamers-two light sources that have different spectral distributions but appear to the person to be the same color.Main results.We demonstrate that: minimization of the visual evoked potential elicited by two flickering light sources identifies the metamer; this approach can distinguish people with color-vision deficits from those with normal color vision; and this metamer-identification process can be automated.Significance.This new method has numerous potential clinical, scientific, and industrial applications.

Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review

Objective.Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain-computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines.Approach.The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc.Main results.The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development.Significance.Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

A Study of Visual Search based Calibration Protocol for EEG Attention Detection

Attention, a multi-faceted cognitive process, is essential in our daily lives. We can measure visual attention using an EEG Brain-Computer Interface for detecting different levels of attention in gaming, performance training, and clinical applications. In attention calibration, we use Flanker task to capture EEG data for attentive class. For EEG data belonging to inattentive class calibration, we instruct subject not focusing on a specific position on screen. We then classify attention levels using binary classifier trained with these surrogate ground-truth classes. However, subjects may not be in desirable attention conditions when performing repetitive boring activities over a long experiment duration. We propose attention calibration protocols in this paper that use simultaneous visual search with an audio directional change paradigm and static white noise as 'attentive' and 'inattentive' conditions, respectively. To compare the performance of proposed calibrations against baselines, we collected data from sixteen healthy subjects. For a fair comparison of classification performance; we used six basic EEG band-power features with a standard binary classifier. With the new calibration protocol, we achieved 74.37 ± 6.56% mean subject accuracy, which is about 3.73 ± 2.49% higher than the baseline, but there were no statistically significant differences. According to post-experiment survey results, new calibrations are more effective in inducing desired perceived attention levels. We will improve calibration protocols with reliable attention classifier modeling to enable better attention recognition based on these promising results.

Subject-Specific Cognitive Workload Classification Using EEG-Based Functional Connectivity and Deep Learning

Cognitive workload is a crucial factor in tasks involving dynamic decision-making and other real-time and high-risk situations. Neuroimaging techniques have long been used for estimating cognitive workload. Given the portability, cost-effectiveness and high time-resolution of EEG as compared to fMRI and other neuroimaging modalities, an efficient method of estimating an individual’s workload using EEG is of paramount importance. Multiple cognitive, psychiatric and behavioral phenotypes have already been known to be linked with “functional connectivity”, i.e., correlations between different brain regions. In this work, we explored the possibility of using different model-free functional connectivity metrics along with deep learning in order to efficiently classify the cognitive workload of the participants. To this end, 64-channel EEG data of 19 participants were collected while they were doing the traditional n-back task. These data (after pre-processing) were used to extract the functional connectivity features, namely Phase Transfer Entropy (PTE), Mutual Information (MI) and Phase Locking Value (PLV). These three were chosen to do a comprehensive comparison of directed and non-directed model-free functional connectivity metrics (allows faster computations). Using these features, three deep learning classifiers, namely CNN, LSTM and Conv-LSTM were used for classifying the cognitive workload as low (1-back), medium (2-back) or high (3-back). With the high inter-subject variability in EEG and cognitive workload and recent research highlighting that EEG-based functional connectivity metrics are subject-specific, subject-specific classifiers were used. Results show the state-of-the-art multi-class classification accuracy with the combination of MI with CNN at 80.87%, followed by the combination of PLV with CNN (at 75.88%) and MI with LSTM (at 71.87%). The highest subject specific performance was achieved by the combinations of PLV with Conv-LSTM, and PLV with CNN with an accuracy of 97.92%, followed by the combination of MI with CNN (at 95.83%) and MI with Conv-LSTM (at 93.75%). The results highlight the efficacy of the combination of EEG-based model-free functional connectivity metrics and deep learning in order to classify cognitive workload. The work can further be extended to explore the possibility of classifying cognitive workload in real-time, dynamic and complex real-world scenarios.

Time in Brain: How Biological Rhythms Impact on EEG Signals and on EEG-Derived Brain Networks

Electroencephalography (EEG) is a widely employed tool for exploring brain dynamics and is used extensively in various domains, ranging from clinical diagnosis via neuroscience, cognitive science, cognitive psychology, psychophysiology, neuromarketing, neurolinguistics, and pharmacology to research on brain computer interfaces. EEG is the only technique that enables the continuous recording of brain dynamics over periods of time that range from a few seconds to hours and days and beyond. When taking long-term recordings, various endogenous and exogenous biological rhythms may impinge on characteristics of EEG signals. While the impact of the circadian rhythm and of ultradian rhythms on spectral characteristics of EEG signals has been investigated for more than half a century, only little is known on how biological rhythms influence characteristics of brain dynamics assessed with modern EEG analysis techniques. At the example of multiday, multichannel non-invasive and invasive EEG recordings, we here discuss the impact of biological rhythms on temporal changes of various characteristics of human brain dynamics: higher-order statistical moments and interaction properties of multichannel EEG signals as well as local and global characteristics of EEG-derived evolving functional brain networks. Our findings emphasize the need to take into account the impact of biological rhythms in order to avoid erroneous statements about brain dynamics and about evolving functional brain networks.

The Sample Size Matters: To What Extent the Participant Reduction Affects the Outcomes of a Neuroscientific Research. A Case-Study in Neuromarketing Field

The sample size is a crucial concern in scientific research and even more in behavioural neurosciences, where besides the best practice it is not always possible to reach large experimental samples. In this study we investigated how the outcomes of research change in response to sample size reduction. Three indices computed during a task involving the observations of four videos were considered in the analysis, two related to the brain electroencephalographic (EEG) activity and one to autonomic physiological measures, i.e., heart rate and skin conductance. The modifications of these indices were investigated considering five subgroups of sample size (32, 28, 24, 20, 16), each subgroup consisting of 630 different combinations made by bootstrapping n (n = sample size) out of 36 subjects, with respect to the total population (i.e., 36 subjects). The correlation analysis, the mean squared error (MSE), and the standard deviation (STD) of the indexes were studied at the participant reduction and three factors of influence were considered in the analysis: the type of index, the task, and its duration (time length). The findings showed a significant decrease of the correlation associated to the participant reduction as well as a significant increase of MSE and STD (p < 0.05). A threshold of subjects for which the outcomes remained significant and comparable was pointed out. The effects were to some extents sensitive to all the investigated variables, but the main effect was due to the task length. Therefore, the minimum threshold of subjects for which the outcomes were comparable increased at the reduction of the spot duration.

Toward EEG-Based BCI Applications for Industry 4.0: Challenges and Possible Applications

In the last few decades, Brain-Computer Interface (BCI) research has focused predominantly on clinical applications, notably to enable severely disabled people to interact with the environment. However, recent studies rely mostly on the use of non-invasive electroencephalographic (EEG) devices, suggesting that BCI might be ready to be used outside laboratories. In particular, Industry 4.0 is a rapidly evolving sector that aims to restructure traditional methods by deploying digital tools and cyber-physical systems. BCI-based solutions are attracting increasing attention in this field to support industrial performance by optimizing the cognitive load of industrial operators, facilitating human-robot interactions, and make operations in critical conditions more secure. Although these advancements seem promising, numerous aspects must be considered before developing any operational solutions. Indeed, the development of novel applications outside optimal laboratory conditions raises many challenges. In the current study, we carried out a detailed literature review to investigate the main challenges and present criteria relevant to the future deployment of BCI applications for Industry 4.0.

A Multi-Scale Activity Transition Network for Data Translation in EEG Signals Decoding

Electroencephalogram (EEG) is a non-invasive collection method for brain signals. It has broad prospects in brain-computer interface (BCI) applications. Recent advances have shown the effectiveness of the widely used convolutional neural network (CNN) in EEG decoding. However, some studies reveal that a slight disturbance to the inputs, e.g., data translation, can change CNN's outputs. Such instability is dangerous for EEG-based BCI applications because signals in practice are different from training data. In this study, we propose a multi-scale activity transition network (MSATNet) to alleviate the influence of the translation problem in convolution-based models. MSATNet provides an activity state pyramid consisting of multi-scale recurrent neural networks to capture the relationship between brain activities, which is a translation-invariant feature. In the experiment, Kullback-Leibler divergence is applied to measure the degree of translation. The comprehensive results demonstrate that our method surpasses the AUC of 0.0080, 0.0254, 0.0393 in 1, 5, and 10 KL divergence compared to competitors with various convolution structures.

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making

In this paper we present, and test in two realistic environments, collaborative Brain-Computer Interfaces (cBCIs) that can significantly increase both the speed and the accuracy of perceptual group decision-making. The key distinguishing features of this work are: (1) our cBCIs combine behavioural, physiological and neural data in such a way as to be able to provide a group decision at any time after the quickest team member casts their vote, but the quality of a cBCI-assisted decision improves monotonically the longer the group decision can wait; (2) we apply our cBCIs to two realistic scenarios of military relevance (patrolling a dark corridor and manning an outpost at night where users need to identify any unidentified characters that appear) in which decisions are based on information conveyed through video feeds; and (3) our cBCIs exploit Event-Related Potentials (ERPs) elicited in brain activity by the appearance of potential threats but, uniquely, the appearance time is estimated automatically by the system (rather than being unrealistically provided to it). As a result of these elements, in the two test environments, groups assisted by our cBCIs make both more accurate and faster decisions than when individual decisions are integrated in more traditional manners.

Mental workload classification based on ignored auditory probes and spatial covariance

Objective.Estimation of mental workload (MWL) levels by electroencephalography (EEG)-based mental state monitoring systems has been widely explored. Using event-related potentials (ERPs), elicited by ignored auditory probes, minimizes intrusiveness and has shown high performance for estimating MWL level when tested in laboratory situations. However, when facing real-world applications, the characteristics of ERP waveforms, like latency and amplitude, are often affected by noise, which leads to a decrease in classification performance. One approach to mitigating this is using spatial covariance, which is less sensitive to latency and amplitude distortion. In this study, we used ignored auditory probes in a single-stimulus paradigm and tested Riemannian processed covariance-based features for MWL level estimation in a realistic flight-control task.Approach.We recorded EEG data with an eight-channel system from participants while they performed a simulated drone-control task and manipulated MWL levels (high and low) by task difficulty. We compared support vector machine classification performance based on frequency band power features versus features generated via the Riemannian log map operator from spatial covariance matrices. We also compared accuracy of using data segmented as auditory ERPs versus non-ERPs, for which data windows did not overlap with the ERPs.Main results.Classification accuracy of both types of features showed no significant difference between ERPs and non-ERPs. When we ignore auditory stimuli to perform continuous decoding, covariance-based features in the gamma band had area under the receiver-operating-characteristic curve (AUC) of 0.883, which was significantly higher than band power features (AUC = 0.749).Significance.This study demonstrates that Riemannian-processed covariance features are viable for MWL classification under a realistic experimental scenario.

Business Simulation Games Analysis Supported by Human-Computer Interfaces: A Systematic Review

This article performs a Systematic Review of studies to answer the question: What are the researches related to the learning process with (Serious) Business Games using data collection techniques with Electroencephalogram or Eye tracking signals? The PRISMA declaration method was used to guide the search and inclusion of works related to the elaboration of this study. The 19 references resulting from the critical evaluation initially point to a gap in investigations into using these devices to monitor serious games for learning in organizational environments. An approximation with equivalent sensing studies in serious games for the contribution of skills and competencies indicates that continuous monitoring measures, such as mental state and eye fixation, proved to identify the players' attention levels effectively. Also, these studies showed effectiveness in the flow at different moments of the task, motivating and justifying the replication of these studies as a source of insights for the optimized design of business learning tools. This study is the first systematic review and consolidates the existing literature on user experience analysis of business simulation games supported by human-computer interfaces.

Semantic Data Mining in Ubiquitous Sensing: A Survey

Mining ubiquitous sensing data is important but also challenging, due to many factors, such as heterogeneous large-scale data that is often at various levels of abstraction. This also relates particularly to the important aspects of the explainability and interpretability of the applied models and their results, and thus ultimately to the outcome of the data mining process. With this, in general, the inclusion of domain knowledge leading towards semantic data mining approaches is an emerging and important research direction. This article aims to survey relevant works in these areas, focusing on semantic data mining approaches and methods, but also on selected applications of ubiquitous sensing in some of the most prominent current application areas. Here, we consider in particular: (1) environmental sensing; (2) ubiquitous sensing in industrial applications of artificial intelligence; and (3) social sensing relating to human interactions and the respective individual and collective behaviors. We discuss these in detail and conclude with a summary of this emerging field of research. In addition, we provide an outlook on future directions for semantic data mining in ubiquitous sensing contexts.

No Difference in Arousal or Cognitive Demands Between Manual and Partially Automated Driving: A Multi-Method On-Road Study

INTRODUCTION

Partial driving automation is not always reliable and requires that drivers maintain readiness to take over control and manually operate the vehicle. Little is known about differences in drivers' arousal and cognitive demands under partial automation and how it may make it difficult for drivers to transition from automated to manual modes. This research examined whether there are differences in drivers' arousal and cognitive demands during manual versus partial automation driving.

METHOD

We compared arousal (using heart rate) and cognitive demands (using the root mean square of successive differences in normal heartbeats; RMSSD, and Detection Response Task; DRT) while 39 younger (M = 28.82 years) and 32 late-middle-aged (M = 52.72 years) participants drove four partially automated vehicles (Cadillac, Nissan Rogue, Tesla, and Volvo) on interstate highways. If compared to manual driving, drivers' arousal and cognitive demands were different under partial automation, then corresponding differences in heart rate, RMSSD, and DRT would be expected. Alternatively, if drivers' arousal and cognitive demands were similar in manual and partially automated driving, no difference in the two driving modes would be expected.

RESULTS

Results suggest no significant differences in heart rate, RMSSD, or DRT reaction time performance between manual and partially automated modes of driving for either younger or late-middle-aged adults across the four test vehicles. A Bayes Factor analysis suggested that heart rate, RMSSD, and DRT data showed extreme evidence in favor of the null hypothesis.

CONCLUSION

This novel study conducted on real roads with a representative sample provides important evidence of no difference in arousal and cognitive demands. Younger and late-middle-aged motorists who are new to partial automation are able to maintain arousal and cognitive demands comparable to manual driving while using the partially automated technology. Drivers who are more experienced with partially automated technology may respond differently than those with limited prior experience.