Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction

Real-Time Mobile Robot Obstacles Detection and Avoidance Through EEG Signals

BACKGROUND/OBJECTIVES

The study explores the integration of human feedback into the control loop of mobile robots for real-time obstacle detection and avoidance using EEG brain-computer interface (BCI) methods. The goal is to assess the possible paradigms applicable to the most current navigation system to enhance safety and interaction between humans and robots.

METHODS

The research explores passive and active brain-computer interface (BCI) technologies to enhance a wheelchair-mobile robot's navigation. In the passive approach, error-related potentials (ErrPs), neural signals triggered when users comment or perceive errors, enable automatic correction of the robot navigation mistakes without direct input or command from the user. In contrast, the active approach leverages steady-state visually evoked potentials (SSVEPs), where users focus on flickering stimuli to control the robot's movements directly. This study evaluates both paradigms to determine the most effective method for integrating human feedback into assistive robotic navigation. This study involves experimental setups where participants control a robot through a simulated environment, and their brain signals are recorded and analyzed to measure the system's responsiveness and the user's mental workload.

RESULTS

The results show that a passive BCI requires lower mental effort but suffers from lower engagement, with a classification accuracy of 72.9%, whereas an active BCI demands more cognitive effort but achieves 84.9% accuracy. Despite this, task achievement accuracy is higher in the passive method (e.g., 71% vs. 43% for subject S2) as a single correct ErrP classification enables autonomous obstacle avoidance, whereas SSVEP requires multiple accurate commands.

CONCLUSIONS

This research highlights the trade-offs between accuracy, mental load, and engagement in BCI-based robot control. The findings support the development of more intuitive assistive robotics, particularly for disabled and elderly users.

Using passive BCI for personalization of assistive wearable devices: a proof-of-concept study

Assistive wearable devices can significantly enhance the quality of life for individuals with movement impairments, aid the rehabilitation process, and augment movement abilities of healthy users. However, personalizing the assistance to individual preferences and needs remains a challenge. Brain-Computer Interface (BCI) offers a promising solution for this personalization problem. The overarching goal of this study is to investigate the feasibility of utilizing passive BCI technology to personalize the assistance provided by a knee exoskeleton. Participants performed seated knee flexion-extension tasks while wearing a one-degree-of-freedom knee exoskeleton with varying levels of applied force. Their brain activities were recorded throughout the movements using electroencephalography (EEG). EEG spectral bands from several brain regions were compared between the conditions with the lowest and highest exoskeleton forces to identify statistically significant changes. A Naive Bayes classifier was trained on these spectral features to distinguish between the two conditions. Statistical analysis revealed significant increases in δ and θ activity and decreases in α and β activity in the frontal, motor, and occipital cortices. These changes suggest heightened attention, concentration, and motor engagement when the task became more difficult. The trained Naive Bayes classifier achieved an average accuracy of approximately 72% in distinguishing between the two conditions. The outcomes of our study demonstrate the potential of passive BCI in personalizing assistance provided by wearable devices. Future research should further explore integrating passive BCI into assistive wearable devices to enhance user experience.

Robots as Mental Health Coaches: A Study of Emotional Responses to Technology-Assisted Stress Management Tasks Using Physiological Signals

The current study investigated the effectiveness of social robots in facilitating stress management interventions for university students by evaluating their physiological responses. We collected electroencephalogram (EEG) brain activity and Galvanic Skin Responses (GSRs) together with self-reported questionnaires from two groups of students who practiced a deep breathing exercise either with a social robot or a laptop. From GSR signals, we obtained the change in participants' arousal level throughout the intervention, and from the EEG signals, we extracted the change in their emotional valence using the neurometric of Frontal Alpha Asymmetry (FAA). While subjective perceptions of stress and user experience did not differ significantly between the two groups, the physiological signals revealed differences in their emotional responses as evaluated by the arousal-valence model. The Laptop group tended to show a decrease in arousal level which, in some cases, was accompanied by negative valence indicative of boredom or lack of interest. On the other hand, the Robot group displayed two patterns; some demonstrated a decrease in arousal with positive valence indicative of calmness and relaxation, and others showed an increase in arousal together with positive valence interpreted as excitement. These findings provide interesting insights into the impact of social robots as mental well-being coaches on students' emotions particularly in the presence of the novelty effect. Additionally, they provide evidence for the efficacy of physiological signals as an objective and reliable measure of user experience in HRI settings.

Decoding the Debate: A Comparative Study of Brain-Computer Interface and Neurofeedback

Brain-Computer Interface (BCI) and Neurofeedback (NF) both rely on the technology to capture brain activity. However, the literature lacks a clear distinction between the two, with some scholars categorizing NF as a special case of BCI while others view BCI as a natural extension of NF, or classify them as fundamentally different entities. This ambiguity hinders the flow of information and expertise among scholars and can cause confusion. To address this issue, we conducted a study comparing BCI and NF from two perspectives: the background and context within which BCI and NF developed, and their system design. We utilized Functional Flow Block Diagram (FFBD) as a system modelling approach to visualize inputs, functions, and outputs to compare BCI and NF at a conceptual level. Our analysis revealed that while NF is a subset of the biofeedback method that requires data from the brain to be extracted and processed, the device performing these tasks is a BCI system by definition. Therefore, we conclude that NF should be considered a specific application of BCI technology. By clarifying the relationship between BCI and NF, we hope to facilitate better communication and collaboration among scholars in these fields.

Real-time estimation of EEG-based engagement in different tasks

Objective.Recent trends in brain-computer interface (BCI) research concern the passive monitoring of brain activity, which aim to monitor a wide variety of cognitive states. Engagement is such a cognitive state, which is of interest in contexts such as learning, entertainment or rehabilitation. This study proposes a novel approach for real-time estimation of engagement during different tasks using electroencephalography (EEG).Approach.Twenty-three healthy subjects participated in the BCI experiment. A modified version of the d2 test was used to elicit engagement. Within-subject classification models which discriminate between engaging and resting states were trained based on EEG recorded during a d2 test based paradigm. The EEG was recorded using eight electrodes and the classification model was based on filter-bank common spatial patterns and a linear discriminant analysis. The classification models were evaluated in cross-task applications, namely when playing Tetris at different speeds (i.e. slow, medium, fast) and when watching two videos (i.e. advertisement and landscape video). Additionally, subjects' perceived engagement was quantified using a questionnaire.Main results.The models achieved a classification accuracy of 90% on average when tested on an independent d2 test paradigm recording. Subjects' perceived and estimated engagement were found to be greater during the advertisement compared to the landscape video (p= 0.025 andp<0.001, respectively); greater during medium and fast compared to slow Tetris speed (p<0.001, respectively); not different between medium and fast Tetris speeds. Additionally, a common linear relationship was observed for perceived and estimated engagement (rrm= 0.44,p<0.001). Finally, theta and alpha band powers were investigated, which respectively increased and decreased during more engaging states.Significance.This study proposes a task-specific EEG engagement estimation model with cross-task capabilities, offering a framework for real-world applications.

Classifying human emotions in HRI: applying global optimization model to EEG brain signals

Significant efforts have been made in the past decade to humanize both the form and function of social robots to increase their acceptance among humans. To this end, social robots have recently been combined with brain-computer interface (BCI) systems in an attempt to give them an understanding of human mental states, particularly emotions. However, emotion recognition using BCIs poses several challenges, such as subjectivity of emotions, contextual dependency, and a lack of reliable neuro-metrics for real-time processing of emotions. Furthermore, the use of BCI systems introduces its own set of limitations, such as the bias-variance trade-off, dimensionality, and noise in the input data space. In this study, we sought to address some of these challenges by detecting human emotional states from EEG brain activity during human-robot interaction (HRI). EEG signals were collected from 10 participants who interacted with a Pepper robot that demonstrated either a positive or negative personality. Using emotion valence and arousal measures derived from frontal brain asymmetry (FBA), several machine learning models were trained to classify human's mental states in response to the robot personality. To improve classification accuracy, all proposed classifiers were subjected to a Global Optimization Model (GOM) based on feature selection and hyperparameter optimization techniques. The results showed that it is possible to classify a user's emotional responses to the robot's behavior from the EEG signals with an accuracy of up to 92%. The outcome of the current study contributes to the first level of the Theory of Mind (ToM) in Human-Robot Interaction, enabling robots to comprehend users' emotional responses and attribute mental states to them. Our work advances the field of social and assistive robotics by paving the way for the development of more empathetic and responsive HRI in the future.

Human-robot collaborative task planning using anticipatory brain responses

Human-robot interaction (HRI) describes scenarios in which both human and robot work as partners, sharing the same environment or complementing each other on a joint task. HRI is characterized by the need for high adaptability and flexibility of robotic systems toward their human interaction partners. One of the major challenges in HRI is task planning with dynamic subtask assignment, which is particularly challenging when subtask choices of the human are not readily accessible by the robot. In the present work, we explore the feasibility of using electroencephalogram (EEG) based neuro-cognitive measures for online robot learning of dynamic subtask assignment. To this end, we demonstrate in an experimental human subject study, featuring a joint HRI task with a UR10 robotic manipulator, the presence of EEG measures indicative of a human partner anticipating a takeover situation from human to robot or vice-versa. The present work further proposes a reinforcement learning based algorithm employing these measures as a neuronal feedback signal from the human to the robot for dynamic learning of subtask-assignment. The efficacy of this algorithm is validated in a simulation-based study. The simulation results reveal that even with relatively low decoding accuracies, successful robot learning of subtask-assignment is feasible, with around 80% choice accuracy among four subtasks within 17 minutes of collaboration. The simulation results further reveal that scalability to more subtasks is feasible and mainly accompanied with longer robot learning times. These findings demonstrate the usability of EEG-based neuro-cognitive measures to mediate the complex and largely unsolved problem of human-robot collaborative task planning.

Driving drowsiness detection using spectral signatures of EEG-based neurophysiology

Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks.

Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics.

Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen’s kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods.

Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.

A Sparse Representation Classification Scheme for the Recognition of Affective and Cognitive Brain Processes in Neuromarketing

In this work, we propose a novel framework to recognize the cognitive and affective processes of the brain during neuromarketing-based stimuli using EEG signals. The most crucial component of our approach is the proposed classification algorithm that is based on a sparse representation classification scheme. The basic assumption of our approach is that EEG features from a cognitive or affective process lie on a linear subspace. Hence, a test brain signal can be represented as a linear (or weighted) combination of brain signals from all classes in the training set. The class membership of the brain signals is determined by adopting the Sparse Bayesian Framework with graph-based priors over the weights of linear combination. Furthermore, the classification rule is constructed by using the residuals of linear combination. The experiments on a publicly available neuromarketing EEG dataset demonstrate the usefulness of our approach. For the two classification tasks offered by the employed dataset, namely affective state recognition and cognitive state recognition, the proposed classification scheme manages to achieve a higher classification accuracy compared to the baseline and state-of-the art methods (more than 8% improvement in classification accuracy).

Development of Automatic Controlled Walking Assistive Device Based on Fatigue and Emotion Detection

The world’s aging population is increasing. The number of elderly individuals having walking impairments is also increasing. Adequate exercise is becoming necessary for them. Therefore, several walking assistive devices have been developed or are under development. However, elderly individuals may have low motivation for exercising, or they may experience physical damage by excessive fatigue. This study proposed a method to enable elderly individuals to exercise with a positive emotion and prevent damage such as muscle fatigue. We proposed a 3D human condition model to control the walking assistive device. It includes the arousal, pleasure, and fatigue dimensions. With regard to the arousal and pleasure dimensions, we used heartbeat and electromyography (EEG) signals to train a deep neural network (DNN) model to identify human emotions. For fatigue detection, we proposed a method based on near-infrared spectroscopy (NIRS) to detect muscle fatigue. All the sensors are portable. This implies that it can be used for outdoor activities. Then, we proposed a walking strategy based on a 3D human condition model to control the walking assistive device. Finally, we tested the effectiveness of the automatic control system. The wearing of the walking assistive device and implementation of the walking strategy can delay the fatigue time by approximately 24% and increase the walking distance by approximately 16%. In addition, we succeeded in visualizing the distribution of emotion during each walking method variation. It was verified that the walking strategy can improve the mental condition of a user to a certain extent. These results showed the effectiveness of the proposed system. It could help elderlies maintain higher levels of motivation and prevent muscle damage by walking exercise, using the walking assistive device.

Evaluation of a New Lightweight EEG Technology for Translational Applications of Passive Brain-Computer Interfaces

Technologies like passive brain-computer interfaces (BCI) can enhance human-machine interaction. Anyhow, there are still shortcomings in terms of easiness of use, reliability, and generalizability that prevent passive-BCI from entering real-life situations. The current work aimed to technologically and methodologically design a new gel-free passive-BCI system for out-of-the-lab employment. The choice of the water-based electrodes and the design of a new lightweight headset met the need for easy-to-wear, comfortable, and highly acceptable technology. The proposed system showed high reliability in both laboratory and realistic settings, performing not significantly different from the gold standard based on gel electrodes. In both cases, the proposed system allowed effective discrimination (AUC > 0.9) between low and high levels of workload, vigilance, and stress even for high temporal resolution (<10 s). Finally, the generalizability of the proposed system has been tested through a cross-task calibration. The system calibrated with the data recorded during the laboratory tasks was able to discriminate the targeted human factors during the realistic task reaching AUC values higher than 0.8 at 40 s of temporal resolution in case of vigilance and workload, and 20 s of temporal resolution for the stress monitoring. These results pave the way for ecologic use of the system, where calibration data of the realistic task are difficult to obtain.

A simulated experiment to explore robotic dialogue strategies for people with dementia

Introduction

Persons with dementia (PwDs) often show symptoms of repetitive questioning, which brings great burdens on caregivers. Conversational robots hold promise of helping cope with PwDs’ repetitive behavior. This paper develops an adaptive conversation strategy to answer PwDs’ repetitive questions, follow up with new questions to distract PwDs from repetitive behavior, and stimulate their conversation and cognition.

Methods

We propose a general reinforcement learning model to interact with PwDs with repetitive questioning. Q-learning is exploited to learn adaptive conversation strategy (from the perspectives of rate and difficulty level of follow-up questions) for four simulated PwDs. A demonstration is presented using a humanoid robot.

Results

The designed Q-learning model performs better than random action selection model. The RL-based conversation strategy is adaptive to PwDs with different cognitive capabilities and engagement levels. In the demonstration, the robot can answer a user’s repetitive questions and further come up with a follow-up question to engage the user in continuous conversations.

Conclusions

The designed Q-learning model demonstrates noteworthy effectiveness in adaptive action selection. This may provide some insights towards developing conversational social robots to cope with repetitive questioning by PwDs and increase their quality of life.

Integration of Augmented Reality and Brain-Computer Interface Technologies for Health Care Applications: Exploratory and Prototyping Study

Background

Augmented reality (AR) and brain-computer interface (BCI) are promising technologies that have a tremendous potential to revolutionize health care. While there has been a growing interest in these technologies for medical applications in the recent years, the combined use of AR and BCI remains a fairly unexplored area that offers significant opportunities for improving health care professional education and clinical practice. This paper describes a recent study to explore the integration of AR and BCI technologies for health care applications.

Objective

The described effort aims to advance an understanding of how AR and BCI technologies can effectively work together to transform modern health care practice by providing new mechanisms to improve patient and provider learning, communication, and shared decision-making.

Methods

The study methods included an environmental scan of AR and BCI technologies currently used in health care, a use case analysis for a combined AR-BCI capability, and development of an integrated AR-BCI prototype solution for health care applications.

Results

The study resulted in a novel interface technology solution that enables interoperability between consumer-grade wearable AR and BCI devices and provides the users with an ability to control digital objects in augmented reality using neural commands. The article discusses this novel solution within the context of practical digital health use cases developed during the course of the study where the combined AR and BCI technologies are anticipated to produce the most impact.

Conclusions

As one of the pioneering efforts in the area of AR and BCI integration, the study presents a practical implementation pathway for AR-BCI integration and provides directions for future research and innovation in this area.

Theory of Mind and Preference Learning at the Interface of Cognitive Science, Neuroscience, and AI: A Review

Theory of Mind (ToM)-the ability of the human mind to attribute mental states to others-is a key component of human cognition. In order to understand other people's mental states or viewpoint and to have successful interactions with others within social and occupational environments, this form of social cognition is essential. The same capability of inferring human mental states is a prerequisite for artificial intelligence (AI) to be integrated into society, for example in healthcare and the motoring industry. Autonomous cars will need to be able to infer the mental states of human drivers and pedestrians to predict their behavior. In the literature, there has been an increasing understanding of ToM, specifically with increasing cognitive science studies in children and in individuals with Autism Spectrum Disorder. Similarly, with neuroimaging studies there is now a better understanding of the neural mechanisms that underlie ToM. In addition, new AI algorithms for inferring human mental states have been proposed with more complex applications and better generalisability. In this review, we synthesize the existing understanding of ToM in cognitive and neurosciences and the AI computational models that have been proposed. We focus on preference learning as an area of particular interest and the most recent neurocognitive and computational ToM models. We also discuss the limitations of existing models and hint at potential approaches to allow ToM models to fully express the complexity of the human mind in all its aspects, including values and preferences.

Emotion-Driven Analysis and Control of Human-Robot Interactions in Collaborative Applications

The utilization of robotic systems has been increasing in the last decade. This increase has been derived by the evolvement in the computational capabilities, communication systems, and the information systems of the manufacturing systems which is reflected in the concept of Industry 4.0. Furthermore, the robotics systems are continuously required to address new challenges in the industrial and manufacturing domain, like keeping humans in the loop, among other challenges. Briefly, the keeping humans in the loop concept focuses on closing the gap between humans and machines by introducing a safe and trustworthy environment for the human workers to work side by side with robots and machines. It aims at increasing the engagement of the human as the automation level increases rather than replacing the human, which can be nearly impossible in some applications. Consequently, the collaborative robots (Cobots) have been created to allow physical interaction with the human worker. However, these cobots still lack of recognizing the human emotional state. In this regard, this paper presents an approach for adapting cobot parameters to the emotional state of the human worker. The approach utilizes the Electroencephalography (EEG) technology for digitizing and understanding the human emotional state. Afterwards, the parameters of the cobot are instantly adjusted to keep the human emotional state in a desirable range which increases the confidence and the trust between the human and the cobot. In addition, the paper includes a review on technologies and methods for emotional sensing and recognition. Finally, this approach is tested on an ABB YuMi cobot with commercially available EEG headset.