Software Usability Testing Using EEG-Based Emotion Detection and Deep Learning

It is becoming increasingly attractive to detect human emotions using electroencephalography (EEG) brain signals. EEG is a reliable and cost-effective technology used to measure brain activities. This paper proposes an original framework for usability testing based on emotion detection using EEG signals, which can significantly affect software production and user satisfaction. This approach can provide an in-depth understanding of user satisfaction accurately and precisely, making it a valuable tool in software development. The proposed framework includes a recurrent neural network algorithm as a classifier, a feature extraction algorithm based on event-related desynchronization and event-related synchronization analysis, and a new method for selecting EEG sources adaptively for emotion recognition. The framework results are promising, achieving 92.13%, 92.67%, and 92.24% for the valence-arousal-dominance dimensions, respectively.

A Bimodal Emotion Recognition Approach through the Fusion of Electroencephalography and Facial Sequences

In recent years, human-computer interaction (HCI) systems have become increasingly popular. Some of these systems demand particular approaches for discriminating actual emotions through the use of better multimodal methods. In this work, a deep canonical correlation analysis (DCCA) based multimodal emotion recognition method is presented through the fusion of electroencephalography (EEG) and facial video clips. A two-stage framework is implemented, where the first stage extracts relevant features for emotion recognition using a single modality, while the second stage merges the highly correlated features from the two modalities and performs classification. Convolutional neural network (CNN) based Resnet50 and 1D-CNN (1-Dimensional CNN) have been utilized to extract features from facial video clips and EEG modalities, respectively. A DCCA-based approach was used to fuse highly correlated features, and three basic human emotion categories (happy, neutral, and sad) were classified using the SoftMax classifier. The proposed approach was investigated based on the publicly available datasets called MAHNOB-HCI and DEAP. Experimental results revealed an average accuracy of 93.86% and 91.54% on the MAHNOB-HCI and DEAP datasets, respectively. The competitiveness of the proposed framework and the justification for exclusivity in achieving this accuracy were evaluated by comparison with existing work.

A Brain Controlled Command-Line Interface to Enhance the Accessibility of Severe Motor Disabled People to Personnel Computer

There are many applications controlled by the brain signals to bridge the gap in the digital divide between the disabled and the non-disabled people. The deployment of novel assistive technologies using brain-computer interface (BCI) will go a long way toward achieving this lofty goal, especially after the successes demonstrated by these technologies in the daily life of people with severe disabilities. This paper contributes in this direction by proposing an integrated framework to control the operating system functionalities using Electroencephalography signals. Different signal processing algorithms were applied to remove artifacts, extract features, and classify trials. The proposed approach includes different classification algorithms dedicated to detecting the P300 responses efficiently. The predicted commands passed through a socket to the API system, permitting the control of the operating system functionalities. The proposed system outperformed those obtained by the winners of the BCI competition and reached an accuracy average of 94.5% according to the offline approach. The framework was evaluated according to the online process and achieved an excellent accuracy attaining 97% for some users but not less than 90% for others. The suggested framework enhances the information accessibility for people with severe disabilities and helps them perform their daily tasks efficiently. It permits the interaction between the user and personal computers through the brain signals without any muscular efforts.

Embedded Brain Computer Interface: State-of-the-Art in Research

There is a wide area of application that uses cerebral activity to restore capabilities for people with severe motor disabilities, and actually the number of such systems keeps growing. Most of the current BCI systems are based on a personal computer. However, there is a tremendous interest in the implementation of BCIs on a portable platform, which has a small size, faster to load, much lower price, lower resources, and lower power consumption than those for full PCs. Depending on the complexity of the signal processing algorithms, it may be more suitable to work with slow processors because there is no need to allow excess capacity of more demanding tasks. So, in this review, we provide an overview of the BCIs development and the current available technology before discussing experimental studies of BCIs.

Emotion detection using electroencephalography signals and a zero-time windowing-based epoch estimation and relevant electrode identification

Recognizing emotions using biological brain signals requires accurate and efficient signal processing and feature extraction methods. Existing methods use several techniques to extract useful features from a fixed number of electroencephalography (EEG) channels. The primary objective of this study was to improve the performance of emotion recognition using brain signals by applying a novel and adaptive channel selection method that acknowledges that brain activity has a unique behavior that differs from one person to another and one emotional state to another. Moreover, we propose identifying epochs, which are the instants at which excitation is maximum, during the emotion to improve the system’s accuracy. We used the zero-time windowing method to extract instantaneous spectral information using the numerator group-delay function to accurately detect the epochs in each emotional state. Different classification scheme were defined using QDC and RNN and evaluated using the DEAP database. The experimental results showed that the proposed method is highly competitive compared with existing studies of multi-class emotion recognition. The average accuracy rate exceeded 89%. Compared with existing algorithms dealing with 9 emotions, the proposed method enhanced the accuracy rate by 8%. Moreover, experiment shows that the proposed system outperforms similar approaches discriminating between 3 and 4 emotions only. We also found that the proposed method works well, even when applying conventional classification algorithms.

EEG-Based BCI System to Detect Fingers Movements

The advancement of assistive technologies toward the restoration of the mobility of paralyzed and/or amputated limbs will go a long way. Herein, we propose a system that adopts the brain-computer interface technology to control prosthetic fingers with the use of brain signals. To predict the movements of each finger, complex electroencephalogram (EEG) signal processing algorithms should be applied to remove the outliers, extract features, and be able to handle separately the five human fingers. The proposed method deals with a multi-class classification problem. Our machine learning strategy to solve this problem is built on an ensemble of one-class classifiers, each of which is dedicated to the prediction of the intention to move a specific finger. Regions of the brain that are sensitive to the movements of the fingers are identified and located. The average accuracy of the proposed EEG signal processing chain reached 81% for five subjects. Unlike the majority of existing prototypes that allow only one single finger to be controlled and only one movement to be performed at a time, the system proposed will enable multiple fingers to perform movements simultaneously. Although the proposed system classifies five tasks, the obtained accuracy is too high compared with a binary classification system. The proposed system contributes to the advancement of a novel prosthetic solution that allows people with severe disabilities to perform daily tasks in an easy manner.

A dynamic and self-adaptive classification algorithm for motor imagery EEG signals

BACKGROUND

Brain-computer interface (BCI) is a communication pathway applied for pathological analysis or functional substitution. BCI based on functional substitution enables the recognition of a subject's intention to control devices such as prosthesis and wheelchairs. Discrimination of electroencephalography (EEG) trials related to left- and right-hand movements requires complex EEG signal processing to achieve good system performance.

NEW METHOD

In this study, a novel dynamic and self-adaptive algorithm (DSAA) based on the least-squares method is proposed to select the most appropriate feature extraction and classification algorithms couple for each subject. Specifically, the best couple identified during the training of the system is updated during online testing in order to check the stability of the selected couple and maintain high system accuracy.

RESULTS

Extensive and systematic experiments were conducted on public datasets of 17 subjects in the BCI-competition and the results show an improved performance for DSAA over other selected state-of-the-art methods.

COMPARISON WITH EXISTING METHODS

The results show that the proposed system enhanced the classification accuracy for the three chosen public datasets by 8% compared to other approaches. Moreover, the proposed system was successful in selecting the best path despite the unavailability of reference labels.

CONCLUSIONS

Performing dynamic and self-adaptive selection for the best feature extraction and classification algorithm couple increases the recognition rate of trials despite the unavailability of reference trial labels. This approach allows the development of a complete BCI system with excellent accuracy.

BCWB

Web access and web resources open many horizons, their usage increases in all life aspects including government, education, commerce and entertainment, where the key to such resources lies in Web browsers. Acknowledging the importance of universal accessibility to web resources, the W3C has developed a series of guidelines into a Web Accessibility Initiative (WAI), with the goal of providing access to web resources for people with disabilities. In order to bridge the gap in the digital divide between the disabled and the non-disabled people, the authors believe that the development of novel assistive technologies using new human-computer interfaces will go a long way towards achieving this lofty goal. In this paper, they present a P300 Electroencephalography Brain-controlled Web browser to enhance the accessibility of people with severe motor disabilities to Web resources. It enhances their interaction with the Web taking their needs into account. The proposed Web browser satisfies the Mankoff's requirements of a system that would “allow true web access.”

Single Trial EEG Patterns for the Prediction of Individual Differences in Fluid Intelligence

Assessing a person's intelligence level is required in many situations, such as career counseling and clinical applications. EEG evoked potentials in oddball task and fluid intelligence score are correlated because both reflect the cognitive processing and attention. A system for prediction of an individual's fluid intelligence level using single trial Electroencephalography (EEG) signals has been proposed. For this purpose, we employed 2D and 3D contents and 34 subjects each for 2D and 3D, which were divided into low-ability (LA) and high-ability (HA) groups using Raven's Advanced Progressive Matrices (RAPM) test. Using visual oddball cognitive task, neural activity of each group was measured and analyzed over three midline electrodes (Fz, Cz, and Pz). To predict whether an individual belongs to LA or HA group, features were extracted using wavelet decomposition of EEG signals recorded in visual oddball task and support vector machine (SVM) was used as a classifier. Two different types of Haar wavelet transform based features have been extracted from the band (0.3 to 30 Hz) of EEG signals. Statistical wavelet features and wavelet coefficient features from the frequency bands 0.0–1.875 Hz (delta low) and 1.875–3.75 Hz (delta high), resulted in the 100 and 98% prediction accuracies, respectively, both for 2D and 3D contents. The analysis of these frequency bands showed clear difference between LA and HA groups. Further, discriminative values of the features have been validated using statistical significance tests and inter-class and intra-class variation analysis. Also, statistical test showed that there was no effect of 2D and 3D content on the assessment of fluid intelligence level. Comparisons with state-of-the-art techniques showed the superiority of the proposed system.

Evaluation of Image Forgery Detection Using Multi-Scale Weber Local Descriptors

Due to the maturing of digital image processing techniques, there are many tools that can forge an image easily without leaving visible traces and lead to the problem of the authentication of digital images. Based on the assumption that forgery alters the texture micro-patterns in a digital image and texture descriptors can be used for modeling this change; we employed two stat-of-the-art local texture descriptors: multi-scale Weber's law descriptor (multi-WLD) and multi-scale local binary pattern (multi-LBP) for splicing and copy-move forgery detection. As the tamper traces are not visible to open eyes, so the chrominance components of an image encode these traces and were used for modeling tamper traces with the texture descriptors. To reduce the dimension of the feature space and get rid of redundant features, we employed locally learning based (LLB) algorithm. For identifying an image as authentic or tampered, Support vector machine (SVM) was used. This paper presents the thorough investigation for the validation of this forgery detection method. The experiments were conducted on three benchmark image data sets, namely, CASIA v1.0, CASIA v2.0, and Columbia color. The experimental results showed that the accuracy rate of multi-WLD based method was 94.19% on CASIA v1.0, 96.52% on CASIA v2.0, and 94.17% on Columbia data set. It is not only significantly better than multi-LBP based method, but also it outperforms other stat-of-the-art similar forgery detection methods.