The potential of photoplethysmogram and galvanic skin response in emotion recognition using nonlinear features

Lemniscate of Bernoulli's map quantifiers: innovative measures for EEG emotion recognition

Thanks to the advent of affective computing, designing an automatic human emotion recognition system for clinical and non-clinical applications has attracted the attention of many researchers. Currently, multi-channel electroencephalogram (EEG)-based emotion recognition is a fundamental but challenging issue. This experiment envisioned developing a new scheme for automated EEG affect recognition. An innovative nonlinear feature engineering approach was presented based on Lemniscate of Bernoulli's Map (LBM), which belongs to the family of chaotic maps, in line with the EEG's nonlinear nature. As far as the authors know, LBM has not been utilized for biological signal analysis. Next, the map was characterized using several graphical indices. The feature vector was imposed on the feature selection algorithm while evaluating the role of the feature vector dimension on emotion recognition rates. Finally, the efficiency of the features on emotion recognition was appraised using two conventional classifiers and validated using the Database for Emotion Analysis using Physiological signals (DEAP) and SJTU Emotion EEG Dataset-IV (SEED-IV) benchmark databases. The experimental results showed a maximum accuracy of 92.16% for DEAP and 90.7% for SEED-IV. Achieving higher recognition rates compared to the state-of-art EEG emotion recognition systems suggest the proposed method based on LBM could have potential both in characterizing bio-signal dynamics and detecting affect-deficit disorders.

Emotion Recognition Using a Novel Granger Causality Quantifier and Combined Electrodes of EEG

Electroencephalogram (EEG) connectivity patterns can reflect neural correlates of emotion. However, the necessity of evaluating bulky data for multi-channel measurements increases the computational cost of the EEG network. To date, several approaches have been presented to pick the optimal cerebral channels, mainly depending on available data. Consequently, the risk of low data stability and reliability has increased by reducing the number of channels. Alternatively, this study suggests an electrode combination approach in which the brain is divided into six areas. After extracting EEG frequency bands, an innovative Granger causality-based measure was introduced to quantify brain connectivity patterns. The feature was subsequently subjected to a classification module to recognize valence-arousal dimensional emotions. A Database for Emotion Analysis Using Physiological Signals (DEAP) was used as a benchmark database to evaluate the scheme. The experimental results revealed a maximum accuracy of 89.55%. Additionally, EEG-based connectivity in the beta-frequency band was able to effectively classify dimensional emotions. In sum, combined EEG electrodes can efficiently replicate 32-channel EEG information.

A biosensing system employing nonlinear dynamic analysis-assisted neural network for drug-induced cardiotoxicity assessment

Preclinical investigation of drug-induced cardiotoxicity is of importance for drug development. To evaluate such cardiotoxicity, in vitro high-throughput interdigitated electrode-based recording of cardiomyocytes mechanical beating is widely used. To automatically analyze the features from the beating signals for drug-induced cardiotoxicity assessment, artificial neural network analysis is conventionally employed and signals are segmented into cycles and feature points are located in the cycles. However, signal segmentation and location of feature points for different signal shapes require design of specific algorithms. Consequently, this may lower the efficiency of research and the applications of such algorithms in signals with different morphologies are limited. Here, we present a biosensing system that employs nonlinear dynamic analysis-assisted neural network (NDANN) to avoid the signal segmentation process and directly extract features from beating signal time series. By processing beating time series with fixed time duration to avoid the signal segmentation process, this NDANN-based biosensing system can identify drug-induced cardiotoxicity with accuracy over 0.99. The individual drugs were classified with high accuracies over 0.94 and drug-induced cardiotoxicity levels were accurately predicted. We also evaluated the generalization performance of the NDANN-based biosensing system in assessing drug-induced cardiotoxicity through an independent dataset. This system achieved accuracy of 0.85-0.95 for different drug concentrations in identification of drug-induced cardiotoxicity. This result demonstrates that our NDANN-based biosensing system has the capacity of screening newly developed drugs, which is crucial in practical applications. This NDANN-based biosensing system can work as a new screening platform for drug-induced cardiotoxicity and improve the efficiency of bio-signal processing.

Innovative Poincare's plot asymmetry descriptors for EEG emotion recognition

Given the importance of emotion recognition in both medical and non-medical applications, designing an automatic system has captured the attention of several scholars. Currently, EEG-based emotion recognition has a special position, which has not fulfilled the desired accuracy rates yet. This experiment intended to provide novel EEG asymmetry measures to improve emotion recognition rates. Four emotional states have been classified using the k-nearest neighbor (kNN), support vector machine, and Naïve Bayes. Feature selection has been performed, and the role of employing a different number of top-ranked features on emotion recognition rates has been assessed. To validate the efficiency of the proposed scheme, two public databases, including the SJTU Emotion EEG Dataset-IV (SEED-IV) and a Database for Emotion Analysis using Physiological signals (DEAP) were evaluated. The experimental results indicated that kNN outperformed the other classifiers with a maximum accuracy of 95.49 and 98.63% using SEED-IV and DEAP datasets, respectively. In conclusion, the results of the proposed novel EEG-asymmetry measures make the framework a superior one compared to the state-of-art EEG emotion recognition approaches.

Verhulst map measures: new biomarkers for heart rate classification

Recording, monitoring, and analyzing biological signals has received significant attention in medicine. A fundamental phase for understanding a bio-system under various conditions is to process the corresponding bio-signal appropriately. To this effect, different conventional and nonlinear approaches have been proposed. However, since the non-stationary properties of the bio-signals are not revealed by traditional linear methods, nonlinear dynamical techniques play a crucial role in examining the behavior of a bio-system. This work proposes new bio-markers based on the chaotic nature of the biomedical signals. These measures were introduced using the Verhulst map, a simple tool for characterizing the morphology of the reconstructed phase space. For this purpose, we extracted the features from the heart rate (HR) signals of six groups of meditators and non-meditators. For a typical classification problem, the performance of some conventional classifiers, including the k-nearest neighbor, support vector machine, and Naïve Bayes, was appraised separately. In addition, the competence of a hybrid classification strategy was inspected using majority voting. The results indicated a maximum accuracy, F1-score, and sensitivity of 100%. These findings reveal that the proposed framework is eminently capable of analyzing and classifying the HR signals of the groups. In conclusion, the Verhulst diagram-based measures are simple and based on the dynamics of the bio-signals, which can be served for quantifying different signals in medical systems.

Linking Multi-Layer Dynamical GCN With Style-Based Recalibration CNN for EEG-Based Emotion Recognition

Electroencephalography (EEG)-based emotion computing has become one of the research hotspots of human-computer interaction (HCI). However, it is difficult to effectively learn the interactions between brain regions in emotional states by using traditional convolutional neural networks because there is information transmission between neurons, which constitutes the brain network structure. In this paper, we proposed a novel model combining graph convolutional network and convolutional neural network, namely MDGCN-SRCNN, aiming to fully extract features of channel connectivity in different receptive fields and deep layer abstract features to distinguish different emotions. Particularly, we add style-based recalibration module to CNN to extract deep layer features, which can better select features that are highly related to emotion. We conducted two individual experiments on SEED data set and SEED-IV data set, respectively, and the experiments proved the effectiveness of MDGCN-SRCNN model. The recognition accuracy on SEED and SEED-IV is 95.08 and 85.52%, respectively. Our model has better performance than other state-of-art methods. In addition, by visualizing the distribution of different layers features, we prove that the combination of shallow layer and deep layer features can effectively improve the recognition performance. Finally, we verified the important brain regions and the connection relationships between channels for emotion generation by analyzing the connection weights between channels after model learning.

Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning

Emotion recognition gained increasingly prominent attraction from a multitude of fields recently due to their wide use in human-computer interaction interface, therapy, and advanced robotics, etc. Human speech, gestures, facial expressions, and physiological signals can be used to recognize different emotions. Despite the discriminating properties to recognize emotions, the first three methods have been regarded as ineffective as the probability of human's voluntary and involuntary concealing the real emotions can not be ignored. Physiological signals, on the other hand, are capable of providing more objective, and reliable emotion recognition. Based on physiological signals, several methods have been introduced for emotion recognition, yet, predominantly such approaches are invasive involving the placement of on-body sensors. The efficacy and accuracy of these approaches are hindered by the sensor malfunctioning and erroneous data due to human limbs movement. This study presents a non-invasive approach where machine learning complements the impulse radio ultra-wideband (IR-UWB) signals for emotion recognition. First, the feasibility of using IR-UWB for emotion recognition is analyzed followed by determining the state of emotions into happiness, disgust, and fear. These emotions are triggered using carefully selected video clips to human subjects involving both males and females. The convincing evidence that different breathing patterns are linked with different emotions has been leveraged to discriminate between different emotions. Chest movement of thirty-five subjects is obtained using IR-UWB radar while watching the video clips in solitude. Extensive signal processing is applied to the obtained chest movement signals to estimate respiration rate per minute (RPM). The RPM estimated by the algorithm is validated by repeated measurements by a commercially available Pulse Oximeter. A dataset is maintained comprising gender, RPM, age, and associated emotions which are further used with several machine learning algorithms for automatic recognition of human emotions. Experiments reveal that IR-UWB possesses the potential to differentiate between different human emotions with a decent accuracy of 76% without placing any on-body sensors. Separate analysis for male and female participants reveals that males experience high arousal for happiness while females experience intense fear emotions. For disgust emotion, no large difference is found for male and female participants. To the best of the authors' knowledge, this study presents the first non-invasive approach using the IR-UWB radar for emotion recognition.

Human Emotion Recognition using Polar-Based Lagged Poincare Plot Indices of Eye-Blinking Data

Emotion recognition using bio-signals is currently a hot and challenging topic in human–computer interferences, robotics, and affective computing. A broad range of literature has been published by analyzing the internal/external behaviors of the subjects in confronting emotional events/stimuli. Eye movements, as an external behavior, are frequently used in the multi-modal emotion recognition system. On the other hand, classic statistical features of the signal have generally been assessed, and the evaluation of its dynamics has been neglected so far. For the first time, the dynamics of single-modal eye-blinking data are characterized. Novel polar-based indices of the lagged Poincare plots were introduced. The optimum lag was estimated using mutual information. After reconstruction of the plot, the polar measures of all points were characterized using statistical measures. The support vector machine (SVM), decision tree, and Naïve Bayes were implemented to complete the process of classification. The highest accuracy of 100% with an average accuracy of 84.17% was achieved for fear/sad discrimination using SVM. The suggested framework provided outstanding performances in terms of recognition rates, simplicity of the methodology, and less computational cost. Our results also showed that eye-blinking data possesses the potential for emotion recognition, especially in classifying fear emotion.

Positive and Negative Emotion Classification Based on Multi-channel

The EEG features of different emotions were extracted based on multi-channel and forehead channels in this study. The EEG signals of 26 subjects were collected by the emotional video evoked method. The results show that the energy ratio and differential entropy of the frequency band can be used to classify positive and negative emotions effectively, and the best effect can be achieved by using an SVM classifier. When only the forehead and forehead signals are used, the highest classification accuracy can reach 66%. When the data of all channels are used, the highest accuracy of the model can reach 82%. After channel selection, the best model of this study can be obtained. The accuracy is more than 86%.

Review of wearable technologies and machine learning methodologies for systematic detection of mild traumatic brain injuries

Mild traumatic brain injuries (mTBIs) are the most common type of brain injury. Timely diagnosis of mTBI is crucial in making 'go/no-go' decision in order to prevent repeated injury, avoid strenuous activities which may prolong recovery, and assure capabilities of high-level performance of the subject. If undiagnosed, mTBI may lead to various short- and long-term abnormalities, which include, but are not limited to impaired cognitive function, fatigue, depression, irritability, and headaches. Existing screening and diagnostic tools to detect acute andearly-stagemTBIs have insufficient sensitivity and specificity. This results in uncertainty in clinical decision-making regarding diagnosis and returning to activity or requiring further medical treatment. Therefore, it is important to identify relevant physiological biomarkers that can be integrated into a mutually complementary set and provide a combination of data modalities for improved on-site diagnostic sensitivity of mTBI. In recent years, the processing power, signal fidelity, and the number of recording channels and modalities of wearable healthcare devices have improved tremendously and generated an enormous amount of data. During the same period, there have been incredible advances in machine learning tools and data processing methodologies. These achievements are enabling clinicians and engineers to develop and implement multiparametric high-precision diagnostic tools for mTBI. In this review, we first assess clinical challenges in the diagnosis of acute mTBI, and then consider recording modalities and hardware implementation of various sensing technologies used to assess physiological biomarkers that may be related to mTBI. Finally, we discuss the state of the art in machine learning-based detection of mTBI and consider how a more diverse list of quantitative physiological biomarker features may improve current data-driven approaches in providing mTBI patients timely diagnosis and treatment.

Physiological Sensors Based Emotion Recognition While Experiencing Tactile Enhanced Multimedia

Emotion recognition has increased the potential of affective computing by getting an instant feedback from users and thereby, have a better understanding of their behavior. Physiological sensors have been used to recognize human emotions in response to audio and video content that engages single (auditory) and multiple (two: auditory and vision) human senses, respectively. In this study, human emotions were recognized using physiological signals observed in response to tactile enhanced multimedia content that engages three (tactile, vision, and auditory) human senses. The aim was to give users an enhanced real-world sensation while engaging with multimedia content. To this end, four videos were selected and synchronized with an electric fan and a heater, based on timestamps within the scenes, to generate tactile enhanced content with cold and hot air effect respectively. Physiological signals, i.e., electroencephalography (EEG), photoplethysmography (PPG), and galvanic skin response (GSR) were recorded using commercially available sensors, while experiencing these tactile enhanced videos. The precision of the acquired physiological signals (including EEG, PPG, and GSR) is enhanced using pre-processing with a Savitzky-Golay smoothing filter. Frequency domain features (rational asymmetry, differential asymmetry, and correlation) from EEG, time domain features (variance, entropy, kurtosis, and skewness) from GSR, heart rate and heart rate variability from PPG data are extracted. The K nearest neighbor classifier is applied to the extracted features to classify four (happy, relaxed, angry, and sad) emotions. Our experimental results show that among individual modalities, PPG-based features gives the highest accuracy of 78.57% as compared to EEG- and GSR-based features. The fusion of EEG, GSR, and PPG features further improved the classification accuracy to 79.76% (for four emotions) when interacting with tactile enhanced multimedia.

Noninvasive Blood Pressure Classification Based on Photoplethysmography Using K-Nearest Neighbors Algorithm: A Feasibility Study

Blood pressure (BP) is an important parameter for the early detection of heart disease because it is associated with symptoms of hypertension or hypotension. A single photoplethysmography (PPG) method for the classification of BP can automatically analyze BP symptoms. Users can immediately know the condition of their BP to ensure early detection. In recent years, deep learning methods have presented outstanding performance in classification applications. However, there are two main problems in deep learning classification methods: classification accuracy and time consumption during training. We attempt to address these limitations and propose a method for the classification of BP using the K-nearest neighbors (KNN) algorithm based on PPG. We collected data for 121 subjects from the PPG–BP figshare database. We divided the subjects into three classification levels, namely normotension, prehypertension, and hypertension, according to the BP levels of the Joint National Committee report. The F1 scores of these three classification trials were 100%, 100%, and 90.80%, respectively. Hence, it is validated that the proposed method can achieve improved classification accuracy without additional manual pre-processing of PPG. Our proposed method achieves higher accuracy than convolutional neural networks (deep learning), bagged tree, logistic regression, and AdaBoost tree.