1
|
Goshvarpour A, Goshvarpour A. Innovative Poincare's plot asymmetry descriptors for EEG emotion recognition. Cogn Neurodyn 2022; 16:545-559. [PMID: 35603058 PMCID: PMC9120274 DOI: 10.1007/s11571-021-09735-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2021] [Revised: 09/18/2021] [Accepted: 10/13/2021] [Indexed: 10/20/2022] Open
Abstract
Given the importance of emotion recognition in both medical and non-medical applications, designing an automatic system has captured the attention of several scholars. Currently, EEG-based emotion recognition has a special position, which has not fulfilled the desired accuracy rates yet. This experiment intended to provide novel EEG asymmetry measures to improve emotion recognition rates. Four emotional states have been classified using the k-nearest neighbor (kNN), support vector machine, and Naïve Bayes. Feature selection has been performed, and the role of employing a different number of top-ranked features on emotion recognition rates has been assessed. To validate the efficiency of the proposed scheme, two public databases, including the SJTU Emotion EEG Dataset-IV (SEED-IV) and a Database for Emotion Analysis using Physiological signals (DEAP) were evaluated. The experimental results indicated that kNN outperformed the other classifiers with a maximum accuracy of 95.49 and 98.63% using SEED-IV and DEAP datasets, respectively. In conclusion, the results of the proposed novel EEG-asymmetry measures make the framework a superior one compared to the state-of-art EEG emotion recognition approaches.
Collapse
Affiliation(s)
- Atefeh Goshvarpour
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Ateke Goshvarpour
- Department of Biomedical Engineering, Imam Reza International University, Rezvan Campus, Phalestine Sq., Mashhad, Razavi Khorasan Iran
| |
Collapse
|
2
|
Goshvarpour A, Goshvarpour A. Human Emotion Recognition using Polar-Based Lagged Poincare Plot Indices of Eye-Blinking Data. INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE AND APPLICATIONS 2021. [DOI: 10.1142/s1469026821500231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Emotion recognition using bio-signals is currently a hot and challenging topic in human–computer interferences, robotics, and affective computing. A broad range of literature has been published by analyzing the internal/external behaviors of the subjects in confronting emotional events/stimuli. Eye movements, as an external behavior, are frequently used in the multi-modal emotion recognition system. On the other hand, classic statistical features of the signal have generally been assessed, and the evaluation of its dynamics has been neglected so far. For the first time, the dynamics of single-modal eye-blinking data are characterized. Novel polar-based indices of the lagged Poincare plots were introduced. The optimum lag was estimated using mutual information. After reconstruction of the plot, the polar measures of all points were characterized using statistical measures. The support vector machine (SVM), decision tree, and Naïve Bayes were implemented to complete the process of classification. The highest accuracy of 100% with an average accuracy of 84.17% was achieved for fear/sad discrimination using SVM. The suggested framework provided outstanding performances in terms of recognition rates, simplicity of the methodology, and less computational cost. Our results also showed that eye-blinking data possesses the potential for emotion recognition, especially in classifying fear emotion.
Collapse
Affiliation(s)
- Atefeh Goshvarpour
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Ateke Goshvarpour
- Department of Biomedical Engineering, Imam Reza International University, Mashhad, Razavi Khorasan, Iran
| |
Collapse
|
3
|
Pham M, Do HM, Su Z, Bishop A, Sheng W. Negative Emotion Management Using a Smart Shirt and a Robot Assistant. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3067867] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
|
4
|
Blanco-Ruiz M, Sainz-de-Baranda C, Gutiérrez-Martín L, Romero-Perales E, López-Ongil C. Emotion Elicitation Under Audiovisual Stimuli Reception: Should Artificial Intelligence Consider the Gender Perspective? INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17228534. [PMID: 33213064 PMCID: PMC7698584 DOI: 10.3390/ijerph17228534] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 11/02/2020] [Accepted: 11/10/2020] [Indexed: 11/16/2022]
Abstract
Identification of emotions triggered by different sourced stimuli can be applied to automatic systems that help, relieve or protect vulnerable groups of population. The selection of the best stimuli allows to train these artificial intelligence-based systems in a more efficient and precise manner in order to discern different risky situations, characterized either by panic or fear emotions, in a clear and accurate way. The presented research study has produced a dataset of audiovisual stimuli (UC3M4Safety database) that triggers a complete range of emotions, with a high level of agreement and with a discrete emotional categorization, as well as quantitative categorization in the Pleasure-Arousal-Dominance Affective space. This database is adequate for the machine learning algorithms contained in these automatic systems. Furthermore, this work analyses the effects of gender in the emotion elicitation under audiovisual stimuli, which can help to better design the final solution. Particularly, the focus is set on emotional responses to audiovisual stimuli reproducing situations experienced by women, such as gender-based violence. A statistical study of gender differences in emotional response was carried out on 1332 participants (811 women and 521 men). The average responses per video is around 84 (SD = 22). Data analysis was carried out with RStudio®.
Collapse
Affiliation(s)
- Marian Blanco-Ruiz
- University Institute on Gender Studies, Universidad Carlos III de Madrid, 28903 Getafe, Spain; (M.B.-R.); (L.G.-M.); (E.R.-P.); (C.L.-O.)
- Department of Communication Sciences and Sociology, Faculty of Communication Sciences, Universidad Rey Juan Carlos, 28943 Fuenlabrada, Spain
| | - Clara Sainz-de-Baranda
- University Institute on Gender Studies, Universidad Carlos III de Madrid, 28903 Getafe, Spain; (M.B.-R.); (L.G.-M.); (E.R.-P.); (C.L.-O.)
- Department of Communication and Media Studies, Faculty of Humanities, Communication and Library and Science, Universidad Carlos III de Madrid, Getafe, 28903 Madrid, Spain
- Correspondence: ; Tel.: +34-916249737
| | - Laura Gutiérrez-Martín
- University Institute on Gender Studies, Universidad Carlos III de Madrid, 28903 Getafe, Spain; (M.B.-R.); (L.G.-M.); (E.R.-P.); (C.L.-O.)
- Electronic Technology Department, School of Engineering. Universidad Carlos III de Madrid, Leganés, 28911 Madrid, Spain
| | - Elena Romero-Perales
- University Institute on Gender Studies, Universidad Carlos III de Madrid, 28903 Getafe, Spain; (M.B.-R.); (L.G.-M.); (E.R.-P.); (C.L.-O.)
- Electronic Technology Department, School of Engineering. Universidad Carlos III de Madrid, Leganés, 28911 Madrid, Spain
| | - Celia López-Ongil
- University Institute on Gender Studies, Universidad Carlos III de Madrid, 28903 Getafe, Spain; (M.B.-R.); (L.G.-M.); (E.R.-P.); (C.L.-O.)
- Electronic Technology Department, School of Engineering. Universidad Carlos III de Madrid, Leganés, 28911 Madrid, Spain
| |
Collapse
|
5
|
Evaluation of Novel Entropy-Based Complex Wavelet Sub-bands Measures of PPG in an Emotion Recognition System. J Med Biol Eng 2020. [DOI: 10.1007/s40846-020-00526-7] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
6
|
Chen Y, Zhang L, Zhang B, Zhan CA. Short-term HRV in young adults for momentary assessment of acute mental stress. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2019.101746] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
7
|
Goshvarpour A, Goshvarpour A. A Novel Approach for EEG Electrode Selection in Automated Emotion Recognition Based on Lagged Poincare’s Indices and sLORETA. Cognit Comput 2019. [DOI: 10.1007/s12559-019-09699-z] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
8
|
Goshvarpour A, Goshvarpour A. The potential of photoplethysmogram and galvanic skin response in emotion recognition using nonlinear features. AUSTRALASIAN PHYSICAL & ENGINEERING SCIENCES IN MEDICINE 2019; 43:10.1007/s13246-019-00825-7. [PMID: 31776972 DOI: 10.1007/s13246-019-00825-7] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2019] [Accepted: 11/20/2019] [Indexed: 12/15/2022]
Abstract
Recently, developing an accurate automatic emotion recognition system using a minimum number of bio-signals has become a challenging issue in "affective computing." This study aimed to propose a reliable system by examining nonlinear dynamics of photoplethysmogram (PPG) and galvanic skin response (GSR). To address this goal, two strategies were adopted. First, the efficiency of each signal in valence/arousal based emotion categorization was examined. Then, the proficiency of a hybrid feature, by combining both GSR and PPG features was studied. Lyapunov exponents, lagged Poincare's measures, and approximate entropy were extracted to characterize the irregularity and chaotic behavior of the phase space. To discriminate two levels of arousal and two levels of the valence, a probabilistic neural network (PNN) with different sigma adjustment parameter was examined. The results showed that the phase space geometry and consequently, the signal dynamics are influenced by the emotional music video. Additionally, distinctive patterns of the phase space behavior were observed under the influence of different lags. For both signals, the most irregularity was observed during the high valence, and the least irregularity was seen during the low valence. Consequently, signals' irregularity is affected by the valence dimension. The results showed that the fusion has more potential for emotion recognition than that of using each signal separately. For sigma = 0.1, the highest recognition rate was 100% in a subject-dependent mode. In a subject-independent mode, the maximum accuracies of 88.57 and 86.8% were obtained for arousal and valence dimensions, respectively.
Collapse
Affiliation(s)
- Atefeh Goshvarpour
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Ateke Goshvarpour
- Department of Biomedical Engineering, Imam Reza International University, Mashhad, Razavi Khorasan, Iran.
- Imam Reza International University, Rezvan Campus (Female Students), Phalestine Sq., PO. BOX 91735-553, Mashhad, Razavi Khorasan, Iran.
| |
Collapse
|
9
|
|
10
|
Goshvarpour A, Goshvarpour A. Human identification using a new matching Pursuit-based feature set of ECG. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 172:87-94. [PMID: 30902130 DOI: 10.1016/j.cmpb.2019.02.009] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2018] [Revised: 01/25/2019] [Accepted: 02/12/2019] [Indexed: 06/09/2023]
Abstract
BACKGROUND AND OBJECTIVE In recent years, many attempts have been made to design reliable systems for identifying individuals using biometrics. Electrocardiogram (ECG) biometric is one of the newest methods that not only offers unique characteristics of individuals for human identification, but also the possibility of counterfeiting it is negligible. In this paper, our objective was to develop an identification system using a non-fiducial one-lead ECG feature set based on a sparse algorithm. METHODS The ECG signals of 90 participants were decomposed using a matching pursuit (MP) and several statistical and nonlinear measures were extracted from the MP coefficients. Then, the performance of ECG characteristics delivered by MP analysis in human identification was evaluated by the probabilistic neural network (PNN) and k-nearest neighbor (kNN) with one vs. all strategy. The role of the feature set in classification rates was also tested in different modes, including linear attributes, nonlinear indices, all features, features selected by principal component analysis (PCA), and features selected by linear discriminant analysis (LDA). RESULTS Experimental results showed that (1) the highest recognition rate was 99.68%; (2) the performance of the PNN was superior to the kNN; and (3) selecting features with LDA resulted in higher identification rates. CONCLUSIONS The results are prominent from the performance perspective because it gives higher recognition rates over the group of 90 participants. The great performance of the proposed identification system advocates that it can be employed confidently in different smart systems.
Collapse
Affiliation(s)
- Ateke Goshvarpour
- Department of Biomedical Engineering, Imam Reza International University, Mashhad, Razavi Khorasan, Iran.
| | - Atefeh Goshvarpour
- Graduated from Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran.
| |
Collapse
|
11
|
Goshvarpour A, Goshvarpour A. EEG spectral powers and source localization in depressing, sad, and fun music videos focusing on gender differences. Cogn Neurodyn 2018; 13:161-173. [PMID: 30956720 DOI: 10.1007/s11571-018-9516-y] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2018] [Revised: 11/07/2018] [Accepted: 12/11/2018] [Indexed: 12/14/2022] Open
Abstract
Previously, gender-specific affective responses have been shown using neurophysiological signals. The present study intended to compare the differences in electroencephalographic (EEG) power spectra and EEG brain sources between men and women during the exposure of affective music video stimuli. The multi-channel EEG signals of 15 males and 15 females available in the database for emotion analysis using physiological signals were studied, while subjects were watching sad, depressing, and fun music videos. Seven EEG frequency bands were computed using average Fourier cross-spectral matrices. Then, standardized low-resolution electromagnetic tomography (sLORETA) was used to localize regions involved specifically in these emotional responses. To evaluate gender differences, independent sample t test was calculated for the sLORETA source powers. Our results showed that (1) the mean EEG power for all frequency bands in the women's group was significantly higher than that of the men's group; (2) spatial distribution differentiation between men and women was detected in all EEG frequency bands. (3) This difference has been related to the emotional stimuli, which was more evident for negative emotions. Taken together, our results showed that men and women recruited dissimilar brain networks for processing sad, depressing, and fun audio-visual stimuli.
Collapse
Affiliation(s)
- Atefeh Goshvarpour
- 1Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Ateke Goshvarpour
- 2Department of Biomedical Engineering, Imam Reza International University, Mashhad, Razavi Khorasan Iran
| |
Collapse
|
12
|
Goshvarpour A, Goshvarpour A, Abbasi A. EVALUATION OF SIGNAL PROCESSING TECHNIQUES IN DISCRIMINATING ECG SIGNALS OF MEN AND WOMEN DURING REST CONDITION AND EMOTIONAL STATES. BIOMEDICAL ENGINEERING: APPLICATIONS, BASIS AND COMMUNICATIONS 2018. [DOI: 10.4015/s101623721850028x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Great range of electrocardiogram (ECG) signal processing methods can be found in the literature. In addition, the importance of gender differences in physiological activities was also identified in various conditions. This article aims to provide a comprehensive evaluation of linear and nonlinear ECG parameters to indicate suitable signal processing approaches which can show significant differences between men and women. These differences were investigated in two conditions: (i) during rest condition, and (ii) during the affective image inducements. A wide range of parameters from time-, frequency-, wavelet-, and nonlinear-techniques were examined. Applying the Wilcoxon rank sum test, significant differences between two genders were inspected. The analysis was performed on 47 college students at rest condition and while subjects watching four types of affective pictures, including sadness, happiness, fear, and peacefulness. The impact of these emotions on the results was also investigated. The results indicated that 72.95% and 72.61% of all features were significantly different between male and female in rest condition and affective inducements, respectively. In addition, the highest percentage of the significant difference between ECG parameters of men and women was achieved using nonlinear characteristics. Considering all features together, the highest significant difference between two genders was achieved for negative emotions, including sadness and fear. In conclusion, the results of this study emphasized the importance of gender role in cardiac responses during rest condition and different emotional states. Since these gender differences are well manifested by nonlinear signal processing techniques, dynamical gender-specific ECG system may improve the automatic emotion recognition accuracies.
Collapse
Affiliation(s)
- Ateke Goshvarpour
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Atefeh Goshvarpour
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| | - Ataollah Abbasi
- Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran
| |
Collapse
|