1
|
Proverbio AM, Santoni S, Adorni R. ERP Markers of Valence Coding in Emotional Speech Processing. iScience 2020; 23:100933. [PMID: 32151976 PMCID: PMC7063241 DOI: 10.1016/j.isci.2020.100933] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Revised: 12/20/2019] [Accepted: 02/19/2020] [Indexed: 11/01/2022] Open
Abstract
How is auditory emotional information processed? The study's aim was to compare cerebral responses to emotionally positive or negative spoken phrases matched for structure and content. Twenty participants listened to 198 vocal stimuli while detecting filler phrases containing first names. EEG was recorded from 128 sites. Three event-related potential (ERP) components were quantified and found to be sensitive to emotional valence since 350 ms of latency. P450 and late positivity were enhanced by positive content, whereas anterior negativity was larger to negative content. A similar set of markers (P300, N400, LP) was found previously for the processing of positive versus negative affective vocalizations, prosody, and music, which suggests a common neural mechanism for extracting the emotional content of auditory information. SwLORETA applied to potentials recorded between 350 and 550 ms showed that negative speech activated the right temporo/parietal areas (BA40, BA20/21), whereas positive speech activated the left homologous and inferior frontal areas.
Collapse
Affiliation(s)
- Alice Mado Proverbio
- Milan Center for Neuroscience, Department of Psychology, University of Milano-Bicocca, Piazza dell'Ateneo Nuovo 1, Milan, Italy.
| | - Sacha Santoni
- Milan Center for Neuroscience, Department of Psychology, University of Milano-Bicocca, Piazza dell'Ateneo Nuovo 1, Milan, Italy
| | - Roberta Adorni
- Milan Center for Neuroscience, Department of Psychology, University of Milano-Bicocca, Piazza dell'Ateneo Nuovo 1, Milan, Italy
| |
Collapse
|
2
|
Proverbio AM, Benedetto F, Guazzone M. Shared neural mechanisms for processing emotions in music and vocalizations. Eur J Neurosci 2019; 51:1987-2007. [DOI: 10.1111/ejn.14650] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 11/21/2019] [Accepted: 12/05/2019] [Indexed: 12/21/2022]
Affiliation(s)
- Alice Mado Proverbio
- Department of Psychology University of Milano‐Bicocca Milan Italy
- Milan Center for Neuroscience Milan Italy
| | - Francesco Benedetto
- Department of Psychology University of Milano‐Bicocca Milan Italy
- Milan Center for Neuroscience Milan Italy
| | - Martina Guazzone
- Department of Psychology University of Milano‐Bicocca Milan Italy
- Milan Center for Neuroscience Milan Italy
| |
Collapse
|
3
|
Dissanayake T, Rajapaksha Y, Ragel R, Nawinne I. An Ensemble Learning Approach for Electrocardiogram Sensor Based Human Emotion Recognition. SENSORS (BASEL, SWITZERLAND) 2019; 19:E4495. [PMID: 31623279 PMCID: PMC6832168 DOI: 10.3390/s19204495] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Revised: 09/27/2019] [Accepted: 10/05/2019] [Indexed: 12/04/2022]
Abstract
Recently, researchers in the area of biosensor based human emotion recognition have used different types of machine learning models for recognizing human emotions. However, most of them still lack the ability to recognize human emotions with higher classification accuracy incorporating a limited number of bio-sensors. In the domain of machine learning, ensemble learning methods have been successfully applied to solve different types of real-world machine learning problems which require improved classification accuracies. Emphasising on that, this research suggests an ensemble learning approach for developing a machine learning model that can recognize four major human emotions namely: anger; sadness; joy; and pleasure incorporating electrocardiogram (ECG) signals. As feature extraction methods, this analysis combines four ECG signal based techniques, namely: heart rate variability; empirical mode decomposition; with-in beat analysis; and frequency spectrum analysis. The first three feature extraction methods are well-known ECG based feature extraction techniques mentioned in the literature, and the fourth technique is a novel method proposed in this study. The machine learning procedure of this investigation evaluates the performance of a set of well-known ensemble learners for emotion classification and further improves the classification results using feature selection as a prior step to ensemble model training. Compared to the best performing single biosensor based model in the literature, the developed ensemble learner has the accuracy gain of 10.77%. Furthermore, the developed model outperforms most of the multiple biosensor based emotion recognition models with a significantly higher classification accuracy gain.
Collapse
Affiliation(s)
- Theekshana Dissanayake
- Department of Computer Engineering, University of Peradeniya, Peradeniya 20400, Sri Lanka.
| | - Yasitha Rajapaksha
- Department of Computer Engineering, University of Peradeniya, Peradeniya 20400, Sri Lanka.
| | - Roshan Ragel
- Department of Computer Engineering, University of Peradeniya, Peradeniya 20400, Sri Lanka.
| | - Isuru Nawinne
- Department of Computer Engineering, University of Peradeniya, Peradeniya 20400, Sri Lanka.
| |
Collapse
|
4
|
Cook ND. The Triadic Roots of Human Cognition: "Mind" Is the Ability to go Beyond Dyadic Associations. Front Psychol 2018; 9:1060. [PMID: 30038590 PMCID: PMC6046464 DOI: 10.3389/fpsyg.2018.01060] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2017] [Accepted: 06/05/2018] [Indexed: 11/13/2022] Open
Abstract
Empirical evidence is reviewed indicating that the extraordinary aspects of the human mind are due to our species' ability to go beyond simple "dyadic associations" and to process the relations among three items of information simultaneously. Classic explanations of the "triadic" nature of human skills have been advocated by various scholars in the context of the evolution of human cognition. Here I summarize the core processes as found in (i) the syntax of language, (ii) tool-usage, and (iii) joint attention. I then review the triadic foundations of two perceptual phenomena of great importance in human aesthetics: (iv) harmony perception and (v) pictorial depth perception. In all five subfields of human psychology, most previous work has emphasized the recursive, hierarchical complexity of such "higher cognition," but a strongly reductionist approach indicates that the core mechanisms are triadic. It is concluded that the cognitive skills traditionally considered to be "uniquely" human require three-way associational processing that most non-Primate animal species find difficult or impossible, but all members of Homo sapiens - regardless of small cultural differences - find easy and inherently intriguing.
Collapse
Affiliation(s)
- Norman D. Cook
- Department of Informatics, Kansai University, Osaka, Japan
| |
Collapse
|
5
|
Cook ND. Calculation of the acoustical properties of triadic harmonies. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2017; 142:3748. [PMID: 29289060 DOI: 10.1121/1.5018342] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
The author reports that the harmonic "tension" and major/minor "valence" of pitch combinations can be calculated directly from acoustical properties without relying on concepts from traditional harmony theory. The capability to compute the well-known types of harmonic triads means that their perception is not simply a consequence of learning an arbitrary cultural "idiom" handed down from the Italian Renaissance. On the contrary, for typical listeners familiar with diatonic music, attention to certain, definable, acoustical features underlies the perception of the valence (modality) and the inherent tension (instability) of three-tone harmonies.
Collapse
Affiliation(s)
- Norman D Cook
- Department of Informatics, Kansai University, 2-1 Reizenji, Takatsuki, Osaka, 569-1095, Japan
| |
Collapse
|
6
|
Musical chords and emotion: Major and minor triads are processed for emotion. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2014; 15:15-31. [DOI: 10.3758/s13415-014-0309-4] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
7
|
Chen L, Mao X, Wei P, Compare A. Speech emotional features extraction based on electroglottograph. Neural Comput 2013; 25:3294-317. [PMID: 24047321 DOI: 10.1162/neco_a_00523] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
This study proposes two classes of speech emotional features extracted from electroglottography (EGG) and speech signal. The power-law distribution coefficients (PLDC) of voiced segments duration, pitch rise duration, and pitch down duration are obtained to reflect the information of vocal folds excitation. The real discrete cosine transform coefficients of the normalized spectrum of EGG and speech signal are calculated to reflect the information of vocal tract modulation. Two experiments are carried out. One is of proposed features and traditional features based on sequential forward floating search and sequential backward floating search. The other is the comparative emotion recognition based on support vector machine. The results show that proposed features are better than those commonly used in the case of speaker-independent and content-independent speech emotion recognition.
Collapse
Affiliation(s)
- Lijiang Chen
- School of Electronic and Information Engineering, Beihang University, Beijing 100191, China
| | | | | | | |
Collapse
|
8
|
Van Puyvelde M, Loots G, Vinck B, De Coster L, Matthijs L, Mouvet K, Pattyn N. The Interplay Between Tonal Synchrony and Social Engagement in Mother-Infant Interaction. INFANCY 2013. [DOI: 10.1111/infa.12007] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Affiliation(s)
- Martine Van Puyvelde
- Research Group Interpersonal, Discursive and Narrative Studies Faculty of Psychology and Educational Sciences; Vrije Universiteit Brussel
| | | | - Bart Vinck
- Centre for Economy and Management; Hogeschool-Universiteit Brussel
| | - Lotta De Coster
- Faculty of Psychological and Educational Sciences; Université Libre de Bruxelles
| | - Liesbeth Matthijs
- Research Group Interpersonal, Discursive and Narrative Studies Faculty of Psychology and Educational Sciences; Vrije Universiteit Brussel
| | - Kimberley Mouvet
- Research Group Interpersonal, Discursive and Narrative Studies Faculty of Psychology and Educational Sciences; Vrije Universiteit Brussel
| | - Nathalie Pattyn
- Department Communication, Information; Systems and Sensors (CISS) Royal Military Academy Brussels (RMA); Faculty of Experimental and Applied Psychology; Vrije Universiteit Brussel (VUB)
| |
Collapse
|
9
|
Emotion-Aware Assistive System for Humanistic Care Based on the Orange Computing Concept. APPLIED COMPUTATIONAL INTELLIGENCE AND SOFT COMPUTING 2012. [DOI: 10.1155/2012/183610] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Mental care has become crucial with the rapid growth of economy and technology. However, recent movements, such as green technologies, place more emphasis on environmental issues than on mental care. Therefore, this study presents an emerging technology called orange computing for mental care applications. Orange computing refers to health, happiness, and physiopsychological care computing, which focuses on designing algorithms and systems for enhancing body and mind balance. The representative color of orange computing originates from a harmonic fusion of passion, love, happiness, and warmth. A case study on a human-machine interactive and assistive system for emotion care was conducted in this study to demonstrate the concept of orange computing. The system can detect emotional states of users by analyzing their facial expressions, emotional speech, and laughter in a ubiquitous environment. In addition, the system can provide corresponding feedback to users according to the results. Experimental results show that the system can achieve an accurate audiovisual recognition rate of 81.8% on average, thereby demonstrating the feasibility of the system. Compared with traditional questionnaire-based approaches, the proposed system can offer real-time analysis of emotional status more efficiently.
Collapse
|
10
|
Abstract
We have undertaken an fMRI study of harmony perception in order to determine the relationship between the diatonic triads of Western harmony and brain activation. Subjects were 12 right-handed, male non-musicians. All stimuli consisted of two harmonic triads that did not contain dissonant intervals of 1 or 2 semitones, but differed between them by 0, ±1, ±2 or ±3 semitones and therefore differed in terms of their inherent stability (major and minor chords) or instability (diminished and augmented chords). These musical stimuli were chosen on the basis of a psychoacoustical model of triadic harmony that has previously been shown to explain the fundamental regularities of traditional harmony theory. The brain response to the chords could be distinguished within the right orbitofrontal cortex and cuneus/posterior cingulate gyrus. Moreover, the strongest hemodynamic responses were found for conditions of rising pitch leading from harmonic tension to modal resolution.
Collapse
Affiliation(s)
- Takashi X Fujisawa
- Graduate School of Biomedical Sciences, Nagasaki University, Nagasaki, Japan
| | | |
Collapse
|
11
|
Rattanyu K, Mizukawa M. Emotion Recognition Based on ECG Signals for Service Robots in the Intelligent Space During Daily Life. JOURNAL OF ADVANCED COMPUTATIONAL INTELLIGENCE AND INTELLIGENT INFORMATICS 2011. [DOI: 10.20965/jaciii.2011.p0582] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper presents our approach for emotion recognition based on Electrocardiogram (ECG) signals. We propose to use the ECG’s inter-beat features together with within-beat features in our recognition system. In order to reduce the feature space, post hoc tests in the Analysis of Variance (ANOVA) were employed to select the set of eleven most significant features. We conducted experiments on twelve subjects using the International Affective Picture System (IAPS) database. RF-ECG sensors were attached to the subject’s skin to monitor the ECG signal via wireless connection. Results showed that our eleven feature approach outperforms the conventional three feature approach. For simultaneous classification of six emotional states: anger, fear, disgust, sadness, neutral, and joy, the Correct Classification Ratio (CCR) showed significant improvement from 37.23% to over 61.44%. Our system was able to monitor human emotion wirelessly without affecting the subject’s activities. Therefore it is suitable to be integrated with service robots to provide assistive and healthcare services.
Collapse
|