Emotion Recognition from ECG Signals Using Wavelet Scattering and Machine Learning

Transformer-Driven Affective State Recognition from Wearable Physiological Data in Everyday Contexts

The rapid advancement in wearable physiological measurement technology in recent years has brought affective computing closer to everyday life scenarios. Recognizing affective states in daily contexts holds significant potential for applications in human-computer interaction and psychiatry. Addressing the challenge of long-term, multi-modal physiological data in everyday settings, this study introduces a Transformer-based algorithm for affective state recognition, designed to fully exploit the temporal characteristics of signals and the interrelationships between different modalities. Utilizing the DAPPER dataset, which comprises continuous 5-day wrist-worn recordings of heart rate, skin conductance, and tri-axial acceleration from 88 subjects, our Transformer-based model achieved an average binary classification accuracy of 71.5% for self-reported positive or negative affective state sampled at random moments during daily data collection, and 60.29% and 61.55% for the five-class classification based on valence and arousal scores. The results of this study demonstrate the feasibility of applying affective state recognition based on wearable multi-modal physiological signals in everyday contexts.

A novel ECG-based approach for classifying psychiatric disorders: Leveraging wavelet scattering networks

Individuals with neuropsychiatric disorders experience both physical and mental difficulties, hindering their ability to live healthy lives and participate in daily activities. It is challenging to diagnose these disorders due to a lack of reliable diagnostic tests and the complex symptoms and treatments for various disorders. Generally, psychiatric disorders are identified manually by doctors using questionnaires, which may be prone to subjectivity and human errors. A few automated systems have recently been developed to identify these disorders using physiological signals, including electroencephalogram (EEG) and electrocardiogram (ECG) signals. Often, EEG signals are used to identify psychiatric disorders, but the EEG signals are nonlinear and non-stationary in nature and hence are relatively complex to analyze when compared to the ECG signals. The ECG signals in psychiatric patients are used due to the connection between the heart and brain. The proposed study is aimed at investigating the use of ECG signals for the automated identification of neuropsychiatric disorders, including bipolar disorder (BD), depression (DP), and schizophrenia (SZ). Generally, convolution neural networks (CNNs) have proven to be effective in accurately identifying psychological conditions. However, their application requires a large amount of data and technical expertise. The wavelet scattering network (WSN), a variant of CNNs, was introduced to overcome these limitations. The WSN is a network capable of accurately detecting unique patterns in the signal. The proposed research incorporated the WSN network and was conducted using a Psychiatric ECG Beat Dataset with a population of 233 subjects, of whom 198 were diagnosed with multiple psychiatric disorders, and 35 were control subjects. ECG signals from 3570 heartbeats were collected and analyzed using wavelet scattering-based feature extraction and machine learning techniques. The Fine K-Nearest Neighbor (FKNN) algorithm produced the best results with an average classification accuracy of 99.8% and a Kappa value of 0.996 using a ten-fold cross-validation. The model yielded an accuracy of 99.78%, 99.94%, 99.98%, and 100% for automated identification of BD, DP, SZ, and control subjects, respectively, with F1 scores and precision values close to 1. The proposed method could also help in the automated clinical detection of different psychiatric disorders.

EEG Dataset for the Recognition of Different Emotions Induced in Voice-User Interaction

Electroencephalography (EEG)-based open-access datasets are available for emotion recognition studies, where external auditory/visual stimuli are used to artificially evoke pre-defined emotions. In this study, we provide a novel EEG dataset containing the emotional information induced during a realistic human-computer interaction (HCI) using a voice user interface system that mimics natural human-to-human communication. To validate our dataset via neurophysiological investigation and binary emotion classification, we applied a series of signal processing and machine learning methods to the EEG data. The maximum classification accuracy ranged from 43.3% to 90.8% over 38 subjects and classification features could be interpreted neurophysiologically. Our EEG data could be used to develop a reliable HCI system because they were acquired in a natural HCI environment. In addition, auxiliary physiological data measured simultaneously with the EEG data also showed plausible results, i.e., electrocardiogram, photoplethysmogram, galvanic skin response, and facial images, which could be utilized for automatic emotion discrimination independently from, as well as together with the EEG data via the fusion of multi-modal physiological datasets.

Automated detection of depression using wavelet scattering networks

Today, depression is a common problem that affects many people all over the world. It can impact a person's mood and quality of life unless identified and treated immediately. Due to the hectic and stressful modern life seems to be, depression has become a leading cause of mental health illnesses. Signals from electroencephalograms (EEG) are frequently used to detect depression. It is difficult, time-consuming, and highly skilled to manually detect depression using EEG data analysis. Hence, in the proposed study, an automated depression detection system using EEG signals is proposed. The proposed study uses a clinically available dataset and dataset provided by the Department of Psychiatry at the Government Medical College (GMC) in Kozhikode, Kerala, India which consisted of 15 depressed patients and 15 healthy subjects and a publically available Multi-modal Open Dataset (MODMA) for Mental-disorder Analysis available at UK Data service reshare that consisted of 24 depressed patients and 29 healthy subjects. In this study, we have developed a novel Deep Wavelet Scattering Network (DWSN) for the automated detection of depression EEG signals. The best-performing classifier is then chosen by feeding the features into several machine-learning algorithms. For the clinically available GMC dataset, Medium Neural Network (MNN) achieved the highest accuracy of 99.95% with a Kappa value of 0.999. Using the suggested methods, the precision, recall, and F1-score are all 1. For the MODMA dataset, Wide Neural Network (WNN) achieved the highest accuracy of 99.3% with a Kappa value of 0.987. Using the suggested methods, the precision, recall, and F1-score are all 0.99. In comparison to all current methodologies, the performance of the suggested research is superior. The proposed method can be used to automatically diagnose depression both at home and in clinical settings.

ECG Multi-Emotion Recognition Based on Heart Rate Variability Signal Features Mining

Heart rate variability (HRV) serves as a significant physiological measure that mirrors the regulatory capacity of the cardiac autonomic nervous system. It not only indicates the extent of the autonomic nervous system's influence on heart function but also unveils the connection between emotions and psychological disorders. Currently, in the field of emotion recognition using HRV, most methods focus on feature extraction through the comprehensive analysis of signal characteristics; however, these methods lack in-depth analysis of the local features in the HRV signal and cannot fully utilize the information of the HRV signal. Therefore, we propose the HRV Emotion Recognition (HER) method, utilizing the amplitude level quantization (ALQ) technique for feature extraction. First, we employ the emotion quantification analysis (EQA) technique to impartially assess the semantic resemblance of emotions within the domain of emotional arousal. Then, we use the ALQ method to extract rich local information features by analyzing the local information in each frequency range of the HRV signal. Finally, the extracted features are classified using a logistic regression (LR) classification algorithm, which can achieve efficient and accurate emotion recognition. According to the experiment findings, the approach surpasses existing techniques in emotion recognition accuracy, achieving an average accuracy rate of 84.3%. Therefore, the HER method proposed in this paper can effectively utilize the local features in HRV signals to achieve efficient and accurate emotion recognition. This will provide strong support for emotion research in psychology, medicine, and other fields.

Classification of Low Earth Orbit (LEO) Resident Space Objects' (RSO) Light Curves Using a Support Vector Machine (SVM) and Long Short-Term Memory (LSTM)

Light curves are plots of brightness measured over time. In the field of Space Situational Awareness (SSA), light curves of Resident Space Objects (RSOs) can be utilized to infer information about an RSO such as the type of object, its attitude, and its shape. Light curves of RSOs in geostationary orbit (GEO) have been a main research focus for many years due to the availability of long time series data spanning hours. Given that a large portion of RSOs are in low Earth orbit (LEO), it is of great importance to study trends in LEO light curves as well. The challenge with LEO light curves is that they tend to be short, typically no longer than a few minutes, which makes them difficult to analyze with typical time series techniques. This study presents a novel approach to observational LEO light curve classification. We extract features from light curves using a wavelet scattering transformation which is used as an input for a machine learning classifier. We performed light curve classification using both a conventional machine learning approach, namely a support vector machine (SVM), and a deep learning technique, long short-term memory (LSTM), to compare the results. LSTM outperforms SVM for LEO light curve classification with a 92% accuracy. This proves the viability of RSO classification by object type and spin rate from real LEO light curves.

Design and Development of a Non-Contact ECG-Based Human Emotion Recognition System Using SVM and RF Classifiers

Emotion recognition becomes an important aspect in the development of human-machine interaction (HMI) systems. Positive emotions impact our lives positively, whereas negative emotions may cause a reduction in productivity. Emotionally intelligent systems such as chatbots and artificially intelligent assistant modules help make our daily life routines effortless. Moreover, a system which is capable of assessing the human emotional state would be very helpful to assess the mental state of a person. Hence, preventive care could be offered before it becomes a mental illness or slides into a state of depression. Researchers have always been curious to find out if a machine could assess human emotions precisely. In this work, a unimodal emotion classifier system in which one of the physiological signals, an electrocardiogram (ECG) signal, has been used is proposed to classify human emotions. The ECG signal was acquired using a capacitive sensor-based non-contact ECG belt system. The machine-learning-based classifiers developed in this work are SVM and random forest with 10-fold cross-validation on three different sets of ECG data acquired for 45 subjects (15 subjects in each age group). The minimum classification accuracies achieved with SVM and RF emotion classifier models are 86.6% and 98.2%, respectively.

Electrocardiogram Monitoring Wearable Devices and Artificial-Intelligence-Enabled Diagnostic Capabilities: A Review

Worldwide, population aging and unhealthy lifestyles have increased the incidence of high-risk health conditions such as cardiovascular diseases, sleep apnea, and other conditions. Recently, to facilitate early identification and diagnosis, efforts have been made in the research and development of new wearable devices to make them smaller, more comfortable, more accurate, and increasingly compatible with artificial intelligence technologies. These efforts can pave the way to the longer and continuous health monitoring of different biosignals, including the real-time detection of diseases, thus providing more timely and accurate predictions of health events that can drastically improve the healthcare management of patients. Most recent reviews focus on a specific category of disease, the use of artificial intelligence in 12-lead electrocardiograms, or on wearable technology. However, we present recent advances in the use of electrocardiogram signals acquired with wearable devices or from publicly available databases and the analysis of such signals with artificial intelligence methods to detect and predict diseases. As expected, most of the available research focuses on heart diseases, sleep apnea, and other emerging areas, such as mental stress. From a methodological point of view, although traditional statistical methods and machine learning are still widely used, we observe an increasing use of more advanced deep learning methods, specifically architectures that can handle the complexity of biosignal data. These deep learning methods typically include convolutional and recurrent neural networks. Moreover, when proposing new artificial intelligence methods, we observe that the prevalent choice is to use publicly available databases rather than collecting new data.

A new deep convolutional neural network incorporating attentional mechanisms for ECG emotion recognition

Using ECG signals captured by wearable devices for emotion recognition is a feasible solution. We propose a deep convolutional neural network incorporating attentional mechanisms for ECG emotion recognition. In order to address the problem of individuality differences in emotion recognition tasks, we incorporate an improved Convolutional Block Attention Module (CBAM) into the proposed deep convolutional neural network. The deep convolutional neural network is responsible for capturing ECG features. Channel attention in CBAM is responsible for adding weight information to ECG features of different channels and spatial attention is responsible for the weighted representation of ECG features of different regions inside the channel. We used three publicly available datasets, WESAD, DREAMER, and ASCERTAIN, for the ECG emotion recognition task. The new state-of-the-art results are set in three datasets for multi-class classification results, WESAD for tri-class results, and ASCERTAIN for two-category results, respectively. A large number of experiments are performed, providing an interesting analysis of the design of the convolutional structure parameters and the role of the attention mechanism used. We propose to use large convolutional kernels to improve the effective perceptual field of the model and thus fully capture the ECG signal features, which achieves better performance compared to the commonly used small kernels. In addition, channel attention and spatial attention were added to the deep convolutional model separately to explore their contribution levels. We found that in most cases, channel attention contributed to the model at a higher level than spatial attention.

A System of Emotion Recognition and Judgment and Its Application in Adaptive Interactive Game

A system of emotion recognition and judgment (SERJ) based on a set of optimal signal features is established, and an emotion adaptive interactive game (EAIG) is designed. The change in a player's emotion can be detected with the SERJ during the process of playing the game. A total of 10 subjects were selected to test the EAIG and SERJ. The results show that the SERJ and designed EAIG are effective. The game adapted itself by judging the corresponding special events triggered by a player's emotion and, as a result, enhanced the player's game experience. It was found that, in the process of playing the game, a player's perception of the change in emotion was different, and the test experience of a player had an effect on the test results. A SERJ that is based on a set of optimal signal features is better than a SERJ that is based on the conventional machine learning-based method.

Classification of Motor-Imagery Tasks Using a Large EEG Dataset by Fusing Classifiers Learning on Wavelet-Scattering Features

Brain-computer or brain-machine interface technology allows humans to control machines using their thoughts via brain signals. In particular, these interfaces can assist people with neurological diseases for speech understanding or physical disabilities for operating devices such as wheelchairs. Motor-imagery tasks play a basic role in brain-computer interfaces. This study introduces an approach for classifying motor-imagery tasks in a brain-computer interface environment, which remains a challenge for rehabilitation technology using electroencephalogram sensors. Methods used and developed for addressing the classification include wavelet time and image scattering networks, fuzzy recurrence plots, support vector machines, and classifier fusion. The rationale for combining outputs from two classifiers learning on wavelet-time and wavelet-image scattering features of brain signals, respectively, is that they are complementary and can be effectively fused using a novel fuzzy rule-based system. A large-scale challenging electroencephalogram dataset of motor imagery-based brain-computer interface was used to test the efficacy of the proposed approach. Experimental results obtained from within-session classification show the potential application of the new model that achieves an improvement of 7% in classification accuracy over the best existing classifier using state-of-the-art artificial intelligence (76% versus 69%, respectively). For the cross-session experiment, which imposes a more challenging and practical classification task, the proposed fusion model improves the accuracy by 11% (54% versus 65%). The technical novelty presented herein and its further exploration are promising for developing a reliable sensor-based intervention for assisting people with neurodisability to improve their quality of life.

A Review on the Applications of Time-Frequency Methods in ECG Analysis

The joint time-frequency analysis method represents a signal in both time and frequency. Thus, it provides more information compared to other one-dimensional methods. Several researchers recently used time-frequency methods such as the wavelet transform, short-time Fourier transform, empirical mode decomposition and reported impressive results in various electrophysiological studies. The current review provides comprehensive knowledge about different time-frequency methods and their applications in various ECG-based analyses. Typical applications include ECG signal denoising, arrhythmia detection, sleep apnea detection, biometric identification, emotion detection, and driver drowsiness detection. The paper also discusses the limitations of these methods. The review will form a reference for future researchers willing to conduct research in the same field.

Emotion Recognition: Photoplethysmography and Electrocardiography in Comparison

Automatically recognizing negative emotions, such as anger or stress, and also positive ones, such as euphoria, can contribute to improving well-being. In real-life, emotion recognition is a difficult task since many of the technologies used for this purpose in both laboratory and clinic environments, such as electroencephalography (EEG) and electrocardiography (ECG), cannot realistically be used. Photoplethysmography (PPG) is a non-invasive technology that can be easily integrated into wearable sensors. This paper focuses on the comparison between PPG and ECG concerning their efficacy in detecting the psychophysical and affective states of the subjects. It has been confirmed that the levels of accuracy in the recognition of affective variables obtained by PPG technology are comparable to those achievable with the more traditional ECG technology. Moreover, the affective psychological condition of the participants (anxiety and mood levels) may influence the psychophysiological responses recorded during the experimental tests.

Automated Emotion Identification Using Fourier-Bessel Domain-Based Entropies

Human dependence on computers is increasing day by day; thus, human interaction with computers must be more dynamic and contextual rather than static or generalized. The development of such devices requires knowledge of the emotional state of the user interacting with it; for this purpose, an emotion recognition system is required. Physiological signals, specifically, electrocardiogram (ECG) and electroencephalogram (EEG), were studied here for the purpose of emotion recognition. This paper proposes novel entropy-based features in the Fourier-Bessel domain instead of the Fourier domain, where frequency resolution is twice that of the latter. Further, to represent such non-stationary signals, the Fourier-Bessel series expansion (FBSE) is used, which has non-stationary basis functions, making it more suitable than the Fourier representation. EEG and ECG signals are decomposed into narrow-band modes using FBSE-based empirical wavelet transform (FBSE-EWT). The proposed entropies of each mode are computed to form the feature vector, which are further used to develop machine learning models. The proposed emotion detection algorithm is evaluated using publicly available DREAMER dataset. K-nearest neighbors (KNN) classifier provides accuracies of 97.84%, 97.91%, and 97.86% for arousal, valence, and dominance classes, respectively. Finally, this paper concludes that the obtained entropy features are suitable for emotion recognition from given physiological signals.

Wearable Electrocardiogram Quality Assessment Using Wavelet Scattering and LSTM

As the fast development of wearable devices and Internet of things technologies, real-time monitoring of ECG signals is quite critical for cardiovascular diseases. However, dynamic ECG signals recorded in free-living conditions suffered from extremely serious noise pollution. Presently, most algorithms for ECG signal evaluation were designed to divide signals into acceptable and unacceptable. Such classifications were not enough for real-time cardiovascular disease monitoring. In the study, a wearable ECG quality database with 50,085 recordings was built, including A/B/C (or high quality/medium quality/low quality) three quality grades (A: high quality signals can be used for CVD detection; B: slight contaminated signals can be used for heart rate extracting; C: heavily polluted signals need to be abandoned). A new SQA classification method based on a three-layer wavelet scattering network and transfer learning LSTM was proposed in this study, which can extract more systematic and comprehensive characteristics by analyzing the signals thoroughly and deeply. Experimental results (mACC = 98.56%, mF₁ = 98.55%, Se_A = 97.90%, Se_B = 98.16%, Se_C = 99.60%, +P_A = 98.52%, +P_B = 97.60%, +P_C = 99.54%, F_1A = 98.20%, F_1B = 97.90%, F_1C = 99.60%) and real data validations proved that this proposed method showed the high accuracy, robustness, and computationally efficiency. It has the ability to evaluate the long-term dynamic ECG signal quality. It is advantageous to promoting cardiovascular disease monitoring by removing contaminating signals and selecting high-quality signal segments for further analysis.

Model of Emotion Judgment Based on Features of Multiple Physiological Signals

The model of emotion judgment based on features of multiple physiological signals was investi-gated. In total, 40 volunteers participated in the experiment by playing a computer game while their physiological signals (skin electricity, electrocardiogram (ECG), pulse wave, and facial electromy-ogram (EMG)) were acquired. The volunteers were asked to complete an emotion questionnaire where six typical events that appeared in the game were included, and each volunteer rated their own emotion when experiencing the six events. Based on the analysis of game events, the signal data were cut into segments and the emotional trends were classified. The correlation between data segments and emotional trends was built using a statistical method combined with the questionnaire responses. The set of optimal signal features was obtained by processing the data of physiological signals, extracting the features of signal data, reducing the dimensionality of signal features, and classifying the emotion based on the set of signal data. Finally, the model of emotion judgment was established by selecting the features with a significance of 0.01 based on the correlation between the features in the set of optimal signal features and emotional trends.

Research on recognition and classification of pulse signal features based on EPNCC

To rapidly obtain the complete characterization information of pulse signals and to verify the sensitivity and validity of pulse signals in the clinical diagnosis of related diseases. In this paper, an improved PNCC method is proposed as a supplementary feature to enable the complete characterization of pulse signals. In this paper, the wavelet scattering method is used to extract time-domain features from impulse signals, and EEMD-based improved PNCC (EPNCC) is used to extract frequency-domain features. The time–frequency features are mixed into a convolutional neural network for final classification and recognition. The data for this study were obtained from the MIT-BIH-mimic database, which was used to verify the effectiveness of the proposed method. The experimental analysis of three types of clinical symptom pulse signals showed an accuracy of 98.3% for pulse classification and recognition. The method is effective in complete pulse characterization and improves pulse classification accuracy under the processing of the three clinical pulse signals used in the paper.

Classification of heart sounds based on quality assessment and wavelet scattering transform

Automatic classification of heart sound plays an important role in the diagnosis of cardiovascular diseases. In this study, a heart sound sample classification method based on quality assessment and wavelet scattering transform was proposed. First, the ratio of zero crossings (RZC) and the root mean square of successive differences (RMSSD) were used for assessing the quality of heart sound signal. The first signal segment conforming to the threshold standard was selected as the current sample for the continuous heart sound signal. Using the wavelet scattering transform, the wavelet scattering coefficients were expanded according to the wavelet scale dimension, to obtain the features. Support vector machine (SVM) was used for classification, and the classification results for the samples were obtained using the wavelet scale dimension voting approach. The effects of RZC and RMSSD on the results are discussed in detail. On the database of PhysioNet Computing in Cardiology Challenge 2016 (CinC 2016), the proposed method yields 92.23% accuracy (Acc), 96.62% sensitivity (Se), 90.65% specificity (Sp), and 93.64% measure of accuracy (Macc). The results show that the proposed method can effectively classify normal and abnormal heart sound samples with high accuracy.