Emotion classification with multichannel physiological signals using hybrid feature and adaptive decision fusion

Multimodal optimal matching and augmentation method for small sample gesture recognition

In human-computer interaction, gesture recognition based on physiological signals offers advantages such as a more natural and fast interaction mode and less constrained by the environment than visual-based. Surface electromyography-based gesture recognition has significantly progressed. However, since individuals have physical differences, researchers must collect data multiple times from each user to train the deep learning model. This data acquisition process can be particularly burdensome for non-healthy users. Researchers are currently exploring transfer learning and data augmentation techniques to enhance the accuracy of small-sample gesture recognition models. However, challenges persist, such as negative transfer and limited diversity in training samples, leading to suboptimal recognition performance. Therefore, We introduce motion information into sEMG-based recognition and propose a multimodal optimal matching and augmentation method for small sample gesture recognition, achieving efficient gesture recognition with only one acquisition per gesture. Firstly, this method utilizes the optimal matching signal selection module to select the most similar signals from the existing data to the new user as the training set, reducing inter-domain differences. Secondly, the similarity calculation augmentation module enhances the diversity of the training set. Finally, the Modal-type embedding enhances the information interaction between each mode signal. We evaluated the effectiveness on Self-collected Stroke Patient, the Ninapro DB1 dataset and the Ninapro DB5 dataset and achieved accuracies of 93.69%, 91.65% and 98.56%, respectively. These results demonstrate that the method achieved performance comparable to traditional recognition models while significantly reducing the collected data.

Dynamic emotion intensity estimation from physiological signals facilitating interpretation via appraisal theory

Appraisal models, such as the Scherer's Component Process Model (CPM), represent an elegant framework for the interpretation of emotion processes, advocating for computational models that capture emotion dynamics. Today's emotion recognition research, however, typically classifies discrete qualities or categorised dimensions, neglecting the dynamic nature of emotional processes and thus limiting interpretability based on appraisal theory. In our research, we estimate emotion intensity from multiple physiological features associated to the CPM's neurophysiological component using dynamical models with the aim of bringing insights into the relationship between physiological dynamics and perceived emotion intensity. To this end, we employ nonlinear autoregressive exogeneous (NARX) models, as their parameters can be interpreted within the CPM. In our experiment, emotions of varying intensities are induced for three distinct qualities while physiological signals are measured, and participants assess their subjective feeling in real time. Using data-extracted physiological features, we train intrasubject and intersubject intensity models using a genetic algorithm, which outperform traditional sliding-window linear regression, providing a robust basis for interpretation. The NARX model parameters obtained, interpreted by appraisal theory, indicate consistent heart rate parameters in the intersubject models, suggesting a large temporal contribution that aligns with the CPM-predicted changes.

Prenatal anxiety recognition model integrating multimodal physiological signal

Anxiety among pregnant women can significantly impact their overall well-being. However, the development of data-driven HCI interventions for this demographic is often hindered by data scarcity and collection challenges. In this study, we leverage the Empatica E4 wristband to gather physiological data from pregnant women in both resting and relaxed states. Additionally, we collect subjective reports on their anxiety levels. We integrate features from signals including Blood Volume Pulse (BVP), Skin Temperature (SKT), and Inter-Beat Interval (IBI). Employing a Support Vector Machine (SVM) algorithm, we construct a model capable of evaluating anxiety levels in pregnant women. Our model attains an emotion recognition accuracy of 69.3%, marking achievements in HCI technology tailored for this specific user group. Furthermore, we introduce conceptual ideas for biofeedback on maternal emotions and its interactive mechanism, shedding light on improved monitoring and timely intervention strategies to enhance the emotional health of pregnant women.

Analysis of Physiological Response of Attention and Stress States in Teleoperation Performance of Social Tasks

Some studies addressed monitoring mental states by physiological responses analysis in robots' teleoperation in traditional applications such as inspection and exploration; however, no study analyzed the physiological response during teleoperated social tasks to the best of our knowledge. We analyzed the physiological response of attention and stress mental states by computing the correlation between multimodal biomarkers and performance, pleasure-arousal scale, and workload. Physiological data were recorded during simulated teleoperated social tasks to induce mental states, such as normal, attention, and stress. The results showed that task performance and workload subscales achieved moderate correlations with some multimodal biomarkers. The correlations depended on the induced state. The cognitive workload was related to brain biomarkers of attention in the frontal and frontal-central regions. These regions were close to the frontopolar region, which is commonly reported in attentional studies. Thus, some multimodal biomarkers of attention and stress mental states could monitor or predict metrics related to the performance in teleoperation of social tasks.

A System of Emotion Recognition and Judgment and Its Application in Adaptive Interactive Game

A system of emotion recognition and judgment (SERJ) based on a set of optimal signal features is established, and an emotion adaptive interactive game (EAIG) is designed. The change in a player's emotion can be detected with the SERJ during the process of playing the game. A total of 10 subjects were selected to test the EAIG and SERJ. The results show that the SERJ and designed EAIG are effective. The game adapted itself by judging the corresponding special events triggered by a player's emotion and, as a result, enhanced the player's game experience. It was found that, in the process of playing the game, a player's perception of the change in emotion was different, and the test experience of a player had an effect on the test results. A SERJ that is based on a set of optimal signal features is better than a SERJ that is based on the conventional machine learning-based method.

Feature extraction of 3D Chinese rose model based on color and shape features

Flower classification is of great importance to the research fields of plants, food, and medicine. Due to more abundant information on three-dimensional (3D) flower models than two-dimensional 2D images, it makes the 3D models more suitable for flower classification tasks. In this study, a feature extraction and classification method were proposed based on the 3D models of Chinese roses. Firstly, the shape distribution method was used to extract the sharpness and contour features of 3D flower models, and the color features were obtained from the Red-Green-Blue (RGB) color space. Then, the RF-OOB method was employed to rank the extracted flower features. A shape descriptor based on the unique attributes of Chinese roses was constructed, χ² distance was adopted to measure the similarity between different Chinese roses. Experimental results show that the proposed method was effective for the retrieval and classification tasks of Chinese roses, and the average classification accuracy was approximately 87%, which can meet the basic retrieval requirements of 3D flower models. The proposed method promotes the classification of Chinese roses from 2D space to 3D space, which broadens the research method of flower classification.

Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals

The use of machine learning (ML) techniques in affective computing applications focuses on improving the user experience in emotion recognition. The collection of input data (e.g., physiological signals), together with expert annotations are part of the established standard supervised learning methodology used to train human emotion recognition models. However, these models generally require large amounts of labeled data, which is expensive and impractical in the healthcare context, in which data annotation requires even more expert knowledge. To address this problem, this paper explores the use of the self-supervised learning (SSL) paradigm in the development of emotion recognition methods. This approach makes it possible to learn representations directly from unlabeled signals and subsequently use them to classify affective states. This paper presents the key concepts of emotions and how SSL methods can be applied to recognize affective states. We experimentally analyze and compare self-supervised and fully supervised training of a convolutional neural network designed to recognize emotions. The experimental results using three emotion datasets demonstrate that self-supervised representations can learn widely useful features that improve data efficiency, are widely transferable, are competitive when compared to their fully supervised counterparts, and do not require the data to be labeled for learning.

Model of Emotion Judgment Based on Features of Multiple Physiological Signals

The model of emotion judgment based on features of multiple physiological signals was investi-gated. In total, 40 volunteers participated in the experiment by playing a computer game while their physiological signals (skin electricity, electrocardiogram (ECG), pulse wave, and facial electromy-ogram (EMG)) were acquired. The volunteers were asked to complete an emotion questionnaire where six typical events that appeared in the game were included, and each volunteer rated their own emotion when experiencing the six events. Based on the analysis of game events, the signal data were cut into segments and the emotional trends were classified. The correlation between data segments and emotional trends was built using a statistical method combined with the questionnaire responses. The set of optimal signal features was obtained by processing the data of physiological signals, extracting the features of signal data, reducing the dimensionality of signal features, and classifying the emotion based on the set of signal data. Finally, the model of emotion judgment was established by selecting the features with a significance of 0.01 based on the correlation between the features in the set of optimal signal features and emotional trends.