Activity Recognition Using Wearable Physiological Measurements: Selection of Features from a Comprehensive Literature Study

The State-of-the-Art Sensing Techniques in Human Activity Recognition: A Survey

Human activity recognition (HAR) has become an intensive research topic in the past decade because of the pervasive user scenarios and the overwhelming development of advanced algorithms and novel sensing approaches. Previous HAR-related sensing surveys were primarily focused on either a specific branch such as wearable sensing and video-based sensing or a full-stack presentation of both sensing and data processing techniques, resulting in weak focus on HAR-related sensing techniques. This work tries to present a thorough, in-depth survey on the state-of-the-art sensing modalities in HAR tasks to supply a solid understanding of the variant sensing principles for younger researchers of the community. First, we categorized the HAR-related sensing modalities into five classes: mechanical kinematic sensing, field-based sensing, wave-based sensing, physiological sensing, and hybrid/others. Specific sensing modalities are then presented in each category, and a thorough description of the sensing tricks and the latest related works were given. We also discussed the strengths and weaknesses of each modality across the categorization so that newcomers could have a better overview of the characteristics of each sensing modality for HAR tasks and choose the proper approaches for their specific application. Finally, we summarized the presented sensing techniques with a comparison concerning selected performance metrics and proposed a few outlooks on the future sensing techniques used for HAR tasks.

Datasets for Automated Affect and Emotion Recognition from Cardiovascular Signals Using Artificial Intelligence- A Systematic Review

Simple Summary

We reviewed the literature on the publicly available datasets used to automatically recognise emotion and affect using artificial intelligence (AI) techniques. We were particularly interested in databases with cardiovascular (CV) data. Additionally, we assessed the quality of the included papers. We searched the sources until 31 August 2020. Each step of identification was carried out independently by two reviewers to maintain the credibility of our review. In case of disagreement, we discussed them. Each action was first planned and described in a protocol that we posted on the Open Science Framework (OSF) platform. We selected 18 works focused on providing datasets of CV signals for automated affect and emotion recognition. In total, data for 812 participants aged 17 to 47 were analysed. The most frequently recorded signal was electrocardiography. The authors most often used video stimulation. Noticeably, we did not find much necessary information in many of the works, resulting in mainly low quality among included papers. Researchers in this field should focus more on how they carry out experiments.

Abstract

Our review aimed to assess the current state and quality of publicly available datasets used for automated affect and emotion recognition (AAER) with artificial intelligence (AI), and emphasising cardiovascular (CV) signals. The quality of such datasets is essential to create replicable systems for future work to grow. We investigated nine sources up to 31 August 2020, using a developed search strategy, including studies considering the use of AI in AAER based on CV signals. Two independent reviewers performed the screening of identified records, full-text assessment, data extraction, and credibility. All discrepancies were resolved by discussion. We descriptively synthesised the results and assessed their credibility. The protocol was registered on the Open Science Framework (OSF) platform. Eighteen records out of 195 were selected from 4649 records, focusing on datasets containing CV signals for AAER. Included papers analysed and shared data of 812 participants aged 17 to 47. Electrocardiography was the most explored signal (83.33% of datasets). Authors utilised video stimulation most frequently (52.38% of experiments). Despite these results, much information was not reported by researchers. The quality of the analysed papers was mainly low. Researchers in the field should concentrate more on methodology.

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

Emotion Recognition Based on Skin Potential Signals with a Portable Wireless Device

Emotion recognition is of great importance for artificial intelligence, robots, and medicine etc. Although many techniques have been developed for emotion recognition, with certain successes, they rely heavily on complicated and expensive equipment. Skin potential (SP) has been recognized to be correlated with human emotions for a long time, but has been largely ignored due to the lack of systematic research. In this paper, we propose a single SP-signal-based method for emotion recognition. Firstly, we developed a portable wireless device to measure the SP signal between the middle finger and left wrist. Then, a video induction experiment was designed to stimulate four kinds of typical emotion (happiness, sadness, anger, fear) in 26 subjects. Based on the device and video induction, we obtained a dataset consisting of 397 emotion samples. We extracted 29 features from each of the emotion samples and used eight well-established algorithms to classify the four emotions based on these features. Experimental results show that the gradient-boosting decision tree (GBDT), logistic regression (LR) and random forest (RF) algorithms achieved the highest accuracy of 75%. The obtained accuracy is similar to, or even better than, that of other methods using multiple physiological signals. Our research demonstrates the feasibility of the SP signal’s integration into existing physiological signals for emotion recognition.

How to Use Heart Rate Variability: Quantification of Vagal Activity in Toddlers and Adults in Long-Term ECG

Recent developments in noninvasive electrocardiogram (ECG) monitoring with small, wearable sensors open the opportunity to record high-quality ECG over many hours in an easy and non-burdening way. However, while their recording has been tremendously simplified, the interpretation of heart rate variability (HRV) data is a more delicate matter. The aim of this paper is to supply detailed methodological discussion and new data material in order to provide a helpful notice of HRV monitoring issues depending on recording conditions and study populations. Special consideration is given to the monitoring over long periods, across periods with different levels of activity, and in adults versus children. Specifically, the paper aims at making users aware of neglected methodological limitations and at providing substantiated recommendations for the selection of appropriate HRV variables and their interpretation. To this end, 30-h HRV data of 48 healthy adults (18–40 years) and 47 healthy toddlers (16–37 months) were analyzed in detail. Time-domain, frequency-domain, and nonlinear HRV variables were calculated after strict signal preprocessing, using six different high-frequency band definitions including frequency bands dynamically adjusted for the individual respiration rate. The major conclusion of the in-depth analyses is that for most applications that implicate long-term monitoring across varying circumstances and activity levels in healthy individuals, the time-domain variables are adequate to gain an impression of an individual’s HRV and, thus, the dynamic adaptation of an organism’s behavior in response to the ever-changing demands of daily life. The sound selection and interpretation of frequency-domain variables requires considerably more consideration of physiological and mathematical principles. For those who prefer using frequency-domain variables, the paper provides detailed guidance and recommendations for the definition of appropriate frequency bands in compliance with their specific recording conditions and study populations.

Arousal Detection in Elderly People from Electrodermal Activity Using Musical Stimuli

The detection of emotions is fundamental in many areas related to health and well-being. This paper presents the identification of the level of arousal in older people by monitoring their electrodermal activity (EDA) through a commercial device. The objective was to recognize arousal changes to create future therapies that help them to improve their mood, contributing to reduce possible situations of depression and anxiety. To this end, some elderly people in the region of Murcia were exposed to listening to various musical genres (flamenco, Spanish folklore, Cuban genre and rock/jazz) that they heard in their youth. Using methods based on the process of deconvolution of the EDA signal, two different studies were carried out. The first, of a purely statistical nature, was based on the search for statistically significant differences for a series of temporal, morphological, statistical and frequency features of the processed signals. It was found that Flamenco and Spanish Folklore presented the highest number of statistically significant parameters. In the second study, a wide range of classifiers was used to analyze the possible correlations between the detection of the EDA-based arousal level compared to the participants' responses to the level of arousal subjectively felt. In this case, it was obtained that the best classifiers are support vector machines, with 87% accuracy for flamenco and 83.1% for Spanish Folklore, followed by K-nearest neighbors with 81.4% and 81.5% for Flamenco and Spanish Folklore again. These results reinforce the notion of familiarity with a musical genre on emotional induction.

A Wrapper Feature Selection Algorithm: An Emotional Assessment Using Physiological Recordings from Wearable Sensors

Assessing emotional state is an emerging application field boosting research activities on the topic of analysis of non-invasive biosignals to find effective markers to accurately determine the emotional state in real-time. Nowadays using wearable sensors, electrocardiogram and thoracic impedance measurements can be recorded, facilitating analyzing cardiac and respiratory functions directly and autonomic nervous system function indirectly. Such analysis allows distinguishing between different emotional states: neutral, sadness, and disgust. This work was specifically focused on the proposal of a k-fold approach for selecting features while training the classifier that reduces the loss of generalization. The performance of the proposed algorithm used as the selection criterion was compared to the commonly used standard error function. The proposed k-fold approach outperforms the conventional method with 4% hit success rate improvement, reaching an accuracy near to 78%. Moreover, the proposed selection criterion method allows the classifier to produce the best performance using a lower number of features at lower computational cost. A reduced number of features reduces the risk of overfitting while a lower computational cost contributes to implementing real-time systems using wearable electronics.