Wireless Sensors System for Stress Detection by Means of ECG and EDA Acquisition

Advanced Necklace for Real-Time PPG Monitoring in Drivers

Monitoring heart rate (HR) through photoplethysmography (PPG) signals is a challenging task due to the complexities involved, even during routine daily activities. These signals can indeed be heavily contaminated by significant motion artifacts resulting from the subjects' movements, which can lead to inaccurate heart rate estimations. In this paper, our objective is to present an innovative necklace sensor that employs low-computational-cost algorithms for heart rate estimation in individuals performing non-abrupt movements, specifically drivers. Our solution facilitates the acquisition of signals with limited motion artifacts and provides acceptable heart rate estimations at a low computational cost. More specifically, we propose a wearable sensor necklace for assessing a driver's well-being by providing information about the driver's physiological condition and potential stress indicators through HR data. This innovative necklace enables real-time HR monitoring within a sleek and ergonomic design, facilitating seamless and continuous data gathering while driving. Prioritizing user comfort, the necklace's design ensures ease of wear, allowing for extended use without disrupting driving activities. The collected physiological data can be transmitted wirelessly to a mobile application for instant analysis and visualization. To evaluate the sensor's performance, two algorithms for estimating the HR from PPG signals are implemented in a microcontroller: a modified version of the mountaineer's algorithm and a sliding discrete Fourier transform. The goal of these algorithms is to detect meaningful peaks corresponding to each heartbeat by using signal processing techniques to remove noise and motion artifacts. The developed design is validated through experiments conducted in a simulated driving environment in our lab, during which drivers wore the sensor necklace. These experiments demonstrate the reliability of the wearable sensor necklace in capturing dynamic changes in HR levels associated with driving-induced stress. The algorithms integrated into the sensor are optimized for low computational cost and effectively remove motion artifacts that occur when users move their heads.

Wrist-Worn Sensor Validation for Heart Rate Variability and Electrodermal Activity Detection in a Stressful Driving Environment

Wearable sensors are widely used to gather psychophysiological data in the laboratory and real-world applications. However, the accuracy of these devices should be carefully assessed. The study focused on testing the accuracy of the Empatica 4 (E4) wristband for the detection of heart rate variability (HRV) and electrodermal activity (EDA) metrics in stress-inducing conditions and growing-risk driving scenarios. Fourteen healthy subjects were recruited for the experimental campaign, where HRV and EDA were recorded over six experimental conditions (Baseline, Video Clip, Scream, No-Risk Driving, Low-Risk Driving, and High-Risk Driving) and by means of two measurement systems: the E4 device and a gold standard system. The overall quality of the E4 data was investigated; agreement and reliability were assessed by performing a Bland-Altman analysis and by computing the Spearman's correlation coefficient. HRV time-domain parameters reported high reliability levels in Baseline (r > 0.72), Video Clip (r > 0.71), and No-Risk Driving (r > 0.67), while HRV frequency domain parameters were sufficient in Baseline (r > 0.58), Video Clip (r > 0.59), No-Risk (r > 0.51), and Low-Risk Driving (r > 0.52). As for the EDA parameters, no correlation was found. Further studies could enhance the HRV and EDA quality through further optimizations of the acquisition protocol and improvement of the processing algorithms.

Time-Dependent Analysis of Human Neurophysiological Activities during an Ecological Olfactory Experience

It has been demonstrated that odors could affect humans at the psychophysiological level. Significant research has been done on odor perception and physiological mechanisms; however, this research was mainly performed in highly controlled conditions in order to highlight the perceptive phenomena and the correlated physiological responses in the time frame of milliseconds. The present study explored how human physiological activity evolves in response to different odor conditions during an ecological olfactory experience on a broader time scale (from 1 to 90 s). Two odors, vanilla and menthol, together with a control condition (blank) were employed as stimuli. Electroencephalographic (EEG) activity in four frequency bands of interest, theta, alpha, low beta, and high beta, and the electrodermal activity (EDA) of the skin conductance level and response (SCL and SCR) were investigated at five time points taken during: (i) the first ten seconds of exposure (short-term analysis) and (ii) throughout the entire exposure to each odor (90 s, long-term analysis). The results revealed significant interactions between the odor conditions and the time periods in the short-term analysis for the overall frontal activity in the theta (p = 0.03), alpha (p = 0.005), and low beta (p = 0.0067) bands, the frontal midline activity in the alpha (p = 0.015) and low beta (p = 0.02) bands, and the SCR component (p = 0.024). For the long-term effects, instead, only one EEG parameter, frontal alpha asymmetry, was significantly sensitive to the considered dimensions (p = 0.037). In conclusion, the present research determined the physiological response to different odor conditions, also demonstrating the sensitivity of the employed parameters in characterizing the dynamic of such response during the time. As an exploratory study, this work points out the relevance of considering the effects of continuous exposure instead of short stimulation when evaluating the human olfactory experience, providing insights for future studies in the field.

Drivers' Mental Engagement Analysis Using Multi-Sensor Fusion Approaches Based on Deep Convolutional Neural Networks

In this paper, we present a comprehensive assessment of individuals' mental engagement states during manual and autonomous driving scenarios using a driving simulator. Our study employed two sensor fusion approaches, combining the data and features of multimodal signals. Participants in our experiment were equipped with Electroencephalogram (EEG), Skin Potential Response (SPR), and Electrocardiogram (ECG) sensors, allowing us to collect their corresponding physiological signals. To facilitate the real-time recording and synchronization of these signals, we developed a custom-designed Graphical User Interface (GUI). The recorded signals were pre-processed to eliminate noise and artifacts. Subsequently, the cleaned data were segmented into 3 s windows and labeled according to the drivers' high or low mental engagement states during manual and autonomous driving. To implement sensor fusion approaches, we utilized two different architectures based on deep Convolutional Neural Networks (ConvNets), specifically utilizing the Braindecode Deep4 ConvNet model. The first architecture consisted of four convolutional layers followed by a dense layer. This model processed the synchronized experimental data as a 2D array input. We also proposed a novel second architecture comprising three branches of the same ConvNet model, each with four convolutional layers, followed by a concatenation layer for integrating the ConvNet branches, and finally, two dense layers. This model received the experimental data from each sensor as a separate 2D array input for each ConvNet branch. Both architectures were evaluated using a Leave-One-Subject-Out (LOSO) cross-validation approach. For both cases, we compared the results obtained when using only EEG signals with the results obtained by adding SPR and ECG signals. In particular, the second fusion approach, using all sensor signals, achieved the highest accuracy score, reaching 82.0%. This outcome demonstrates that our proposed architecture, particularly when integrating EEG, SPR, and ECG signals at the feature level, can effectively discern the mental engagement of drivers.

Multifunctional Biosensing Platform Based on Nickel-Modified Laser-Induced Graphene

Nickel plating electrolytes prepared by using a simple salt solution can achieve nickel plating on laser-induced graphene (LIG) electrodes, which greatly enhances the electrical conductivity, electrochemical properties, wear resistance, and corrosion resistance of LIG. This makes the LIG-Ni electrodes well suited for electrophysiological, strain, and electrochemical sensing applications. The investigation of the mechanical properties of the LIG-Ni sensor and the monitoring of pulse, respiration, and swallowing confirmed that the sensor can sense insignificant deformations to relatively large conformal strains of skin. Modulation of the nickel-plating process of LIG-Ni, followed by chemical modification, may allow for the introduction of glucose redox catalyst Ni2Fe(CN)6 with interestingly strong catalytic effects, which gives LIG-Ni impressive glucose-sensing properties. Additionally, the chemical modification of LIG-Ni for pH and Na+ monitoring also confirmed its strong electrochemical monitoring potential, which demonstrates application prospects in the development of multiple electrochemical sensors for sweat parameters. A more uniform LIG-Ni multi-physiological sensor preparation process provides a prerequisite for the construction of an integrated multi-physiological sensor system. The sensor was validated to have continuous monitoring performance, and its preparation process is expected to form a system for non-invasive physiological parameter signal monitoring, thus contributing to motion monitoring, disease prevention, and disease diagnosis.

Driver Attention Assessment Using Physiological Measures from EEG, ECG, and EDA Signals

In this paper, we consider the evaluation of the mental attention state of individuals driving in a simulated environment. We tested a pool of subjects while driving on a highway and trying to overcome various obstacles placed along the course in both manual and autonomous driving scenarios. Most systems described in the literature use cameras to evaluate features such as blink rate and gaze direction. In this study, we instead analyse the subjects' Electrodermal activity (EDA) Skin Potential Response (SPR), their Electrocardiogram (ECG), and their Electroencephalogram (EEG). From these signals we extract a number of physiological measures, including eye blink rate and beta frequency band power from EEG, heart rate from ECG, and SPR features, then investigate their capability to assess the mental state and engagement level of the test subjects. In particular, and as confirmed by statistical tests, the signals reveal that in the manual scenario the subjects experienced a more challenged mental state and paid higher attention to driving tasks compared to the autonomous scenario. A different experiment in which subjects drove in three different setups, i.e., a manual driving scenario and two autonomous driving scenarios characterized by different vehicle settings, confirmed that manual driving is more mentally demanding than autonomous driving. Therefore, we can conclude that the proposed approach is an appropriate way to monitor driver attention.

Utility of the Full ECG Waveform for Stress Classification

The detection of psychological stress using the electrocardiogram (ECG) signal is most commonly based on the detection of the R peak-the most prominent part of the ECG waveform-and the heart rate variability (HRV) measurements derived from it. For stress detection algorithms focused on short-duration time windows, there is potential benefit in including HRV features derived from the detection of smaller peaks within the ECG waveform: the P, Q, S, and T waves. However, the potential drawback of using these small peaks is their smaller magnitude and subsequent susceptibility to noise, making them more difficult to reliably detect. In this work, we demonstrate the potential benefits of including smaller waves within binary stress classification using a pre-existing data set of ECG recordings from 57 participants (aged 18-40) with a self-reported fear of spiders during exposure to videos of spiders. We also present an analysis of the performance of an automated peak detection algorithm and the reliability of detection for each of the smaller parts of the ECG waveform. We compared two models, one with only R peak features and one with small peak features. They were similar in precision, recall, F1, area under ROC curve (AUC), and accuracy, with the greatest differences less than the standard deviations of each metric. There was a significant difference in the Akaike Information Criterion (AIC), which represented the information loss of the model. The inclusion of novel small peak features made the model 4.29×1028 times more probable to minimize the information loss, and the small peak features showed higher regression coefficients than the R peak features, indicating a stronger relationship with acute psychological stress. This difference and further analysis of the novel features suggest that small peak intervals could be indicative of independent processes within the heart, reflecting a psychophysiological response to stress that has not yet been leveraged in stress detection algorithms.

Towards On-Device Dehydration Monitoring Using Machine Learning from Wearable Device's Data

With the ongoing advances in sensor technology and miniaturization of electronic chips, more applications are researched and developed for wearable devices. Hydration monitoring is among the problems that have been recently researched. Athletes, battlefield soldiers, workers in extreme weather conditions, people with adipsia who have no sensation of thirst, and elderly people who lost their ability to talk are among the main target users for this application. In this paper, we address the use of machine learning for hydration monitoring using data from wearable sensors: accelerometer, magnetometer, gyroscope, galvanic skin response sensor, photoplethysmography sensor, temperature, and barometric pressure sensor. These data, together with new features constructed to reflect the activity level, were integrated with personal features to predict the last drinking time of a person and alert the user when it exceeds a certain threshold. The results of applying different models are compared for model selection for on-device deployment optimization. The extra trees model achieved the least error for predicting unseen data; random forest came next with less training time, then the deep neural network with a small model size, which is preferred for wearable devices with limited memory. Embedded on-device testing is still needed to emphasize the results and test for power consumption.

Development of an EEG Headband for Stress Measurement on Driving Simulators

In this paper, we designed from scratch, realized, and characterized a six-channel EEG wearable headband for the measurement of stress-related brain activity during driving. The headband transmits data over WiFi to a laptop, and the rechargeable battery life is 10 h of continuous transmission. The characterization manifested a measurement error of 6 μV in reading EEG channels, and the bandwidth was in the range [0.8, 44] Hz, while the resolution was 50 nV exploiting the oversampling technique. Thanks to the full metrological characterization presented in this paper, we provide important information regarding the accuracy of the sensor because, in the literature, commercial EEG sensors are used even if their accuracy is not provided in the manuals. We set up an experiment using the driving simulator available in our laboratory at the University of Udine; the experiment involved ten volunteers who had to drive in three scenarios: manual, autonomous vehicle with a “gentle” approach, and autonomous vehicle with an “aggressive” approach. The aim of the experiment was to assess how autonomous driving algorithms impact EEG brain activity. To our knowledge, this is the first study to compare different autonomous driving algorithms in terms of drivers’ acceptability by means of EEG signals. The obtained results demonstrated that the estimated power of beta waves (related to stress) is higher in the manual with respect to autonomous driving algorithms, either “gentle” or “aggressive”.

Exploring Physiological Signal Responses to Traffic-Related Stress in Simulated Driving

In this paper, we propose a relatively noninvasive system that can automatically assess the impact of traffic conditions on drivers. We analyze the physiological signals recorded from a set of individuals while driving in a simulated urban scenario in two different traffic scenarios, i.e., with traffic and without traffic. The experiments were carried out in a laboratory located at the University of Udine, employing a driving simulator equipped with a moving platform. We acquired two Skin Potential Response (SPR) signals from the hands of the drivers, and an electrocardiogram (ECG) signal from their chest. In the proposed scheme, the SPR signals are then processed through a Motion Artifact (MA) removal algorithm such that possible motion artifacts arising during the drive are reduced. An analysis considering the scalogram of the single cleaned SPR signal is presented. This signal, along with the ECG, is then fed to various Machine Learning (ML) algorithms. More specifically, some statistical features are extracted from each signal segment which, after being analyzed through a binary ML model, are labeled as corresponding to a stressful situation or not. Our results confirm the applicability of the proposed approach to identify stress in the two scenarios. This is also in accordance with our findings considering the SPR signal scalograms.

Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods.

The Relationship between Stress Levels Measured by a Questionnaire and the Data Obtained by Smart Glasses and Finger Pulse Oximeters among Polish Dental Students

Stress is a physical, mental, or emotional response to a change and is a significant problem in modern society. In addition to questionnaires, levels of stress may be assessed by monitoring physiological signals, such as via photoplethysmogram (PPG), electroencephalogram (EEG), electrocardiogram (ECG), electrodermal activity (EDA), facial expressions, and head and body movements. In our study, we attempted to find the relationship between the perceived stress level and physiological signals, such as heart rate (HR), head movements, and electrooculographic (EOG) signals. The perceived stress level was acquired by self-assessment questionnaires in which the participants marked their stress level before, during, and after performing a task. The heart rate was acquired with a finger pulse oximeter and the head movements (linear acceleration and angular velocity) and electrooculographic signals were recorded with JINS MEME ES_R smart glasses (JINS Holdings, Inc., Tokyo, Japan). We observed significant differences between the perceived stress level, heart rate, the power of linear acceleration, angular velocity, and EOG signals before performing the task and during the task. However, except for HR, these signals were poorly correlated with the perceived stress level acquired during the task.

Machine Learning for personalised stress detection: Inter-individual variability of EEG-ECG markers for acute-stress response

Stress appears as a response for a broad variety of physiological stimuli. It does vary among individuals in amplitude, phase and frequency. Thus, the necessity for personalised diagnosis is key to prevent stress-related diseases. In order to evaluate stress levels, a multi-sensing system is proposed based on non-invasive EEG and ECG signals. A target population of 24 individuals which age range between 18-23 years old are intentionally exposed to control-induced stress tests while EEG and ECG are simultaneously recorded. The acquired signals are processed by using semisupevised Machine Learning techniques as those provide a patient-specific approach due to key characteristics such as adaptiveness and robustness. In here, a stress metric is proposed that jointly with each individual medical history provide mechanisms to prevent and avoid possible chronic-health issues for individuals whom are more sensitive to stressors. Finally, supervised learning techniques are used to classify the obtained featured clusters to evaluate specific and general subject models in order to pave the way for real time stress monitoring.

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

Unlocking the Subconscious Consumer Bias: A Survey on the Past, Present, and Future of Hybrid EEG Schemes in Neuromarketing

Fueled by early success stories, the neuromarketing domain advanced rapidly during the last 10 years. As exciting new techniques were being adapted from medical research to the commercial domain, many neuroscientists and marketing practitioners have taken the chance to exploit them so as to uncover the answers of the most important marketing questions. Among the available neuroimaging technologies, electroencephalography (EEG) stands out as the less invasive and most affordable method. While not equally precise as other neuroimaging technologies in terms of spatial resolution, it can capture brain activity almost at the speed of cognition. Hence, EEG constitutes a favorable candidate for recording and subsequently decoding the consumers' brain activity. However, despite its wide use in neuromarketing, it cannot provide the complete picture alone. In order to overcome the limitations imposed by a single monitoring method, researchers focus on more holistic approaches. The exploitation of hybrid EEG schemes (e.g., combining EEG with eye-tracking, electrodermal activity, heart rate, and/or other) is ever growing and will hopefully allow neuromarketing to uncover consumers' behavior. Our survey revolves around last-decade hybrid neuromarketing schemes that involve EEG as the dominant modality. Beyond covering the relevant literature and state-of-the-art findings, we also provide future directions on the field, present the limitations that accompany each of the commonly employed monitoring methods and briefly discuss the omni-present ethical scepticizm related to neuromarketing.

A Review of Biophysiological and Biochemical Indicators of Stress for Connected and Preventive Healthcare

Stress is a known contributor to several life-threatening medical conditions and a risk factor for triggering acute cardiovascular events, as well as a root cause of several social problems. The burden of stress is increasing globally and, with that, is the interest in developing effective stress-monitoring solutions for preventive and connected health, particularly with the help of wearable sensing technologies. The recent development of miniaturized and flexible biosensors has enabled the development of connected wearable solutions to monitor stress and intervene in time to prevent the progression of stress-induced medical conditions. This paper presents a review of the literature on different physiological and chemical indicators of stress, which are commonly used for quantitative assessment of stress, and the associated sensing technologies.

Towards the Applicability of Measuring the Electrodermal Activity in the Context of Process Model Comprehension: Feasibility Study

Process model comprehension is essential in order to understand the five Ws (i.e., who, what, where, when, and why) pertaining to the processes of organizations. However, research in this context showed that a proper comprehension of process models often poses a challenge in practice. For this reason, a vast body of research exists studying the factors having an influence on process model comprehension. In order to point research towards a neuro-centric perspective in this context, the paper at hand evaluates the appropriateness of measuring the electrodermal activity (EDA) during the comprehension of process models. Therefore, a preliminary test run and a feasibility study were conducted relying on an EDA and physical activity sensor to record the EDA during process model comprehension. The insights obtained from the feasibility study demonstrated that process model comprehension leads to an increased activity in the EDA. Furthermore, EDA-related results indicated significantly that participants were confronted with a higher cognitive load during the comprehension of complex process models. In addition, the experiences and limitations we learned in measuring the EDA during the comprehension of process models are discussed in this paper. In conclusion, the feasibility study demonstrated that the measurement of the EDA could be an appropriate method to obtain new insights into process model comprehension.