Into the Wild: The Challenges of Physiological Stress Detection in Laboratory and Ambulatory Settings

Photoplethysmography as a noninvasive surrogate for microneurography in measuring stress-induced sympathetic nervous activation - A machine learning approach

The sympathetic nervous system (SNS) is essential for the body's immediate response to stress, initiating physiological changes that can be measured through sympathetic nerve activity (SNA). While microneurography (MNG) is the gold standard for direct SNA measurement, its invasive nature limits its practical use in clinical settings. This study investigates the use of multi-wavelength photoplethysmography (PPG) as a non-invasive alternative for SNA measurement. Key features are extracted from the pulsatile components of red and green PPG signals to train a linear regression machine learning (ML) model to predict R-wave-triggered spike count (SPR), a biomarker derived from MNG. The study correlates PPG-derived features with ground truth SPR to develop a predictive model capable of detecting SNA during induced physical stress (isometric handgrip and cold pressor) and cognitive stress (mental arithmetic and Stroop test). Unlike previous research that relies on subjective stress indicators, our work utilizes MNG-derived SPR as an objective ground truth for validation. Our findings demonstrate strong agreement between PPG-predicted SPR values and those obtained via MNG, with red PPG showing a higher correlation. The green wavelength PPG exhibits greater sensitivity in detecting stress-induced SNA, particularly during stress onset, where it outperforms the MNG method in capturing immediate responses to stressors such as mental arithmetic and the cold pressor task. To the best of our knowledge, this is the first study to directly compare PPG-derived SNA estimates with MNG, offering a promising pathway for developing wearable, non-invasive tools for continuous stress monitoring and sympathetic arousal detection.

Do delayed responses introduce bias in ecological momentary assessment? Evidence from comparisons between self-reported and objective physical activity

Introduction

Delayed responses are a common yet often overlooked aspect of participant compliance in ecological momentary assessment (EMA) research. This study investigated whether response delays introduce selection bias in the moments captured by EMA.

Methods

Participants (n = 339) self-reported their physical activity behaviors using EMA five times a day over 7 days while wearing a continuous physical activity monitor. The continuous activity monitor data provided an objective reference value to evaluate potential biases in delayed EMA self-reports.

Results

Results showed that participants were significantly more likely to delay EMA responses when they were prompted during higher levels of physical activity, and they subsequently reduced their activity levels, postponing their response until they were in a significantly less active state. There was no significant evidence that response delays systematically biased the levels of EMA reported activities, although delayed responses were associated with significantly more random errors in EMA reports (with small effect sizes).

Discussion

The results suggest that respondents self-select the moments for answering EMA surveys based on their current activity levels, but brief response delays do not translate into marked reductions in the quality of EMA data.

Socially interactive agents for robotic neurorehabilitation training: conceptualization and proof-of-concept study

Introduction

Individuals with diverse motor abilities often benefit from intensive and specialized rehabilitation therapies aimed at enhancing their functional recovery. Nevertheless, the challenge lies in the restricted availability of neurorehabilitation professionals, hindering the effective delivery of the necessary level of care. Robotic devices hold great potential in reducing the dependence on medical personnel during therapy but, at the same time, they generally lack the crucial human interaction and motivation that traditional in-person sessions provide.

Methods

To bridge this gap, we introduce an AI-based system aimed at delivering personalized, out-of-hospital assistance during neurorehabilitation training. This system includes a rehabilitation training device, affective signal classification models, training exercises, and a socially interactive agent as the user interface. With the assistance of a professional, the envisioned system is designed to be tailored to accommodate the unique rehabilitation requirements of an individual patient. Conceptually, after a preliminary setup and instruction phase, the patient is equipped to continue their rehabilitation regimen autonomously in the comfort of their home, facilitated by a socially interactive agent functioning as a virtual coaching assistant. Our approach involves the integration of an interactive socially-aware virtual agent into a neurorehabilitation robotic framework, with the primary objective of recreating the social aspects inherent to in-person rehabilitation sessions. We also conducted a feasibility study to test the framework with healthy patients.

Results and discussion

The results of our preliminary investigation indicate that participants demonstrated a propensity to adapt to the system. Notably, the presence of the interactive agent during the proposed exercises did not act as a source of distraction; instead, it positively impacted users' engagement.

Lessons From 3 Longitudinal Sensor-Based Human Behavior Assessment Field Studies and an Approach to Support Stakeholder Management: Content Analysis

BACKGROUND

Pervasive technologies are used to investigate various phenomena outside the laboratory setting, providing valuable insights into real-world human behavior and interaction with the environment. However, conducting longitudinal field trials in natural settings remains challenging due to factors such as low recruitment success and high dropout rates due to participation burden or data quality issues with wireless sensing in changing environments.

OBJECTIVE

This study gathers insights and lessons from 3 real-world longitudinal field studies assessing human behavior and derives factors that impacted their research success. We aim to categorize challenges, observe how they were managed, and offer recommendations for designing and conducting studies involving human participants and pervasive technology in natural settings.

METHODS

We developed a qualitative coding framework to categorize and address the unique challenges encountered in real-life studies related to influential factor identification, stakeholder management, data harvesting and management, and analysis and interpretation. We applied inductive reasoning to identify issues and related mitigation actions in 3 separate field studies carried out between 2018 and 2022. These 3 field studies relied on gathering annotated sensor data. The topics involved stress and environmental assessment in an office and a school, collecting self-reports and wrist device and environmental sensor data from 27 participants for 3.5 to 7 months; work activity recognition at a construction site, collecting observations and wearable sensor data from 15 participants for 3 months; and stress recognition in location-independent knowledge work, collecting self-reports and computer use data from 57 participants for 2 to 5 months. Our key extension for the coding framework used a stakeholder identification method to identify the type and role of the involved stakeholder groups, evaluating the nature and degree of their involvement and influence on the field trial success.

RESULTS

Our analysis identifies 17 key lessons related to planning, implementing, and managing a longitudinal, sensor-based field study on human behavior. The findings highlight the importance of recognizing different stakeholder groups, including those not directly involved but whose areas of responsibility are impacted by the study and therefore have the power to influence it. In general, customizing communication strategies to engage stakeholders on their terms and addressing their concerns and expectations is essential, while planning for dropouts, offering incentives for participants, conducting field tests to identify problems, and using tools for quality assurance are relevant for successful outcomes.

CONCLUSIONS

Our findings suggest that field trial implementation should include additional effort to clarify the expectations of stakeholders and to communicate with them throughout the process. Our framework provides a structured approach that can be adopted by other researchers in the field, facilitating robust and comparable studies across different contexts. Constantly managing the possible challenges will lead to better success in longitudinal field trials and developing future technology-based solutions.

Applying artificial intelligence on EDA sensor data to predict stress on minimally invasive robotic-assisted surgery

PURPOSE

This study aims predicting the stress level based on the ergonomic (kinematic) and physiological (electrodermal activity-EDA, blood pressure and body temperature) parameters of the surgeon from their records collected in the previously immediate situation of a minimally invasive robotic surgery activity.

METHODS

For this purpose, data related to the surgeon's ergonomic and physiological parameters were collected during twenty-six robotic-assisted surgical sessions completed by eleven surgeons with different experience levels. Once the dataset was generated, two preprocessing techniques were applied (scaled and normalized), these two datasets were divided into two subsets: with 80% of data for training and cross-validation, and 20% of data for test. Three predictive techniques (multiple linear regression-MLR, support vector machine-SVM and multilayer perceptron-MLP) were applied on training dataset to generate predictive models. Finally, these models were validated on cross-validation and test datasets. After each session, surgeons were asked to complete a survey of their feeling of stress. These data were compared with those obtained using predictive models.

RESULTS

The results showed that MLR combined with the scaled preprocessing achieved the highest R2 coefficient and the lowest error for each parameter analyzed. Additionally, the results for the surgeons' surveys were highly correlated to the results obtained by the predictive models (R2 = 0.8253).

CONCLUSIONS

The linear models proposed in this study were successfully validated on cross-validation and test datasets. This fact demonstrates the possibility of predicting factors that help us to improve the surgeon's health during robotic surgery.

Quantifying Posttraumatic Stress Disorder Symptoms During Traumatic Memories Using Interpretable Markers of Respiratory Variability

BACKGROUND

Posttraumatic stress disorder (PTSD) causes heightened fight-or-flight responses to traumatic memories (i.e., hyperarousal). Although hyperarousal is hypothesized to cause irregular breathing (i.e., respiratory variability), no quantitative markers of respiratory variability have been shown to correspond with PTSD symptoms in humans.

OBJECTIVE

In this study, we define interpretable markers of respiration pattern variability (RPV) and investigate whether these markers respond during traumatic memories, correlate with PTSD symptoms, and differ in patients with PTSD.

METHODS

We recruited 156 veterans from the Vietnam-Era Twin Registry to participate in a trauma recall protocol. From respiratory effort and electrocardiogram measurements, we extracted respiratory timings and rate using a robust quality assessment and fusion approach. We then quantified RPV using the interquartile range and compared RPV between baseline and trauma recall conditions, correlated PTSD symptoms to the difference between trauma recall and baseline RPV (i.e., ∆RPV), and compared ∆RPV between patients with PTSD and trauma-exposed controls. Leveraging a subset of 116 paired twins, we then uniquely controlled for factors shared by co-twins via within-pair analysis for further validation.

RESULTS

We found RPV was increased during traumatic memories (p .001), ∆ RPV was positively correlated with PTSD symptoms (p .05), and patients with PTSD exhibited higher ∆ RPV than trauma-exposed controls (p . 05).

CONCLUSIONS

This paper is the first to elucidate RPV markers that respond during traumatic memories, especially in patients with PTSD, and correlate with PTSD symptoms.

SIGNIFICANCE

These findings encourage future studies outside the clinic, where interpretable markers of respiratory variability are used to track hyperarousal.

Emotion regulation and heart rate variability may identify the optimal posttraumatic stress disorder treatment: analyses from a randomized controlled trial

Introduction

High variability in response and retention rates for posttraumatic stress disorder (PTSD) treatment highlights the need to identify "personalized" or "precision" medicine factors that can inform optimal intervention selection before an individual commences treatment. In secondary analyses from a non-inferiority randomized controlled trial, behavioral and physiological emotion regulation were examined as non-specific predictors (that identify which individuals are more likely to respond to treatment, regardless of treatment type) and treatment moderators (that identify which treatment works best for whom) of PTSD outcome.

Methods

There were 85 US Veterans with clinically significant PTSD symptoms randomized to 6 weeks of either cognitive processing therapy (CPT; n = 44) or a breathing-based yoga practice (Sudarshan kriya yoga; SKY; n = 41). Baseline self-reported emotion regulation (Difficulties in Emotion Regulation Scale) and heart rate variability (HRV) were assessed prior to treatment, and self-reported PTSD symptoms were assessed at baseline, end-of-treatment, 1-month follow-up, and 1-year follow-up.

Results

Greater baseline deficit in self-reported emotional awareness (similar to alexithymia) predicted better overall PTSD improvement in both the short- and long-term, following either CPT or SKY. High self-reported levels of emotional response non-acceptance were associated with better PTSD treatment response with CPT than with SKY. However, all significant HRV indices were stronger moderators than all self-reported emotion regulation scales, both in the short- and long-term. Veterans with lower baseline HRV had better PTSD treatment response with SKY, whereas Veterans with higher or average-to-high baseline HRV had better PTSD treatment response with CPT.

Conclusions

To our knowledge, this is the first study to examine both self-reported emotion regulation and HRV, within the same study, as both non-specific predictors and moderators of PTSD treatment outcome. Veterans with poorer autonomic regulation prior to treatment had better PTSD outcome with a yoga-based intervention, whereas those with better autonomic regulation did better with a trauma-focused psychological therapy. Findings show potential for the use of HRV in clinical practice to personalize PTSD treatment.

Clinical trial registration

ClinicalTrials.gov identifier, NCT02366403.

Designing daily-life research combining experience sampling method with parallel data

BACKGROUND

Ambulatory monitoring is gaining popularity in mental and somatic health care to capture an individual's wellbeing or treatment course in daily-life. Experience sampling method collects subjective time-series data of patients' experiences, behavior, and context. At the same time, digital devices allow for less intrusive collection of more objective time-series data with higher sampling frequencies and for prolonged sampling periods. We refer to these data as parallel data. Combining these two data types holds the promise to revolutionize health care. However, existing ambulatory monitoring guidelines are too specific to each data type, and lack overall directions on how to effectively combine them.

METHODS

Literature and expert opinions were integrated to formulate relevant guiding principles.

RESULTS

Experience sampling and parallel data must be approached as one holistic time series right from the start, at the study design stage. The fluctuation pattern and volatility of the different variables of interest must be well understood to ensure that these data are compatible. Data have to be collected and operationalized in a manner that the minimal common denominator is able to answer the research question with regard to temporal and disease severity resolution. Furthermore, recommendations are provided for device selection, data management, and analysis. Open science practices are also highlighted throughout. Finally, we provide a practical checklist with the delineated considerations and an open-source example demonstrating how to apply it.

CONCLUSIONS

The provided considerations aim to structure and support researchers as they undertake the new challenges presented by this exciting multidisciplinary research field.

Exploring EEG characteristics of multi-level mental stress based on human-machine system

Objective.The understanding of cognitive states is important for the development of human-machine systems (HMSs), and one of the fundamental but challenging issues is the understanding and assessment of the operator's mental stress state in real task scenarios.Approach.In this paper, a virtual unmanned vehicle (UAV) driving task with multi-challenge-level was created to explore the operator's mental stress, and the human brain activity during the task was tracked in real time via electroencephalography (EEG). A mental stress analysis dataset for the virtual UAV task was then developed and used to explore the neural activation patterns associated with mental stress activity. Finally, a multiple attention-based convolutional neural network (MACN) was constructed for automatic stress assessment using the extracted stress-sensitive neural activation features.Main Results.The statistical results of EEG power spectral density (PSD) showed that frontal theta-PSD decreased with increasing task difficulty, and central beta-PSD increased with increasing task difficulty, indicating that neural patterns showed different trends under different levels of mental stress. The performance of the proposed MACN was evaluated based on the dimensional model, and results showed that average three-class classification accuracies of 89.49%/89.88% were respectively achieved for arousal/valence.Significance.The results of this paper suggest that objective assessment of mental stress in a HMS based on a virtual UAV scenario is feasible, and the proposed method provides a promising solution for cognitive computing and applications in human-machine tasks.

Measurement of stress-induced sympathetic nervous activity using multi-wavelength PPG

The onset of stress triggers sympathetic arousal (SA), which causes detectable changes to physiological parameters such as heart rate, blood pressure, dilation of the pupils and sweat release. The objective quantification of SA has tremendous potential to prevent and manage psychological disorders. Photoplethysmography (PPG), a non-invasive method to measure skin blood flow changes, has been used to estimate SA indirectly. However, the impact of various wavelengths of the PPG signal has not been investigated for estimating SA. In this study, we explore the feasibility of using various statistical and nonlinear features derived from peak-to-peak (AC) values of PPG signals of different wavelengths (green, blue, infrared and red) to estimate stress-induced changes in SA and compare their performances. The impact of two physical stressors: and Hand Grip are studied on 32 healthy individuals. Linear (Mean, s.d.) and nonlinear (Katz, Petrosian, Higuchi, SampEn, TotalSampEn) features are extracted from the PPG signal's AC amplitudes to identify the onset, continuation and recovery phases of those stressors. The results show that the nonlinear features are the most promising in detecting stress-induced sympathetic activity. TotalSampEn feature was capable of detecting stress-induced changes in SA for all wavelengths, whereas other features (Petrosian, AvgSampEn) are significant (AUC ≥ 0.8) only for IR and Red wavelengths. The outcomes of this study can be used to make device design decisions as well as develop stress detection algorithms.

Photoplethysmography-based derivation of physiological information using the BioPoint

The BioPoint is a new wireless and wearable device, targeting both the ambulatory and on-site monitoring of biosignals. It is described as being capable of streaming and recording the i) electromyography, ii) electrocardiography, iii) electrodermal activity, iv) photoplethysmography, v) skin temperature and vi) actigraphy simultaneously, while making the raw signals recorded by the sensors readily available. However, an in-depth assessment of the biophysical signals recorded by this device, as well as its ability to derive vital signs and other health metrics, remains to be carried out. Consequently, this work proposes a preliminary study to evaluate the quality of the signals that can be acquired by this wearable with a focus on the derivation of heart rate and peripheral blood oxygenation via photoplethysmography. The device is quantitatively compared to the medical-grade pulse oximeter NoninConnect 3245, by Nonin inc. This study was performed with participants wearing the BioPoint at different positions on the body (finger, wrist, forearm, biceps and plantar arch), while the NoninConnect was worn on the fingertip and used as the ground truth. The results show that the BioPoint can accurately determine both heart rate and oxygen saturation from various locations on the body. However, as the BioPoint's photoplethysmograph is not calibrated it cannot be used for medical purposes (non-medical-grade).

Digital Interventions to Reduce Distress Among Frontline Health Care Providers: Analysis of Self-Perceived Stress

Due to the constraints of the COVID-19 pandemic, healthcare workers have reported behaving in ways that are contrary to their values, which may result in distress and injury. This work is the first of its kind to evaluate the presence of stress in the COVID-19 VR Healthcare Simulation for Distress dataset. The dataset collected passive physiological signals and active mental health questionnaires. This paper focuses on correlating electrocardiogram, respiration, photoplethysmography, and galvanic skin response with the Perceived Stress Scale (PSS)-10 questionnaire. The analysis involved data-driven techniques for a robust evaluation of stress among participants. Low-complexity pre-processing and feature extraction techniques were applied and support vector machine and decision tree models were created to predict the PSS-10 scores of users. Imbalanced data classification techniques were used to further enhance our understanding of the results. Decision tree with oversampling through Synthetic Minority Oversampling Technique achieved an accuracy, precision, recall, and F1 of 93.50%, 93.41%, 93.31%, and 93.35%, respectively. Our findings offer novel results and clinically valuable insights for stress detection and potential for translation to edge computing applications to enhance privacy, longitudinal monitoring, and simplify device requirements.

Continuous physiological signal measurement over 24-hour periods to assess the impact of work-related stress and workplace violence

Work-related stress has long been recognized as an essential factor affecting employees' health and wellbeing. Repeated exposure to acute occupational stressors puts workers at high risk for depression, obesity, hypertension, and early death. Assessment of the effects of acute stress on workers' wellbeing usually relies on subjective self-reports, questionnaires, or measuring biometric and biochemical markers in long-cycle time intervals. This study aimed to develop and validate the use of a multiparameter wearable armband for continuous non-invasive monitoring of physiological states. Two worker populations were monitored 24 h/day: six loggers for one day and six ICU nurses working 12-hr shifts for one week. Stress responses in nurses were highly correlated with changes in heart rate variability (HRV) and pulse transit time (PTT). A rise in the low-to high-frequency (LF/LH) ratio in HRV was also coincident with stress responses. HRV on workdays decreased compared to non-work days, and PTT also exhibited a persistent decrease reflecting increased blood pressure. Compared to loggers, nurses were involved in high-intensity work activities 45% more often but were less active on non-work days. The wearable technology was well accepted by all worker participants and yielded high signal quality, critical factors for long-term non-invasive occupational health monitoring.

Wearables measuring electrodermal activity to assess perceived stress in care: a scoping review

BACKGROUND

Chronic stress responses can lead to physical and behavioural health problems, often experienced and observed in the care of people with intellectual disabilities or people with dementia. Electrodermal activity (EDA) is a bio-signal for stress, which can be measured by wearables and thereby support stress management. However, the how, when and to what extent patients and healthcare providers can benefit is unclear. This study aims to create an overview of available wearables enabling the detection of perceived stress by using EDA.

METHODS

Following the PRISMA-SCR protocol for scoping reviews, four databases were included in the search of peer-reviewed studies published between 2012 and 2022, reporting detection of EDA in relation to self-reported stress or stress-related behaviours. Type of wearable, bodily location, research population, context, stressor type and the reported relationship between EDA and perceived stress were extracted.

RESULTS

Of the 74 included studies, the majority included healthy subjects in laboratory situations. Field studies and studies using machine learning (ML) to predict stress have increased in the last years. EDA is most often measured on the wrist, with offline data processing. Studies predicting perceived stress or stress-related behaviour using EDA features, reported accuracies between 42% and 100% with an average of 82.6%. Of these studies, the majority used ML.

CONCLUSION

Wearable EDA sensors are promising in detecting perceived stress. Field studies with relevant populations in a health or care context are lacking. Future studies should focus on the application of EDA-measuring wearables in real-life situations to support stress management.

The connection between heart rate variability (HRV), neurological health, and cognition: A literature review

The heart and brain have bi-directional influences on each other, including autonomic regulation and hemodynamic connections. Heart rate variability (HRV) measures variation in beat-to-beat intervals. New findings about disorganized sinus rhythm (erratic rhythm, quantified as heart rate fragmentation, HRF) are discussed and suggest overestimation of autonomic activities in HRV changes, especially during aging or cardiovascular events. When excluding HRF, HRV is regulated via the central autonomic network (CAN). HRV acts as a proxy of autonomic activity and is associated with executive functions, decision-making, and emotional regulation in our health and wellbeing. Abnormal changes of HRV (e.g., decreased vagal functioning) are observed in various neurological conditions including mild cognitive impairments, dementia, mild traumatic brain injury, migraine, COVID-19, stroke, epilepsy, and psychological conditions (e.g., anxiety, stress, and schizophrenia). Efforts are needed to improve the dynamic and intriguing heart-brain interactions.

An interpretable machine learning approach to multimodal stress detection in a simulated office environment

BACKGROUND AND OBJECTIVE

Work-related stress affects a large part of today's workforce and is known to have detrimental effects on physical and mental health. Continuous and unobtrusive stress detection may help prevent and reduce stress by providing personalised feedback and allowing for the development of just-in-time adaptive health interventions for stress management. Previous studies on stress detection in work environments have often struggled to adequately reflect real-world conditions in controlled laboratory experiments. To close this gap, in this paper, we present a machine learning methodology for stress detection based on multimodal data collected from unobtrusive sources in an experiment simulating a realistic group office environment (N=90).

METHODS

We derive mouse, keyboard and heart rate variability features to detect three levels of perceived stress, valence and arousal with support vector machines, random forests and gradient boosting models using 10-fold cross-validation. We interpret the contributions of features to the model predictions with SHapley Additive exPlanations (SHAP) value plots.

RESULTS

The gradient boosting models based on mouse and keyboard features obtained the highest average F1 scores of 0.625, 0.631 and 0.775 for the multiclass prediction of perceived stress, arousal and valence, respectively. Our results indicate that the combination of mouse and keyboard features may be better suited to detect stress in office environments than heart rate variability, despite physiological signal-based stress detection being more established in theory and research. The analysis of SHAP value plots shows that specific mouse movement and typing behaviours may characterise different levels of stress.

CONCLUSIONS

Our study fills different methodological gaps in the research on the automated detection of stress in office environments, such as approximating real-life conditions in a laboratory and combining physiological and behavioural data sources. Implications for field studies on personalised, interpretable ML-based systems for the real-time detection of stress in real office environments are also discussed.

ADABase: A Multimodal Dataset for Cognitive Load Estimation

Driver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, while continuously measuring signals from multiple modalities, based on physiological measurements such as ECG, EDA, EMG, PPG, respiration rate, skin temperature and eye tracker data, as well as behavioral measurements such as action units extracted from facial videos, performance metrics like reaction time and subjective feedback using questionnaires, we create ADABase (Autonomous Driving Cognitive Load Assessment Database) As a reference method to induce cognitive load onto subjects, we use the well-established n-back test, in addition to our novel simulator-based k-drive test, motivated by real-world semi-autonomously vehicles. We extract expert features of all measurements and find significant changes in multiple modalities. Ultimately we train and evaluate machine learning algorithms using single and multimodal inputs to distinguish cognitive load levels. We carefully evaluate model behavior and study feature importance. In summary, we introduce a novel cognitive load test, create a cognitive load database, validate changes using statistical tests, introduce novel classification and regression tasks for machine learning and train and evaluate machine learning models.

Ubiquitous computing in light of human phenotypes: foundations, challenges, and opportunities

The interest in human phenotypes has leveraged interdisciplinary efforts encouraging a better understanding of the broad spectrum of psychological and behavioral disorders. Moreover, the usage of mobile and wearable devices along with unobtrusive computational capabilities provides an extensive amount of information that allows the characterization of phenotypes. This article describes the human phenotype through the lens of computational range and reviews state-of-the-art computational phenotyping. Furthermore, the article discusses computational phenotyping's extension concerning the combination of intelligent environments and personal mobile devices, addressing technical, managerial, and ethical challenges. This combination reinforces ubiquitous computational capabilities for phenotyping as a facilitator of interdisciplinary information convergence in favor of clinical and biomedical research.

Utility of the Full ECG Waveform for Stress Classification

The detection of psychological stress using the electrocardiogram (ECG) signal is most commonly based on the detection of the R peak-the most prominent part of the ECG waveform-and the heart rate variability (HRV) measurements derived from it. For stress detection algorithms focused on short-duration time windows, there is potential benefit in including HRV features derived from the detection of smaller peaks within the ECG waveform: the P, Q, S, and T waves. However, the potential drawback of using these small peaks is their smaller magnitude and subsequent susceptibility to noise, making them more difficult to reliably detect. In this work, we demonstrate the potential benefits of including smaller waves within binary stress classification using a pre-existing data set of ECG recordings from 57 participants (aged 18-40) with a self-reported fear of spiders during exposure to videos of spiders. We also present an analysis of the performance of an automated peak detection algorithm and the reliability of detection for each of the smaller parts of the ECG waveform. We compared two models, one with only R peak features and one with small peak features. They were similar in precision, recall, F1, area under ROC curve (AUC), and accuracy, with the greatest differences less than the standard deviations of each metric. There was a significant difference in the Akaike Information Criterion (AIC), which represented the information loss of the model. The inclusion of novel small peak features made the model 4.29×1028 times more probable to minimize the information loss, and the small peak features showed higher regression coefficients than the R peak features, indicating a stronger relationship with acute psychological stress. This difference and further analysis of the novel features suggest that small peak intervals could be indicative of independent processes within the heart, reflecting a psychophysiological response to stress that has not yet been leveraged in stress detection algorithms.

Posttraumatic stress disorder hyperarousal event detection using smartwatch physiological and activity data

Posttraumatic Stress Disorder (PTSD) is a psychiatric condition affecting nearly a quarter of the United States war veterans who return from war zones. Treatment for PTSD typically consists of a combination of in-session therapy and medication. However; patients often experience their most severe PTSD symptoms outside of therapy sessions. Mobile health applications may address this gap, but their effectiveness is limited by the current gap in continuous monitoring and detection capabilities enabling timely intervention. The goal of this article is to develop a novel method to detect hyperarousal events using physiological and activity-based machine learning algorithms. Physiological data including heart rate and body acceleration as well as self-reported hyperarousal events were collected using a tool developed for commercial off-the-shelf wearable devices from 99 United States veterans diagnosed with PTSD over several days. The data were used to develop four machine learning algorithms: Random Forest, Support Vector Machine, Logistic Regression and XGBoost. The XGBoost model had the best performance in detecting onset of PTSD symptoms with over 83% accuracy and an AUC of 0.70. Post-hoc SHapley Additive exPlanations (SHAP) additive explanation analysis showed that algorithm predictions were correlated with average heart rate, minimum heart rate and average body acceleration. Findings show promise in detecting onset of PTSD symptoms which could be the basis for developing remote and continuous monitoring systems for PTSD. Such systems may address a vital gap in just-in-time interventions for PTSD self-management outside of scheduled clinical appointments.

Quantifying Occupational Stress in Intensive Care Unit Nurses: An Applied Naturalistic Study of Correlations Among Stress, Heart Rate, Electrodermal Activity, and Skin Temperature

OBJECTIVE

To identify physiological correlates to stress in intensive care unit nurses.

BACKGROUND

Most research on stress correlates are done in laboratory environments; naturalistic investigation of stress remains a general gap.

METHOD

Electrodermal activity, heart rate, and skin temperatures were recorded continuously for 12-hr nursing shifts (23 participants) using a wrist-worn wearable technology (Empatica E4).

RESULTS

Positive correlations included stress and heart rate (ρ = .35, p < .001), stress and skin temperature (ρ = .49, p < .05), and heart rate and skin temperatures (ρ = .54, p = .0008).

DISCUSSION

The presence and direction of some correlations found in this study differ from those anticipated from prior literature, illustrating the importance of complementing laboratory research with naturalistic studies. Further work is warranted to recognize nursing activities associated with a high level of stress and the underlying reasons associated with changes in physiological responses.

APPLICATION

Heart rate and skin temperature may be used for real-time detection of stress, but more work is needed to validate such surrogate measures.

Effectiveness of the heartbeat interval error and compensation method on heart rate variability analysis

The purpose of this study is to evaluate the effectiveness of heartbeat error and compensation methods on heart rate variability (HRV) with mobile and wearable sensor devices. The HRV analysis extracts multiple indices related to the heart and autonomic nervous system from beat-to-beat intervals. These HRV analysis indices are affected by the heartbeat interval mismatch, which is caused by sampling error from measurement hardware and inherent errors from the state of human body. Although the sampling rate reduction is a common method to reduce power consumption on wearable devices, it degrades the accuracy of the heartbeat interval. Furthermore, wearable devices often use photoplethysmography (PPG) instead of electrocardiogram (ECG) to measure heart rate. However, there are inherent errors between PPG and ECG, because the PPG is affected by blood pressure fluctuations, vascular stiffness, and body movements. This paper evaluates the impact of these errors on HRV analysis using dataset including both ECG and PPG from 28 subjects. The evaluation results showed that the error compensation method improved the accuracy of HRV analysis in time domain, frequency domain and non-linear analysis. Furthermore, the error compensation by the algorithm was found to be effective for both PPG and ECG.

ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:535-541. [PMID: 34891350 DOI: 10.1109/embc46164.2021.9630040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Continuous and multimodal stress detection has been performed recently through wearable devices and machine learning algorithms. However, a well-known and important challenge of working on physiological signals recorded by conventional monitoring devices is missing data due to sensors insufficient contact and interference by other equipment. This challenge becomes more problematic when the user/patient is mentally or physically active or stressed because of more frequent conscious or subconscious movements. In this paper, we propose ReLearn, a robust machine learning framework for stress detection from biomarkers extracted from multimodal physiological signals. ReLearn effectively copes with missing data and outliers both at training and inference phases. ReLearn, composed of machine learning models for feature selection, outlier detection, data imputation, and classification, allows us to classify all samples, including those with missing values at inference. In particular, according to our experiments and stress database, while by discarding all missing data, as a simplistic yet common approach, no prediction can be made for 34% of the data at inference, our approach can achieve accurate predictions, as high as 78%, for missing samples. Also, our experiments show that the proposed framework obtains a cross-validation accuracy of 86.8% even if more than 50% of samples within the features are missing.

Relevance of Objective Measures in Psychiatric Disorders-Rest-Activity Rhythm and Psychophysiological Measures

PURPOSE OF REVIEW

We present a review of recent methods of objective measurement in psychiatry and psychology with a focus on home monitoring and its utility in guiding treatment.

RECENT FINDINGS

For individualized diagnostics and treatment of insomnia, actigraphy can generate clinically useful graphical presentations of sleep timing and patterns. Psychophysiological measures may complement psychometrics by tracking parallel changes in physiological responses and emotional functioning, especially during therapy for trauma symptoms and emotion regulation. It seems that rather than defining universal cut-offs, an individualised range of variability could characterize treatment response. Wearable actigraphy and psychophysiological sensors are promising devices to provide biofeedback and guide treatment. Use of feasible and reliable technology during experimental and clinical procedures may necessitate defining healthy and abnormal responses in different populations and pathological states. We present a "call for action" towards further collaborative work to enable large scale use of objective measures.

Comparing stress prediction models using smartwatch physiological signals and participant self-reports

Recent advances in wearable technology have facilitated the non-obtrusive monitoring of physiological signals, creating opportunities to monitor and predict stress. Researchers have utilized machine learning methods using these physiological signals to develop stress prediction models. Many of these prediction models have utilized objective stressor tasks (e.g., a public speaking task or solving math problems). Alternatively, the subjective user responses with self-reports have also been used for measuring stress. In this paper, we describe a methodological approach (a) to compare the prediction performance of models developed using objective markers of stress using participant-reported subjective markers of stress from self-reports; and (b) to develop personalized stress models by accounting for inter-individual differences. Towards this end, we conducted a laboratory-based study with 32 healthy volunteers. Participants completed a series of stressor tasks-social, cognitive and physical-wearing an instrumented commercial smartwatch that collected physiological signals and participant responses using timed self-reports. After extensive data preprocessing using a combination of signal processing techniques, we developed two types of models: objective stress models using the stressor tasks as labels; and subjective stress models using participant responses to each task as the label for that stress task. We trained and tested several machine learning algorithms-support vector machine (SVM), random forest (RF), gradient boosted trees (GBT), AdaBoost, and Logistic Regression (LR)-and evaluated their performance. SVM had the best performance for the models using the objective stressor (i.e., stressor tasks) with an AUROC of 0.790 and an F-1 score of 0.623. SVM also had the highest performance for the models using the subjective stress (i.e., participant self-reports) with an AUROC of 0.719 and an F-1 score of 0.520. Model performance improved with a personalized threshold model to an AUROC of 0.751 and an F-1 score of 0.599. The performance of the stress models using an instrumented commercial smartwatch was comparable to similar models from other state-of-the-art laboratory-based studies. However, the subjective stress models had a lower performance, indicating the need for further research on the use of self-reports for stress-related studies. The improvement in performance with the personalized threshold-based models provide new directions for building stress prediction models.

Associations Between Physiological Signals Captured Using Wearable Sensors and Self-reported Outcomes Among Adults in Alcohol Use Disorder Recovery: Development and Usability Study

Background

Previous research has highlighted the role of stress in substance misuse and addiction, particularly for relapse risk. Mobile health interventions that incorporate real-time monitoring of physiological markers of stress offer promise for delivering tailored interventions to individuals during high-risk states of heightened stress to prevent alcohol relapse. Before such interventions can be developed, measurements of these processes in ambulatory, real-world settings are needed.

Objective

This research is a proof-of-concept study to establish the feasibility of using a wearable sensor device to continuously monitor stress in an ambulatory setting. Toward that end, we first aimed to examine the quality of 2 continuously monitored physiological signals—electrodermal activity (EDA) and heart rate variability (HRV)—and show that the data follow standard quality measures according to the literature. Next, we examined the associations between the statistical features extracted from the EDA and HRV signals and self-reported outcomes.

Methods

Participants (N=11; female: n=10) were asked to wear an Empatica E4 wearable sensor for continuous unobtrusive physiological signal collection for up to 14 days. During the same time frame, participants responded to a daily diary study using ecological momentary assessment of self-reported stress, emotions, alcohol-related cravings, pain, and discomfort via a web-based survey, which was conducted 4 times daily. Participants also participated in structured interviews throughout the study to assess daily alcohol use and to validate self-reported and physiological stress markers. In the analysis, we first used existing artifact detection methods and physiological signal processing approaches to assess the quality of the physiological data. Next, we examined the descriptive statistics for self-reported outcomes. Finally, we investigated the associations between the features of physiological signals and self-reported outcomes.

Results

We determined that 87.86% (1,032,265/1,174,898) of the EDA signals were clean. A comparison of the frequency of skin conductance responses per minute with previous research confirmed that the physiological signals collected in the ambulatory setting were successful. The results also indicated that the statistical features of the EDA and HRV measures were significantly correlated with the self-reported outcomes, including the number of stressful events marked on the sensor device, positive and negative emotions, and experienced pain and discomfort.

Conclusions

The results demonstrated that the physiological data collected via an Empatica E4 wearable sensor device were consistent with previous literature in terms of the quality of the data and that features of these physiological signals were significantly associated with several self-reported outcomes among a sample of adults diagnosed with alcohol use disorder. These results suggest that ambulatory assessment of stress is feasible and can be used to develop tailored mobile health interventions to enhance sustained recovery from alcohol use disorder.

Ambient Intelligence Based on IoT for Assisting People with Alzheimer’s Disease Through Context Histories

New Internet of Things (IoT) applications are enabling the development of projects that help with monitoring people with different diseases in their daily lives. Alzheimer’s is a disease that affects neurological functions and needs support to maintain maximum independence and security of patients during this stage of life, as the cure and reversal of symptoms have not yet been discovered. The IoT-based monitoring system provides the caregivers’ support in monitoring people with Alzheimer’s disease (AD). This paper presents an ontology-based computational model that receives physiological data from external IoT applications, allowing identification of potentially dangerous behaviors for patients with AD. The main scientific contribution of this work is the specification of a model focusing on Alzheimer’s disease using the analysis of context histories and context prediction, which, considering the state of the art, is the only one that uses analysis of context histories to perform predictions. In this research, we also propose a simulator to generate activities of the daily life of patients, allowing the creation of data sets. These data sets were used to evaluate the contributions of the model and were generated according to the standardization of the ontology. The simulator generated 1026 scenarios applied to guide the predictions, which achieved average accurary of 97.44%. The experiments also allowed the learning of 20 relevant lessons on technological, medical, and methodological aspects that are recorded in this article.

Emotional valence sensing using a wearable facial EMG device

Emotion sensing using physiological signals in real-life situations can be practically valuable. Previous studies have developed wearable devices that record autonomic nervous system activity, which reflects emotional arousal. However, no study determined whether emotional valence can be assessed using wearable devices. To this end, we developed a wearable device to record facial electromyography (EMG) from the corrugator supercilii (CS) and zygomatic major (ZM) muscles. To validate the device, in Experiment 1, we used a traditional wired device and our wearable device, to record participants’ facial EMG while they were viewing emotional films. Participants viewed the films again and continuously rated their recalled subjective valence during the first viewing. The facial EMG signals recorded using both wired and wearable devices showed that CS and ZM activities were, respectively, negatively and positively correlated with continuous valence ratings. In Experiment 2, we used the wearable device to record participants’ facial EMG while they were playing Wii Bowling games and assessed their cued-recall continuous valence ratings. CS and ZM activities were correlated negatively and positively, respectively, with continuous valence ratings. These data suggest the possibility that facial EMG signals recorded by a wearable device can be used to assess subjective emotional valence in future naturalistic studies.

Ultra-Short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors

Human cognitive capabilities are under constant pressure in the modern information society. Cognitive load detection would be beneficial in several applications of human–computer interaction, including attention management and user interface adaptation. However, current research into accurate and real-time biosignal-based cognitive load detection lacks understanding of the optimal and minimal window length in data segmentation which would allow for more timely, continuous state detection. This study presents a comparative analysis of ultra-short (30 s or less) window lengths in cognitive load detection with a wearable device. Heart rate, heart rate variability, galvanic skin response, and skin temperature features are extracted at six different window lengths and used to train an Extreme Gradient Boosting classifier to detect between cognitive load and rest. A 25 s window showed the highest accury (67.6%), which is similar to earlier studies using the same dataset. Overall, model accuracy tended to decrease as the window length decreased, and lowest performance (60.0%) was observed with a 5 s window. The contribution of different physiological features to the classification performance and the most useful features that react in short windows are also discussed. The analysis provides a promising basis for future real-time applications with wearable sensors.

PASS: A Multimodal Database of Physical Activity and Stress for Mobile Passive Body/ Brain-Computer Interface Research

With the burgeoning of wearable devices and passive body/brain-computer interfaces (B/BCIs), automated stress monitoring in everyday settings has gained significant attention recently, with applications ranging from serious games to clinical monitoring. With mobile users, however, challenges arise due to other overlapping (and potentially confounding) physiological responses (e.g., due to physical activity) that may mask the effects of stress, as well as movement artifacts that can be introduced in the measured signals. For example, the classical increase in heart rate can no longer be attributed solely to stress and could be caused by the activity itself. This makes the development of mobile passive B/BCIs challenging. In this paper, we introduce PASS, a multimodal database of Physical Activity and StresS collected from 48 participants. Participants performed tasks of varying stress levels at three different activity levels and provided quantitative ratings of their perceived stress and fatigue levels. To manipulate stress, two video games (i.e., a calm exploration game and a survival game) were used. Peripheral physical activity (electrocardiography, electrodermal activity, breathing, skin temperature) as well as cerebral activity (electroencephalography) were measured throughout the experiment. A complete description of the experimental protocol is provided and preliminary analyses are performed to investigate the physiological reactions to stress in the presence of physical activity. The PASS database, including raw data and subjective ratings has been made available to the research community at http://musaelab.ca/pass-database/. It is hoped that this database will help advance mobile passive B/BCIs for use in everyday settings.

Increases of negative affect following daily hassles are not moderated by neuroticism: An ecological momentary assessment study

The occurrence of daily hassles is associated with increased subsequent levels of negative affect. Neuroticism has been found to exacerbate this effect. So far, most research used single-item measures for the assessment of daily hassles or relied on daily diary studies. This study aimed to examine the interrelations of daily hassles, negative affect reactivity, and neuroticism in daily life employing an extensive inventory of daily hassles. Seventy participants (18-30 years; M = 23.9 years, 59% female) completed a 4-week smartphone-based ecological momentary assessment study reporting the occurrence and perceived strain of daily hassles as well as negative affect at five semi-random signals between 9 a.m. and 8 p.m. Multilevel analyses revealed significant associations between elevated levels of negative affect and higher cumulative daily hassle strain ratings per signal in concurrent and time-lagged analyses. Contrary to our expectations, there was no moderation by neuroticism on these associations. The results suggest that daily hassles can accumulate in their impact on mood in daily life and exert a prolonged effect on negative affect. The absence of a significant moderation by neuroticism may be interpreted in the light of methodological specifics of this study.

Personalized mental stress detection with self-organizing map: From laboratory to the field

Stress has become a major health concern and there is a need to study and develop new digital means for real-time stress detection. Currently, the majority of stress detection research is using population based approaches that lack the capability to adapt to individual differences. They also use supervised learning methods, requiring extensive labeling of training data, and they are typically tested on data collected in a laboratory and thus do not generalize to field conditions. To address these issues, we present multiple personalized models based on an unsupervised algorithm, the Self-Organizing Map (SOM), and we propose an algorithmic pipeline to apply the method for both laboratory and field data. The performance is evaluated on a dataset of physiological measurements from a laboratory test and on a field dataset consisting of four weeks of physiological and smartphone usage data. In these tests, the performance on the field data was steady across the different personalization levels (accuracy around 60%) and a fully personalized model performed the best on the laboratory data, achieving accuracy of 92% which is comparable to state-of-the-art supervised classifiers. These results demonstrate the feasibility of SOM in personalized mental stress detection both in constrained and free-living environment.

Changes in Continuous, Long-Term Heart Rate Variability and Individualized Physiological Responses to Wellness and Vacation Interventions Using a Wearable Sensor

There are many approaches to maintaining wellness, including taking a simple vacation to attending highly structured wellness retreats, which typically regulate the attendee's personal time and activities. In a healthy English-speaking cohort of 112 women and men (aged 30–80 years), this study examined the effects of participating in either a 6-days intensive wellness retreat based on Ayurvedic medicine principles or unstructured 6-days vacation at the same wellness center setting. Heart rate variability (HRV) was monitored continuously using a wearable ECG sensor patch for up to 7 days prior to, during, and 1-month following participation in the interventions. Additionally, salivary cortisol levels were assessed for all participants at multiple times during the day. Continual HRV monitoring data in the real-world setting was seen to be associated with demographic [HRV_ALF: β_Age = 0.98 (95% CI = 0.96–0.98), false discovery rate (FDR) < 0.001] and physiological characteristics [HRV_PLF: β = 0.98 (95% CI = 0.98–1), FDR =0.005] of participants. HRV features were also able to quantify known diurnal variations [HRV_LF/HF: β_{ACT:night vs. early−morning} = 2.69 (SE = 1.26), FDR < 0.001] along with notable inter- and intraperson heterogeneity in response to intervention. A statistically significant increase in HRV_ALF [β = 1.48 (SE = 1.1), FDR < 0.001] was observed for all participants during the resort visit. Personalized HRV analysis at an individual level showed a distinct individualized response to intervention, further supporting the utility of using continuous real-world tracking of HRV at an individual level to objectively measure responses to potentially stressful or relaxing settings.

Using ambulatory assessment to measure dynamic risk processes in affective disorders

BACKGROUND

Rapid advances in the capability and affordability of digital technology have begun to allow for the intensive monitoring of psychological and physiological processes associated with affective disorders in daily life. This technology may enable researchers to overcome some limitations of traditional methods for studying risk in affective disorders, which often (implicitly) assume that risk factors are distal and static - that they do not change over time. In contrast, ambulatory assessment (AA) is particularly suited to measure dynamic "real-world" processes and to detect fluctuations in proximal risk for outcomes of interest.

METHOD

We highlight key questions about proximal and distal risk for affective disorders that AA methods (with multilevel modeling, or fully-idiographic methods) allow researchers to evaluate.

RESULTS

Key questions include between-subject questions to understand who is at risk (e.g., are people with more affective instability at greater risk than others?) and within-subject questions to understand when risk is most acute among those who are at risk (e.g., does suicidal ideation increase when people show more sympathetic activation than usual?). We discuss practical study design and analytic strategy considerations for evaluating questions of risk in context, and the benefits and limitations of self-reported vs. passively-collected AA.

LIMITATIONS

Measurements may only be as accurate as the observation period is representative of individuals' usual life contexts. Active measurement techniques are limited by the ability and willingness to self-report.

CONCLUSIONS

We conclude by discussing how monitoring proximal risk with AA may be leveraged for translation into personalized, real-time interventions to reduce risk.

Ambulatory and Laboratory Stress Detection Based on Raw Electrocardiogram Signals Using a Convolutional Neural Network

The goals of this study are the suggestion of a better classification method for detecting stressed states based on raw electrocardiogram (ECG) data and a method for training a deep neural network (DNN) with a smaller data set. We suggest an end-to-end architecture to detect stress using raw ECGs. The architecture consists of successive stages that contain convolutional layers. In this study, two kinds of data sets are used to train and validate the model: A driving data set and a mental arithmetic data set, which smaller than the driving data set. We apply a transfer learning method to train a model with a small data set. The proposed model shows better performance, based on receiver operating curves, than conventional methods. Compared with other DNN methods using raw ECGs, the proposed model improves the accuracy from 87.39% to 90.19%. The transfer learning method improves accuracy by 12.01% and 10.06% when 10 s and 60 s of ECG signals, respectively, are used in the model. In conclusion, our model outperforms previous models using raw ECGs from a small data set and, so, we believe that our model can significantly contribute to mobile healthcare for stress management in daily life.

Bioimpedance Method for Human Body Hydration Assessment

A high-precision wearable bioimpedance sensor developed at Imec was extensively tested. Unlike known bioimpedance sensors on the market, the new device enables hydration shift measurement in a single person, with no need for averaging over a population. For reaching this target, a method for hydration monitoring in case of altered hydration is tested. An assessment of fluid shift with sensitivity of about 700 ml has been demonstrated, which is comparable with the capabilities of known methods because of the device accuracy, immunity to electrode-skin impedance variation, and due to establishing the impedance baseline prior to fluid shift.