Remote measurement of cognitive stress via heart rate variability

The salivary cortisol classification based on the heart rate variability

OBJECTIVES

Stress is a physiological state that is essential for the survival of living organisms. Heart rate variability (HRV) and cortisol hormone are indicators of the stress system. According to research, it has been demonstrated that the activation of the stress system is not consciously controlled by the individual, but rather occurs subconsciously. It is a novel concept to employ HRV indexes to assess the level of cortisol concentration as a more reliable indicator of stress system activation, as opposed to relying solely on the individual's emotional state.

METHODS

In order to understand the relationship between stress and cortisol secretion and its effect on electrophysiological biomarkers like HRV, the algorithms were designed using machine learning algorithms such as SVM, XGB, and MLP in the 634 adult healthy men (20-50 years old). Trait social stress test was utilized to make wide range of cortisol concentration from no to moderate stress.

RESULTS

These algorithms classified cortisol level between 9:00 AM and 2:00 PM in the optimal (5-15 ng/mL), non (less than 5 and more than 15 ng/mL) range, using HRV indexes (12 features). The XBG algorithm could achieve best classification with an accuracy rate of 99 % and an F1 rate of 99 %. They also indicated the state of an individual's stress system by indicating the concentration level of cortisol, which is its fundamental indicator.

CONCLUSIONS

In addition to classifying stress, the HRV can also classify salivary cortisol in adult health men.

Identifying physiological indicators of the cognitive, thermal, and combined (cognitive-thermal) stress conditions

Physiologically based stress detection systems have proven to be effective in identifying different stress conditions in the body to determine the source of stress and be able to counteract it. However, some stress conditions have not been widely studied, including thermal stress, cognitive stress, and combined (thermal-cognitive) stress conditions, which are frequently encountered in work or school environments. In order to develop systems to detect and differentiate these conditions, it is necessary to identify the physiological indicators that characterize each of them. The present research aims to identify which physiological indicators (heart rate, respiratory rate, galvanic skin response, and local temperature) could differentiate different stress conditions (no-stress, cognitive stress, thermal stress, and combined (thermal-cognitive) stress conditions). Thirty participants were exposed to cognitive, thermal, and combined stress sources while recording their physiological signals. The findings indicate that both mean heart rate and mean galvanic skin response identify moderate thermal and cognitive stress conditions as distinct from a no-stress condition, yet they do not differentiate between the two stress conditions. Additionally, heart rate uniquely identifies the cognitive-thermal stress condition, effectively distinguishing this combined stress condition from the singular stress conditions and the no-stress condition. Mean local temperature specifically signals thermal stress conditions, whereas mean respiratory rate accurately identifies cognitive stress conditions, with both indicators effectively separating these conditions from each other and from the no-stress condition. This is the first basis for differentiating thermal and cognitive stress conditions through physiological indicators.

ST-Phys: Unsupervised Spatio-Temporal Contrastive Remote Physiological Measurement

Remote photoplethysmography (rPPG) is a non-contact method that employs facial videos for measuring physiological parameters. Existing rPPG methods have achieved remarkable performance. However, the success mainly profits from supervised learning over massive labeled data. On the other hand, existing unsupervised rPPG methods fail to fully utilize spatio-temporal features and encounter challenges in low-light or noise environments. To address these problems, we propose an unsupervised contrast learning approach, ST-Phys. We incorporate a low-light enhancement module, a temporal dilated module, and a spatial enhanced module to better deal with long-term dependencies under the random low-light conditions. In addition, we design a circular margin loss, wherein rPPG signals originating from identical videos are attracted, while those from distinct videos are repelled. Our method is assessed on six openly accessible datasets, including RGB and NIR videos. Extensive experiments reveal the superior performance of our proposed ST-Phys over state-of-the-art unsupervised rPPG methods. Moreover, it offers advantages in parameter reduction and noise robustness.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments

BACKGROUND

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings.

METHODS

Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken.

RESULTS

Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05).

CONCLUSION

This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

Py-Feat: Python Facial Expression Analysis Toolbox

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state-of-the-art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00191-4.

Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring

Despite the notable recent developments in the field of remote photoplethysmography (rPPG), extracting a reliable pulse rate variability (PRV) signal still remains a challenge. In this study, eight image-based photoplethysmography (iPPG) extraction methods (GRD, AGRD, PCA, ICA, LE, SPE, CHROM, and POS) were compared in terms of pulse rate (PR) and PRV features. The algorithms were made robust for motion and illumination artifacts by using ad hoc pre- and postprocessing steps. Then, they were systematically tested on the public dataset UBFC-RPPG, containing data from 42 subjects sitting in front of a webcam (30 fps) while playing a time-sensitive mathematical game. The performances of the algorithms were evaluated by statistically comparing iPPG-based and finger-PPG-based PR and PRV features in terms of Spearman's correlation coefficient, normalized root mean square error (NRMSE), and Bland-Altman analysis. The study revealed POS and CHROM techniques to be the most robust for PR estimation and the assessment of overall autonomic nervous system (ANS) dynamics by using PRV features in time and frequency domains. Furthermore, we demonstrated that a reliable characterization of the vagal tone is made possible by computing the Poincaré map of PRV series derived from the POS and CHROM methods. This study supports the use of iPPG systems as promising tools to obtain clinically useful and specific information about ANS dynamics.

Stress Estimation Using Biometric and Activity Indicators to Improve QoL of the Elderly

It is essential to estimate the stress state of the elderly to improve their QoL. Stress states change every day and hour, depending on the activities performed and the duration/intensity. However, most existing studies estimate stress states using only biometric information or specific activities (e.g., sleep duration, exercise duration/amount, etc.) as explanatory variables and do not consider all daily living activities. It is necessary to link various daily living activities and biometric information in order to estimate the stress state more accurately. Specifically, we construct a stress estimation model using machine learning with the answers to a stress status questionnaire obtained every morning and evening as the ground truth and the biometric data during each of the performed activities and the new proposed indicator including biological and activity perspectives as the features. We used the following methods: Baseline Method 1, in which the RRI variance and Lorenz plot area for 4 h after waking and 24 h before the questionnaire were used as features; Baseline Method 2, in which sleep time was added as a feature to Baseline Method 1; the proposed method, in which the Lorenz plot area per activity and total time per activity were added. We compared the results with the proposed method, which added the new indicators as the features. The results of the evaluation experiments using the one-month data collected from five elderly households showed that the proposed method had an average estimation accuracy of 59%, 7% better than Baseline Method 1 (52%) and 4% better than Baseline Method 2 (55%).

Machine Learning for Anxiety Detection Using Biosignals: A Review

Anxiety disorder (AD) is a major mental health illness. However, due to the many symptoms and confounding factors associated with AD, it is difficult to diagnose, and patients remain untreated for a long time. Therefore, researchers have become increasingly interested in non-invasive biosignals, such as electroencephalography (EEG), electrocardiogram (ECG), electrodermal response (EDA), and respiration (RSP). Applying machine learning to these signals enables clinicians to recognize patterns of anxiety and differentiate a sick patient from a healthy one. Further, models with multiple and diverse biosignals have been developed to improve accuracy and convenience. This paper reviews and summarizes studies published from 2012 to 2022 that applied different machine learning algorithms with various biosignals. In doing so, it offers perspectives on the strengths and weaknesses of current developments to guide future advancements in anxiety detection. Specifically, this literature review reveals promising measurement accuracies ranging from 55% to 98% for studies with sample sizes of 10 to 102 participants. On average, studies using only EEG seemed to obtain the best performance, but the most accurate results were obtained with EDA, RSP, and heart rate. Random forest and support vector machines were found to be widely used machine learning methods, and they lead to good results as long as feature selection has been performed. Neural networks are also extensively used and provide good accuracy, with the benefit that no feature selection is needed. This review also comments on the effective combinations of modalities and the success of different models for detecting anxiety.

Automated Remote Pulse Oximetry System (ARPOS)

Current methods of measuring heart rate (HR) and oxygen levels (SPO2) require physical contact, are individualised, and for accurate oxygen levels may also require a blood test. No-touch or non-invasive technologies are not currently commercially available for use in healthcare settings. To date, there has been no assessment of a system that measures HR and SPO2 using commercial off-the-shelf camera technology that utilises R, G, B, and IR data. Moreover, no formal remote photoplethysmography studies have been performed in real-life scenarios with participants at home with different demographic characteristics. This novel study addresses all these objectives by developing, optimising, and evaluating a system that measures the HR and SPO2 of 40 participants. HR and SPO2 are determined by measuring the frequencies from different wavelength band regions using FFT and radiometric measurements after pre-processing face regions of interest (forehead, lips, and cheeks) from colour, IR, and depth data. Detrending, interpolating, hamming, and normalising the signal with FastICA produced the lowest RMSE of 7.8 for HR with the r-correlation value of 0.85 and RMSE 2.3 for SPO2. This novel system could be used in several critical care settings, including in care homes and in hospitals and prompt clinical intervention as required.

Posttraumatic stress disorder hyperarousal event detection using smartwatch physiological and activity data

Posttraumatic Stress Disorder (PTSD) is a psychiatric condition affecting nearly a quarter of the United States war veterans who return from war zones. Treatment for PTSD typically consists of a combination of in-session therapy and medication. However; patients often experience their most severe PTSD symptoms outside of therapy sessions. Mobile health applications may address this gap, but their effectiveness is limited by the current gap in continuous monitoring and detection capabilities enabling timely intervention. The goal of this article is to develop a novel method to detect hyperarousal events using physiological and activity-based machine learning algorithms. Physiological data including heart rate and body acceleration as well as self-reported hyperarousal events were collected using a tool developed for commercial off-the-shelf wearable devices from 99 United States veterans diagnosed with PTSD over several days. The data were used to develop four machine learning algorithms: Random Forest, Support Vector Machine, Logistic Regression and XGBoost. The XGBoost model had the best performance in detecting onset of PTSD symptoms with over 83% accuracy and an AUC of 0.70. Post-hoc SHapley Additive exPlanations (SHAP) additive explanation analysis showed that algorithm predictions were correlated with average heart rate, minimum heart rate and average body acceleration. Findings show promise in detecting onset of PTSD symptoms which could be the basis for developing remote and continuous monitoring systems for PTSD. Such systems may address a vital gap in just-in-time interventions for PTSD self-management outside of scheduled clinical appointments.

An Empirical Approach to Analyzing the Effects of Stress on Individual Creativity in Business Problem-Solving: Emphasis on the Electrocardiogram, Electroencephalogram Methodology

In this study, experiments were conducted on 30 subjects by means of electrocardiogram (ECG) and electroencephalogram (EEG) methodologies as well as a money game to examine the effects of stress on creativity in business problem-solving. The study explained the relationship between creativity and human physiological response using the biopsychosocial model of challenge and threat. The subjects were asked to perform a cognitive mapping task. Based on the brain wave theory, we identified the types of brain waves and locations of brain activities that occurred during the creative problem-solving process in a business environment and studied the effects of stress on creativity. The results of the experiments showed significant differences in creativity in business problem-solving depending on whether or not stress was triggered. Differences were found in the time domain (SDNN, RMSSD) and frequency domain (HF, LF/HF ratio) of heart rates, a physiological stress indicator, between the stress group and the no-stress group. A brain wave analysis confirmed that alpha waves increased in the frontal lobe of the brain during creative business problem-solving but decreased when the subjects were under stress, during which beta waves in the brain increased. This study seeks to examine creativity in business problem-solving by studying the effects of stress on human physiological response and cognitive functions in the hope of providing a new and objective interpretation of existing research results.

Using a 360° Virtual Reality or 2D Video to Learn History Taking and Physical Examination Skills for Undergraduate Medical Students: Pilot Randomized Controlled Trial

BACKGROUND

Learning through a 360° virtual reality (VR) or 2D video represents an alternative way to learn a complex medical education task. However, there is currently no consensus on how best to assess the effects of different learning materials on cognitive load estimates, heart rate variability (HRV), outcomes, and experience in learning history taking and physical examination (H&P) skills.

OBJECTIVE

The aim of this study was to investigate how learning materials (ie, VR or 2D video) impact learning outcomes and experience through changes in cognitive load estimates and HRV for learning H&P skills.

METHODS

This pilot system-design study included 32 undergraduate medical students at an academic teaching hospital. The students were randomly assigned, with a 1:1 allocation, to a 360° VR video group or a 2D video group, matched by age, sex, and cognitive style. The contents of both videos were different with regard to visual angle and self-determination. Learning outcomes were evaluated using the Milestone reporting form. Subjective and objective cognitive loads were estimated using the Paas Cognitive Load Scale, the National Aeronautics and Space Administration Task Load Index, and secondary-task reaction time. Cardiac autonomic function was assessed using HRV measurements. Learning experience was assessed using the AttrakDiff2 questionnaire and qualitative feedback. Statistical significance was accepted at a two-sided P value of <.01.

RESULTS

All 32 participants received the intended intervention. The sample consisted of 20 (63%) males and 12 (38%) females, with a median age of 24 (IQR 23-25) years. The 360° VR video group seemed to have a higher Milestone level than the 2D video group (P=.04). The reaction time at the 10th minute in the 360° VR video group was significantly higher than that in the 2D video group (P<.001). Multiple logistic regression models of the overall cohort showed that the 360° VR video module was independently and positively associated with a reaction time at the 10th minute of ≥3.6 seconds (exp B=18.8, 95% CI 3.2-110.8; P=.001) and a Milestone level of ≥3 (exp B=15.0, 95% CI 2.3-99.6; P=.005). However, a reaction time at the 10th minute of ≥3.6 seconds was not related to a Milestone level of ≥3. A low-frequency to high-frequency ratio between the 5th and 10th minute of ≥1.43 seemed to be inversely associated with a hedonic stimulation score of ≥2.0 (exp B=0.14, 95% CI 0.03-0.68; P=.015) after adjusting for video module. The main qualitative feedback indicated that the 360° VR video module was fun but caused mild dizziness, whereas the 2D video module was easy to follow but tedious.

CONCLUSIONS

Our preliminary results showed that 360° VR video learning may be associated with a better Milestone level than 2D video learning, and that this did not seem to be related to cognitive load estimates or HRV indexes in the novice learners. Of note, an increase in sympathovagal balance may have been associated with a lower hedonic stimulation score, which may have met the learners' needs and prompted learning through the different video modules.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03501641; https://clinicaltrials.gov/ct2/show/NCT03501641.

Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda

Heart rate (HR) is one of the essential vital signs used to indicate the physiological health of the human body. While traditional HR monitors usually require contact with skin, remote photoplethysmography (rPPG) enables contactless HR monitoring by capturing subtle light changes of skin through a video camera. Given the vast potential of this technology in the future of digital healthcare, remote monitoring of physiological signals has gained significant traction in the research community. In recent years, the success of deep learning (DL) methods for image and video analysis has inspired researchers to apply such techniques to various parts of the remote physiological signal extraction pipeline. In this paper, we discuss several recent advances of DL-based methods specifically for remote HR measurement, categorizing them based on model architecture and application. We further detail relevant real-world applications of remote physiological monitoring and summarize various common resources used to accelerate related research progress. Lastly, we analyze the implications of research findings and discuss research gaps to guide future explorations.

Analysis of Dynamic Changes in Cognitive Workload During Cardiac Surgery Perfusionists' Interactions With the Cardiopulmonary Bypass Pump

OBJECTIVE

This novel preliminary study sought to capture dynamic changes in heart rate variability (HRV) as a proxy for cognitive workload among perfusionists while operating the cardiopulmonary bypass (CPB) pump during real-life cardiac surgery.

BACKGROUND

Estimations of operators' cognitive workload states in naturalistic settings have been derived using noninvasive psychophysiological measures. Effective CPB pump operation by perfusionists is critical in maintaining the patient's homeostasis during open-heart surgery. Investigation into dynamic cognitive workload fluctuations, and their relationship with performance, is lacking in the literature.

METHOD

HRV and self-reported cognitive workload were collected from three Board-certified cardiac perfusionists (N = 23 cases). Five HRV components were analyzed in consecutive nonoverlapping 1-min windows from skin incision through sternal closure. Cases were annotated according to predetermined phases: prebypass, three phases during bypass, and postbypass. Values from all 1min time windows within each phase were averaged.

RESULTS

Cognitive workload was at its highest during the time between initiating bypass and clamping the aorta (preclamp phase during bypass), and decreased over the course of the bypass period.

CONCLUSION

We identified dynamic, temporal fluctuations in HRV among perfusionists during cardiac surgery corresponding to subjective reports of cognitive workload. Not only does cognitive workload differ for perfusionists during bypass compared with pre- and postbypass phases, but differences in HRV were also detected within the three bypass phases.

APPLICATION

These preliminary findings suggest the preclamp phase of CPB pump interaction corresponds to higher cognitive workload, which may point to an area warranting further exploration using passive measurement.

Classification and Prediction of Post-Trauma Outcomes Related to PTSD Using Circadian Rhythm Changes Measured via Wrist-Worn Research Watch in a Large Longitudinal Cohort

Post-Traumatic Stress Disorder (PTSD) is a psychiatric condition resulting from threatening or horrifying events. We hypothesized that circadian rhythm changes, measured by a wrist-worn research watch are predictive of post-trauma outcomes.

APPROACH

1618 post-trauma patients were enrolled after admission to emergency departments (ED). Three standardized questionnaires were administered at week eight to measure post-trauma outcomes related to PTSD, sleep disturbance, and pain interference with daily life. Pulse activity and movement data were captured from a research watch for eight weeks. Standard and novel movement and cardiovascular metrics that reflect circadian rhythms were derived using this data. These features were used to train different classifiers to predict the three outcomes derived from week-eight surveys. Clinical surveys administered at ED were also used as features in the baseline models.

RESULTS

The highest cross-validated performance of research watch-based features was achieved for classifying participants with pain interference by a logistic regression model, with an area under the receiver operating characteristic curve (AUC) of 0.70. The ED survey-based model achieved an AUC of 0.77, and the fusion of research watch and ED survey metrics improved the AUC to 0.79.

SIGNIFICANCE

This work represents the first attempt to predict and classify post-trauma symptoms from passive wearable data using machine learning approaches that leverage the circadian desynchrony in a potential PTSD population.

Non-Contact Heart Rate Estimation via Adaptive RGB/NIR Signal Fusion

We propose a non-contact heart rate (HR) estimation method that is robust to various situations, such as bright, low-light, and varying illumination scenes. We utilize a camera that records red, green, and blue (RGB) and near-infrared (NIR) information to capture the subtle skin color changes induced by the cardiac pulse of a person. The key novelty of our method is the adaptive fusion of RGB and NIR signals for HR estimation based on the analysis of background illumination variations. RGB signals are suitable indicators for HR estimation in bright scenes. Conversely, NIR signals are more reliable than RGB signals in scenes with more complex illumination, as they can be captured independently of the changes in background illumination. By measuring the correlations between the lights reflected from the background and facial regions, we adaptively utilize RGB and NIR observations for HR estimation. The experiments demonstrate the effectiveness of the proposed method.

Computer Vision for Brain Disorders Based Primarily on Ocular Responses

Real-time ocular responses are tightly associated with emotional and cognitive processing within the central nervous system. Patterns seen in saccades, pupillary responses, and spontaneous blinking, as well as retinal microvasculature and morphology visualized via office-based ophthalmic imaging, are potential biomarkers for the screening and evaluation of cognitive and psychiatric disorders. In this review, we outline multiple techniques in which ocular assessments may serve as a non-invasive approach for the early detections of various brain disorders, such as autism spectrum disorder (ASD), Alzheimer's disease (AD), schizophrenia (SZ), and major depressive disorder (MDD). In addition, rapid advances in artificial intelligence (AI) present a growing opportunity to use machine learning-based AI, especially computer vision (CV) with deep-learning neural networks, to shed new light on the field of cognitive neuroscience, which is most likely to lead to novel evaluations and interventions for brain disorders. Hence, we highlight the potential of using AI to evaluate brain disorders based primarily on ocular features.

Improving Stress Management and Sleep Hygiene in Intelligent Homes

High stress levels and sleep deprivation may cause several mental or physical health issues, such as depression, impaired memory, decreased motivation, obesity, etc. The COVID-19 pandemic has produced unprecedented changes in our lives, generating significant stress, and worries about health, social isolation, employment, and finances. To this end, nowadays more than ever, it is crucial to deliver solutions that can help people to manage and control their stress, as well as to reduce sleep disturbances, so as to improve their health and overall quality of life. Technology, and in particular Ambient Intelligence Environments, can help towards that direction, when considering that they are able to understand the needs of their users, identify their behavior, learn their preferences, and act and react in their interest. This work presents two systems that have been designed and developed in the context of an Intelligent Home, namely CaLmi and HypnOS, which aim to assist users that struggle with stress and poor sleep quality, respectively. Both of the systems rely on real-time data collected by wearable devices, as well as contextual information retrieved from the ambient facilities of the Intelligent Home, so as to offer appropriate pervasive relaxation programs (CaLmi) or provide personalized insights regarding sleep hygiene (HypnOS) to the residents. This article will describe the design process that was followed, the functionality of both systems, the results of the user studies that were conducted for the evaluation of their end-user applications, and a discussion about future plans.

HRVCam: robust camera-based measurement of heart rate variability

SIGNIFICANCE

Non-contact, camera-based heart rate variability estimation is desirable in numerous applications, including medical, automotive, and entertainment. Unfortunately, camera-based HRV accuracy and reliability suffer due to two challenges: (a) darker skin tones result in lower SNR and (b) relative motion induces measurement artifacts.

AIM

We propose an algorithm HRVCam that provides sufficient robustness to low SNR and motion-induced artifacts commonly present in imaging photoplethysmography (iPPG) signals.

APPROACH

HRVCam computes camera-based HRV from the instantaneous frequency of the iPPG signal. HRVCam uses automatic adaptive bandwidth filtering along with discrete energy separation to estimate the instantaneous frequency. The parameters of HRVCam use the observed characteristics of HRV and iPPG signals.

RESULTS

We capture a new dataset containing 16 participants with diverse skin tones. We demonstrate that HRVCam reduces the error in camera-based HRV metrics significantly (more than 50% reduction) for videos with dark skin and face motion.

CONCLUSION

HRVCam can be used on top of iPPG estimation algorithms to provide robust HRV measurements making camera-based HRV practical.

Effects of Dynamic Resilience on the Reactivity of Vagally Mediated Heart Rate Variability

Dynamic resilience is a novel concept that aims to quantify how individuals are coping while operating in dynamic and complex task environments. A recently developed dynamic resilience measure, derived through autoregressive modeling, offers an avenue toward dynamic resilience classification that may yield valuable information about working personnel for industries such as defense and elite sport. However, this measure classifies dynamic resilience based upon in-task performance rather than self-regulating cognitive structures; thereby, lacking any supported self-regulating cognitive links to the dynamic resilience framework. Vagally mediated heart rate variability (vmHRV) parameters are potential physiological measures that may offer an opportunity to link self-regulating cognitive structures to dynamic resilience given their supported connection to the self-regulation of stress. This study examines if dynamic resilience classifications reveal significant differences in vagal reactivity between higher, moderate and lower dynamic resilience groups, as participants engage in a dynamic, decision-making task. An amended Three Rs paradigm was implemented that examined vagal reactivity across six concurrent vmHRV reactivity segments consisting of lower and higher task load. Overall, the results supported significant differences between higher and moderate dynamic resilience groups' vagal reactivity but rejected significant differences between the lower dynamic resilience group. Additionally, differences in vagal reactivity across vmHRV reactivity segments within an amended Three Rs paradigm were partially supported. Together, these findings offer support toward linking dynamic resilience to temporal self-regulating cognitive structures that play a role in mediating physiological adaptations during task engagement.

Single port robotic radical prostatectomy versus multi-port robotic radical prostatectomy: A human factor analysis during the initial learning curve

BACKGROUND

Studies have thus far neglected to evaluate the impact of the da Vinci single port (SP) robotic platform on surgeon experience and operating room efficiency. We sought to assess the effect of the SP platform on surgeon cognitive load measures during robotic assisted laparoscopic prostatectomy (RALP).

METHODS

We prospectively compared the first 20 SP-RALPs performed at our institution to 20 multi-port (MP)-RALPs performed by a single experienced robotic surgeon. Three multi-dimensional assessment tools were used to evaluate mental and surgical workload, teamwork and workflow disruptions.

RESULTS

No statistically significant differences were found between the MP-RALP and SP-RALP cohorts when evaluated by NASA Task Load Index, Surgery Task Load Index and Observational Teamwork Assessments.

CONCLUSIONS

The SP robotic platform did not adversely affect human factor performance of the surgeon during RALP. Multi-institutional validation will be necessary to confirm these initial findings.

Autonomic Activity and Surgical Flow Disruptions in Healthcare Providers during Cardiac Surgery

Cardiac surgery represents a complex sociotechnical environment relying on a combination of technical and non-technical team-based expertise. Surgical flow disruptions (SFDs) may be influenced by a variety of sources, including social, environmental, and emotional factors affecting healthcare providers (HCPs). Many of these factors can be readily observed, except for emotional factors (i.e. distress), which represents an underappreciated yet critical source of SFDs. The aim of this study was to demonstrate the sensitivity of autonomic activity metrics to detect an SFD during cardiac surgery. We integrated heart rate variability (HRV) analysis with observation-based annotations to allow data triangulation. Following a critical medication administration error by the anesthesiologist in-training, data sources were consulted to identify events precipitating this near-miss event. Using pyphysio, an open-source physiological signal processing package, we analyzed the attending anesthesiologists' HRV, specifically the low frequency (LF) power, high frequency (HF) power, LF/HF ratio, standard deviation of normal-to-normal (SDNN), and root mean square of the successive differences (RMSSD) as indicators of ANS activity. A heightened SNS response in the attending anesthesiologists' physiological arousal was observed as elevations in LF power and LF/HF ratio, as well as depressions in HF power, SDNN, and RMSSD prior to the near-miss event. The attending anesthesiologist subjectively confirmed a state of high distress induced by task-irrelevant environmental factors during this time. Qualitative analysis of audio/video recordings objectively revealed that the autonomic nervous system (ANS) activation detected was temporally associated with an argument over operating room management. This study confirms that it is possible to recognize detrimental psychophysiological influences in cardiac surgery procedures via advanced HRV analysis. To our knowledge, ours is the first such case demonstrating ANS activity coinciding with strong self-reported emotion during live surgery using HRV. Despite extensive experience in the cardiac OR, transient but intense emotional changes may have the potential to disrupt attention processes in even the most experienced HCP. A primary implication of this work is the possibility to detect real-time ANS activity, which could enable personalized interventions to proactively mitigate downstream adverse events. Additional studies on our large database of surgical cases are underway and new studies are actively being planned to confirm this preliminary observation.

Predicting Cognitive Load and Operational Performance in a Simulated Marksmanship Task

Modern operational environments can place significant demands on a service member's cognitive resources, increasing the risk of errors or mishaps due to overburden. The ability to monitor cognitive burden and associated performance within operational environments is critical to improving mission readiness. As a key step toward a field-ready system, we developed a simulated marksmanship scenario with an embedded working memory task in an immersive virtual reality environment. As participants performed the marksmanship task, they were instructed to remember numbered targets and recall the sequence of those targets at the end of the trial. Low and high cognitive load conditions were defined as the recall of three- and six-digit strings, respectively. Physiological and behavioral signals recorded included speech, heart rate, breathing rate, and body movement. These features were input into a random forest classifier that significantly discriminated between the low- and high-cognitive load conditions (AUC = 0.94). Behavioral features of gait were the most informative, followed by features of speech. We also showed the capability to predict performance on the digit recall (AUC = 0.71) and marksmanship (AUC = 0.58) tasks. The experimental framework can be leveraged in future studies to quantify the interaction of other types of stressors and their impact on operational cognitive and physical performance.

Non-contact imaging of peripheral hemodynamics during cognitive and psychological stressors

Peripheral hemodynamics, measured via the blood volume pulse and vasomotion, provide a valuable way of monitoring physiological state. Camera imaging-based systems can be used to measure these peripheral signals without contact with the body, at distances of multiple meters. While researchers have paid attention to non-contact imaging photoplethysmography, the study of peripheral hemodynamics and the effect of autonomic nervous system activity on these signals has received less attention. Using a method, based on a tissue-like model of the skin, we extract melanin [Formula: see text] and hemoglobin [Formula: see text] concentrations from videos of the hand and face and show that significant decreases in peripheral pulse signal power (by 36% ± 29%) and vasomotion signal power (by 50% ± 26%) occur during periods of cognitive and psychological stress. Via three experiments we show that similar results are achieved across different stimuli and regions of skin (face and hand). While changes in peripheral pulse and vasomotion power were significant the changes in pulse rate variability were less consistent across subjects and tasks.

On-line anxiety level detection from biosignals: Machine learning based on a randomized controlled trial with spider-fearful individuals

We present performance results concerning the validation for anxiety level detection based on trained mathematical models using supervised machine learning techniques. The model training is based on biosignals acquired in a randomized controlled trial. Wearable sensors were used to collect electrocardiogram, electrodermal activity, and respiration from spider-fearful individuals. We designed and applied ten approaches for data labeling considering individual biosignals as well as subjective ratings. Performance results revealed a selection of trained models adapted for two-level (low and high) and three-level (low, medium and high) classification of anxiety using a minimal set of six features. We obtained a remarkable accuracy of 89.8% for the two-level classification and of 74.4% for the three-level classification using a short time window length of ten seconds when applying the approach that uses subjective ratings for data labeling. Bagged Trees proved to be the most suitable classifier type among the classification models studied. The trained models will have a practical impact on the feasibility study of an augmented reality exposure therapy based on a therapeutic game for the treatment of arachnophobia.

Heart Rate Variability and Accelerometry as Classification Tools for Monitoring Perceived Stress Levels-A Pilot Study on Firefighters

Chronic stress is the main cause of health problems in high-risk jobs. Wearable sensors can become an ecologically valid method of stress level assessment in real-life applications. We sought to determine a non-invasive technique for objective stress monitoring. Data were collected from firefighters during 24-h shifts using sensor belts equipped with a dry-lead electrocardiograph (ECG) and a three-axial accelerometer. Levels of stress experienced during fire incidents were evaluated via a brief self-assessment questionnaire. Types of physical activity were distinguished basing on accelerometer readings, and heart rate variability (HRV) time series were segmented accordingly into corresponding fragments. Those segments were classified as stress/no-stress conditions. Receiver Operating Characteristic (ROC) analysis showed true positive classification as stress condition for 15% of incidents (while maintaining almost zero False Positive Rate), which parallels the amount of truly stressful incidents reported in the questionnaires. These results show a firm correspondence between the perceived stress level and physiological data. Psychophysiological measurements are reliable indicators of stress even in ecological settings and appear promising for chronic stress monitoring in high-risk jobs, such as firefighting.

Video-Based Pulse Rate Variability Measurement Using Periodic Variance Maximization and Adaptive Two-Window Peak Detection

Many previous studies have shown that the remote photoplethysmography (rPPG) can measure the Heart Rate (HR) signal with very high accuracy. The remote measurement of the Pulse Rate Variability (PRV) signal is also possible, but this is much more complicated because it is then necessary to detect the peaks on the temporal rPPG signal, which is usually quite noisy and has a lower temporal resolution than PPG signals obtained by contact equipment. Since the PRV signal is vital for various applications such as remote recognition of stress and emotion, the improvement of PRV measurement by rPPG is a critical task. Contact based PRV measurement has already been investigated, but the research on remotely measured PRV is very limited. In this paper, we propose to use the Periodic Variance Maximization (PVM) method to extract the rPPG signal and event-related Two-Window algorithm to improve the peak detection for PRV measurement. We have made several contributions. Firstly, we show that the newly proposed PVM method and Two-Window algorithm can be used for PRV measurement in the non-contact scenario. Secondly, we propose a method to adaptively determine the parameters of the Two-Window method. Thirdly, we compare the algorithm with other attempts for improving the non-contact PRV measurement such as the Slope Sum Function (SSF) method and the Local Maximum method. We calculated several features and compared the accuracy based on the ground truth provided by contact equipment. Our experiments showed that this algorithm performed the best of all the algorithms.

Prediction of Self-Perceived Stress and Arousal Based on Electrodermal Activity

Electrodermal activity (EDA) reflects the functions of autonomic nervous system and is often used in evaluation of mental states, e.g. short- and long-term stress. In this study, test subjects were exposed to a 3-phase adapted MIST test (relaxation, arousal, stress) during which EDA was recorded, and the self-perceived stress and arousal were assessed. The objective of the study was to evaluate the feasibility of EDA features to predict the MIST test phases and self-perceived stress and arousal. With EDA features, the test phases were classified with accuracy of 94.1%, and the self-perceived stress and arousal were classified with accuracy of 60.5-72.2%. Results are promising for the use of EDA for long-term assessment of self-perceived stress and arousal during work.

An open-source remote heart rate imaging method with practical apparatus and algorithms

Recent developments in computer science and digital image processing have enabled the extraction of an individual’s heart pulsations from pixel changes in recorded video images of human skin surfaces. This method is termed remote photoplethysmography (rPPG) and can be achieved with consumer-level cameras (e.g., a webcam or mobile camera). The goal of the present publication is two-fold. First, we aim to organize future rPPG software developments in a tractable and nontechnical manner, such that the public gains access to a basic open-source rPPG code, comes to understand its utility, and can follow its most recent progressions. The second goal is to investigate rPPG’s accuracy in detecting heart rates from the skin surfaces of several body parts after physical exercise and under ambient lighting conditions with a consumer-level camera. We report that rPPG is highly accurate when the camera is aimed at facial skin tissue, but that the heart rate recordings from wrist regions are less reliable, and recordings from the calves are unreliable. Facial rPPG remained accurate despite the high heart rates after exercise. The proposed research procedures and the experimental findings provide guidelines for future studies on rPPG.

Using Non-Contact Imaging Photoplethysmography to Recover Diurnal Patterns in Heart Rate

Daily patterns in cardiovascular signals can reveal important information about physiological processes, health and well-being. Traditionally, contact sensors have been used to collect longitudinal data of this kind. However, recent advances in non-contact imaging techniques have led to algorithms that can be used to measure vital signs unobtrusively. Imaging methods are highly scalable due to the availability of webcams and computing devices making them attractive for longitudinal, in-situ measurement. Using a software tool we captured over 1,000 hours of non-contact heart rate measurements, via imaging photoplethysmography. Using these data we were able to recover diurnal patterns in heart rate during the working day. Non-contact sensing techniques hold much promise but also raise ethical issues that need to be addressed seriously within the biomedical engineering community.

Feasibility of assessing ultra-short-term pulse rate variability from video recordings

Objectives

Remote photoplethysmography (rPPG) is a promising non-contact measurement technique for assessing numerous physiological parameters: pulse rate, pulse rate variability (PRV), respiratory rate, pulse wave velocity, blood saturation, blood pressure, etc. To justify its use in ultra-short-term (UST) PRV analysis, which is of great benefit for several healthcare applications, the agreement between rPPG- and PPG-derived UST-PRV metrics was studied.

Approach

Three time-domain metrics—standard deviation of normal-to-normal (NN) intervals (SDNN), root mean square of successive NN interval differences (RMSSD), and the percentage of adjacent NN intervals that differ from each other by more than 50 ms (pNN50)—were extracted from 56 video recordings in a publicly available data set. The selected metrics were calculated on the basis of three groups of 10 s recordings and their average, two groups of 30 s recordings and their average, and a group of 60 s recordings taken from the full-length recordings and then compared with metrics derived from the corresponding reference (PPG) pulse waveform signals by using correlation and effect size parameters, and Bland–Altman plots.

Main results

The results show there is stronger agreement as the recording length increases for SDNN and RMSSD, yet there is no significant change for pNN50. The agreement parameters reach r = 0.841 (p < 0.001), r = 0.529 (p < 0.001), and r = 0.657 (p < 0.001), estimated median bias −1.52, −2.28 ms and −1.95% and a small effect size for SDNN, RMSSD, and pNN50 derived from the 60 s recordings, respectively.

Significance

Remote photoplethysmography-derived UST-PRV metrics manage to capture UST-PRV metrics derived from reference (PPG) recordings well. This feature is highly desirable in numerous applications for the assessment of one’s health and well-being. In future research, the validity of rPPG-derived UST-PRV metrics compared to the gold standard electrocardiography recordings is to be assessed.

Machine Learning for Cognitive Load Classification – A Case Study on Contact-Free Approach

The most common ways of measuring Cognitive Load (CL) is using physiological sensor signals e.g., Electroencephalography (EEG), or Electrocardiogram (ECG). However, these signals are problematic in situations e.g., in dynamic moving environments where the user cannot relax with all the sensors attached to the body and it provides significant noises in the signals. This paper presents a case study using a contact-free approach for CL classification based on Heart Rate Variability (HRV) collected from ECG signal. Here, a contact-free approach i.e., a camera-based system is compared with a contact-based approach i.e., Shimmer GSR+ system in detecting CL. To classify CL, two different Machine Learning (ML) algorithms, mainly, Support Vector Machine (SVM) and k-Nearest-Neighbor (k-NN) have been applied. Based on the gathered Inter-Beat-Interval (IBI) values from both the systems, 13 different HRV features were extracted in a controlled study to determine three levels of CL i.e., S0: low CL, S1: normal CL and S2: high CL. To get the best classification accuracy with the ML algorithms, different optimizations such as kernel functions were chosen with different feature matrices both for binary and combined class classifications. According to the results, the highest average classification accuracy was achieved as 84% on the binary classification i.e. S0 vs S2 using k-NN. The highest F₁ score was achieved 88% using SVM for the combined class considering S0 vs (S1 and S2) for contact-free approach i.e. the camera system. Thus, all the ML algorithms achieved a higher classification accuracy while considering the contact-free approach than contact-based approach.

Non-Contact Remote Measurement of Heart Rate Variability using Near-Infrared Photoplethysmography Imaging

Heart rate variability (HRV) is an important clinical parameter associated with the autonomous nervous system (ANS), age, as well as many diseases such as myocardial infarction, diabetes or renal failure. Gold standard for measurement of HRV is a high-resolution electrocardiogram (ECG). With the current trend towards non-contact and unobtrusive monitoring of vital signs, HRV has also become an interesting and important parameter for non-contact monitoring. In this paper, we present an approach towards non-contact and unobtrusive monitoring of heart rate variability using the camera-based technology of photoplethysmography imaging (PPGI). We investigated the suitability of invisible near-infrared illumination for PPGI, which would enable measurement of HRV in darkness. We compared results obtained using infrared illumination with those obtained using visible light as PPGI illumination and calculated both time-domain as well as frequency-domain HRV parameters. The results achieved with infrared illumination were on par with those using conventional illumination in the visible spectrum. We concluded that infrared illumination enables unobtrusive and non-contact remote HRV measurement in both darkness as well as regular daylight conditions using PPGI.

Reckoning respiratory signals to affectively decipher mental state

Recognizing mental states from physiological signal is a concern not only for medical diagnostics, but also for cognitive science, behavioral studies as well as brain machine interfaces. This study employs an unique approach of solely utilizing the respiration signals in order to decipher mental states. A public dataset, Affective Pacman, is considered for this study, where the various physiological signals are acquired during normal and frustrated mental states. An efficient way to remove the non-linear baseline drifts in the signal is implemented to extract the respiratory features in most effective way. Another major adversity is the presence of class imbalance, which is effectively rectified using Synthetic Minority Oversampling TEchnique (SMOTE). Application of SMOTE algorithm to resolve class imbalance problem, not only increased the classification accuracy, but also reduced the classifier bias towards the majority class, which in turn exceedingly enhanced the classifier sensitivity. The multilayer perceptron classifier performed best with SMOTE generated feature set, with classification accuracy (CA), true positive rate (TPR) and true negative rate (TNR) of 97.9%, 92.6% and 99.3% respectively. The current approach is found to perform better compared to relevant literature.

Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.

Predicting Emotion and Engagement of Workers in Order Picking Based on Behavior and Pulse Waves Acquired by Wearable Devices

Many logistics companies adopt a manual order picking system. In related research, the effect of emotion and engagement on work efficiency and human errors was verified. However, related research has not established a method to predict emotion and engagement during work with high exercise intensity. Therefore, important variables for predicting the emotion and engagement during work with high exercise intensity are not clear. In this study, to clarify the mechanism of occurrence of emotion and engagement during order picking. Then, we clarify the explanatory variables which are important in predicting the emotion and engagement during work with high exercise intensity. We conducted verification experiments. We compared the accuracy of estimating human emotion and engagement by inputting pulse wave, eye movements, and movements to deep neural networks. We showed that emotion and engagement during order picking can be predicted from the behavior of the worker with an accuracy of error rate of 0.12 or less. Moreover, we have constructed a psychological model based on the questionnaire results and show that the work efficiency of workers is improved by giving them clear targets.

Intra-Individual Variability in Vagal Control Is Associated With Response Inhibition Under Stress

Dynamic intra-individual variability (IIV) in cardiac vagal control across multiple situations is believed to contribute to adaptive cognition under stress; however, a dearth of research has empirically tested this notion. To this end, we examined 25 U.S. Army Soldiers (all male, mean age = 30.73, standard deviation (SD) = 7.71) whose high-frequency heart rate variability (HF-HRV) was measured during a resting baseline and during three conditions of a shooting task (training, low stress, high stress). Response inhibition was measured as the correct rejection (CR) of friendly targets during the low and high stress conditions. We tested the association between the SD of HF-HRV across all four task conditions (IIV in vagal control) and changes in response inhibition between low and high stress. Greater differences in vagal control between conditions (larger IIV) were associated with higher tonic vagal control during rest, and stronger stress-related decreases in response inhibition. These results suggest that flexibility in vagal control is supported by tonic vagal control, but this flexibility also uniquely relates to adaptive cognition under stress. Findings are consistent with neurobehavioral and dynamical systems theories of vagal function.

Spectral Estimation Methods for Evaluating iPPG Pulse Rate Variability

Non-contact measurement of physiological parameters, like pulse rate variability (PRV), has numerous applications in medicine and affective computing. PRV is an informative measure of autonomic nervous system activity. Spectral estimation from unevenly sampled, non-stationary data is integral to pulse rate variability frequency-domain analysis. We present the first comparison of results of PRV computation using the Lomb-Scargle method and Bayesian Spectral Estimation. The Lomb-Scargle method performs well, even in the presence of missing beats. However, the Bayesian Spectral Estimation method has advantages when tracking changes in amplitude and frequency. We illustrate these characteristics with results from synthetic data and real non-contact imaging photoplethysmography measurements.

Effect of motion artifact on digital camera based heart rate measurement

Remote health monitoring is an emerging field in biomedical technology. Digital camera based heart rate measurement method is a recent development which would make remote health monitoring reliable and sustainable in future. This paper presents an investigation on the effect of motion artifact on digital camera-based heart rate measurement. The paper will discuss details on the principles and effects of motion artifacts on photoplethysmography signals. An experiment is conducted using publicly available MAHNOB-HCI database. We have investigated the effects of static scenarios, scenarios involving rigid motion and scenarios involving non-rigid motion. The experiment was tested on state of the art digital camera based heart rate measuring methods. The results showed the effectiveness of the methods and provide a direction to overcome/minimize the effect of motion artifacts for future research.

Analysis of the Relationship between Road Accidents and Psychophysical State of Drivers through Wearable Devices

A driver’s behavior and their psychophysical state are the most common causes of road accidents. The research presented in the paper proposes a method that allows the identification of highly dangerous road stretches/intersections in advance, based on the localization of stressful/relaxing situations measured on drivers. These were measured through the collection of physiological parameters using wearable devices. A correlation between stressful/relaxing situations and locations with high accident rates, based on a historical statistical database (black spots), was investigated. A series of driving tests was conducted in the city of Milan. The first set was mostly oriented to the research and validation of the parameters related to the driver’s psychophysical state. Subsequent tests allowed the definition of a correlation between black spots and relaxing/stressful areas. The results showed that the most stressful areas for drivers fell mainly within those with high accident rates. Furthermore, 80% of the most dangerous zones of the route were identified using this method, thus confirming the validity of the approach as a support tool for a priori preventive analysis for road safety. The wearable devices allowed the study and the integration of specific elements relating to human behavior in the field of road safety, which typically involves a technical-engineering approach.

Self-adaptive signal separation for non-contact heart rate estimation from facial video in realistic environments

OBJECTIVE

Recent research indicates that facial epidermis color varies with the rhythm of heat beats. It can be captured by consumer-level cameras and, astonishingly, be adopted to estimate heart rate (HR). The HR estimated remains not as precise as required in a practical environment where illumination interference, facial expressions, or motion artifacts are involved, although numerous methods have been proposed in the last few years. A novel algorithm is proposed to make non-contact HR estimation technique more robust.

APPROACH

First, the face of the subject is detected and tracked to follow the head movement. The facial region then falls into several blocks, and the chrominance feature of each block is extracted to establish a raw HR sub-signal. Self-adaptive signal separation is performed to separate the noiseless HR sub-signals from raw sub-signals. On that basis, the noiseless sub-signals full of HR information are selected using a weight-based scheme to establish the holistic HR signal, from which the average HR is computed adopting wavelet transform and data filtering.

MAIN RESULTS

Forty subjects took part in our experiments, whose facial videos were recorded by a normal webcam with the frame rate of 30 fps under ambient lighting conditions. The average HR estimated by our method correlates strongly with ground truth measurements, as indicated in experimental results measured in a static scenario with the Pearson correlation r  =  0.980 and a dynamic scenario with the Pearson correlation r  =  0.897. In addition, our method, compared to the newest method, decreases the error rate by 38.63% and increases the Pearson correlation by 15.59%.

SIGNIFICANCE

This work proposes a robust method for non-contact HR measurement in a realistic environment. Results of comparative experiments indicate that our method out-performs state-of-the-art non-contact HR estimation methods in realistic environments.

Vital Sign Monitoring Through the Back Using an UWB Impulse Radar With Body Coupled Antennas

Radar devices can be used in nonintrusive situations to monitor vital sign, through clothes or behind walls. By detecting and extracting body motion linked to physiological activity, accurate simultaneous estimations of both heart rate (HR) and respiration rate (RR) is possible. However, most research to date has focused on front monitoring of superficial motion of the chest. In this paper, body penetration of electromagnetic (EM) wave is investigated to perform back monitoring of human subjects. Using body-coupled antennas and an ultra-wideband (UWB) pulsed radar, in-body monitoring of lungs and heart motion was achieved. An optimised location of measurement in the back of a subject is presented, to enhance signal-to-noise ratio and limit attenuation of reflected radar signals. Phase-based detection techniques are then investigated for back measurements of vital sign, in conjunction with frequency estimation methods that reduce the impact of parasite signals. Finally, an algorithm combining these techniques is presented to allow robust and real-time estimation of both HR and RR. Static and dynamic tests were conducted, and demonstrated the possibility of using this sensor in future health monitoring systems, especially in the form of a smart car seat for driver monitoring.

Heart rate variability analysis during muscle fatigue due to prolonged isometric contraction

Fatigue can be defined as the muscular condition occurring before the inability to perform a task. It can be assessed through the evaluation of the median and mean frequency of the spectrum of the surface electromyography series. Previous studies investigated the relationship between heartbeat dynamics and muscular activity. However, exploitation of such cardiovascular measures to automatically identify muscle fatigue during fatiguing exercises is still missing. To this extent, HRV signals were gathered from 32 subjects during an isometric contraction task, and features defined in the time, frequency and nonlinear domains were investigated. We used surface electromyography to label the occurrence of muscle fatigue. Statistically significant differences were observed by comparing features related to fatigued subjects with the non-fatigued ones. Moreover, a pattern recognition system capable to achieve an average accuracy of 78.24% was implemented. These results confirmed the hypothesis that a relationship between heartbeat dynamics and muscle fatigue might exist.

Measuring pulse rate variability using long-range, non-contact imaging photoplethysmography

Camera-based measurement of the blood volume pulse via non-contact, imaging photoplethysmography is a very popular approach for measuring pulse rate using a remote imaging sensor. Comparatively less attention has been paid to the usefulness of the method for measuring features of pulse rate variability, and even less focus has been put on the accuracy of any cardiac activity feature that can be achieved at long imager-to-subject distances. In this study, video was recorded from 19 participants, while at rest, at a distance of 25 meters from the imaging sensor. A digital camera was used to record video while cardiovascular measures of both electrical and optical physiological ground truth were recorded. Pulse rate data obtained from the imager using a common blind source separation and periodogram approach were compared to physiological ground truth signals. The quality of the recovered blood volume pulse morphology was sufficient to calculate time-domain measures of pulse rate using inter-pulse interval (IPI) time series. Following this, several features of pulse rate variability were calculated from the IPI time series and compared to those calculated from the corresponding physiological ground truth signals. Use of the time-domain data as compared to the periodogram approach to measure pulse rate reduced the error in the estimate from 1.6 to 0.2 beats per minute. Correlation analysis (r2) between the camera-based measures of pulse rate variability and ECG-derived heart rate variability ranged from 0.779 to 0.973; these results are of comparable outcome to those obtained at imager-to-subject distances of no more than 3 meters. This study demonstrates that pulse rates of less than one beat-per-minute error can be obtained when the recovered blood volume pulse morphology is of adequate quality to resolve systolic onsets for individual cardiac cycles. Further, this approach can yield data of very promising quality for estimating measures of pulse rate variability.