Advancements in noncontact, multiparameter physiological measurements using a webcam

Modular continuous wavelet processing of biosignals: extracting heart rate and oxygen saturation from a video signal

A novel method of extracting heart rate and oxygen saturation from a video-based biosignal is described. The method comprises a novel modular continuous wavelet transform approach which includes: performing the transform, undertaking running wavelet archetyping to enhance the pulse information, extraction of the pulse ridge time-frequency information [and thus a heart rate (HRvid) signal], creation of a wavelet ratio surface, projection of the pulse ridge onto the ratio surface to determine the ratio of ratios from which a saturation trending signal is derived, and calibrating this signal to provide an absolute saturation signal (SvidO2). The method is illustrated through its application to a video photoplethysmogram acquired during a porcine model of acute desaturation. The modular continuous wavelet transform-based approach is advocated by the author as a powerful methodology to deal with noisy, non-stationary biosignals in general.

Noncontact Vision-Based Cardiopulmonary Monitoring in Different Sleeping Positions

Individuals with obstructive sleep apnea (OSA) can experience partial or complete collapse of the upper airway during sleep. This condition affects between 10-17% of adult men and 3-9% of adult women, requiring arousal to resume regular breathing. Frequent arousals disrupt proper sleeping patterns and cause daytime sleepiness. Untreated OSA has been linked to serious medical issues including cardiovascular disease and diabetes. Unfortunately, diagnosis rates are low (∼20%) and current sleep monitoring options are expensive, time consuming, and uncomfortable. Toward the development of a convenient, noncontact OSA monitoring system, this paper presents a simple, computer vision-based method to monitor cardiopulmonary signals (respiratory and heart rates) during sleep. System testing was performed with 17 healthy participants in five different simulated sleep positions. To monitor cardiopulmonary rates, distinctive points are automatically detected and tracked in infrared image sequences. Blind source separation is applied to extract candidate signals of interest. The optimal respiratory and heart rates are determined using periodicity measures based on spectral analysis. Estimates were validated by comparison to polysomnography recordings. The system achieved a mean percentage error of 3.4% and 5.0% for respiratory rate and heart rate, respectively. This study represents an important step in building an accessible, unobtrusive solution for sleep apnea diagnosis.

Computer-Vision-Guided Human Pulse Rate Estimation: A Review

Human pulse rate (PR) can be estimated in several ways, including measurement instruments that directly count the PR through contact- and noncontact-based approaches. Over the last decade, computer-vision-assisted noncontact-based PR estimation has evolved significantly. Such techniques can be adopted for clinical purposes to mitigate some of the limitations of contact-based techniques. However, existing vision-guided noncontact-based techniques have not been benchmarked with respect to a challenging dataset. In view of this, we present a systematic review of such techniques implemented over a uniform computing platform. We have simultaneously recorded the PR and video of 14 volunteers. Five sets of data have been recorded for every volunteer using five different experimental conditions by varying the distance from the camera and illumination condition. Pros and cons of the existing noncontact image- and video-based PR techniques have been discussed with respect to our dataset. Experimental evaluation suggests that image- or video-based PR estimation can be highly effective for nonclinical purposes, and some of these approaches are very promising toward developing clinical applications. The present review is the first in this field of contactless vision-guided PR estimation research.

Sensitivity Enhanced Vital Sign Detection Based on Antenna Reflection Coefficient Variation

This paper presents a vital sign detection sensor based on reflection coefficient variance from an antenna used in wireless communication devices. The near-field effect is estimated by performing 3D full-wave simulations using a dipole antenna and the magnitude variation of the reflection coefficient induced by human thorax movement due to heart and lungs is observed. The results support the possibility of vital sign detection based on the magnitude variation of the reflection coefficient from an antenna, which can be explained as a narrowband modulation scheme. In particular, a sensitivity enhancement method is proposed and analyzed, and experiments are carried out for heartbeat detection using a dipole antenna with the proposed system. Experimental results are compared between the direct detection and sensitivity enhancement detection schemes. FM signal is also applied to confirm that the proposed sensor works properly in conjunction with an existing communication system. The proposed cardiopulmonary detection sensor is implemented with off-the-shelf components at 2.4 GHz and excellent performance is obtained.

Employing an Incentive Spirometer to Calibrate Tidal Volumes Estimated from a Smartphone Camera

A smartphone-based tidal volume (V_T) estimator was recently introduced by our research group, where an Android application provides a chest movement signal whose peak-to-peak amplitude is highly correlated with reference V_T measured by a spirometer. We found a Normalized Root Mean Squared Error (NRMSE) of 14.998% ± 5.171% (mean ± SD) when the smartphone measures were calibrated using spirometer data. However, the availability of a spirometer device for calibration is not realistic outside clinical or research environments. In order to be used by the general population on a daily basis, a simple calibration procedure not relying on specialized devices is required. In this study, we propose taking advantage of the linear correlation between smartphone measurements and V_T to obtain a calibration model using information computed while the subject breathes through a commercially-available incentive spirometer (IS). Experiments were performed on twelve (N = 12) healthy subjects. In addition to corroborating findings from our previous study using a spirometer for calibration, we found that the calibration procedure using an IS resulted in a fixed bias of −0.051 L and a RMSE of 0.189 ± 0.074 L corresponding to 18.559% ± 6.579% when normalized. Although it has a small underestimation and slightly increased error, the proposed calibration procedure using an IS has the advantages of being simple, fast, and affordable. This study supports the feasibility of developing a portable smartphone-based breathing status monitor that provides information about breathing depth, in addition to the more commonly estimated respiratory rate, on a daily basis.

Monitoring of Heart and Breathing Rates Using Dual Cameras on a Smartphone

Some smartphones have the capability to process video streams from both the front- and rear-facing cameras simultaneously. This paper proposes a new monitoring method for simultaneous estimation of heart and breathing rates using dual cameras of a smartphone. The proposed approach estimates heart rates using a rear-facing camera, while at the same time breathing rates are estimated using a non-contact front-facing camera. For heart rate estimation, a simple application protocol is used to analyze the varying color signals of a fingertip placed in contact with the rear camera. The breathing rate is estimated from non-contact video recordings from both chest and abdominal motions. Reference breathing rates were measured by a respiration belt placed around the chest and abdomen of a subject; reference heart rates (HR) were determined using the standard electrocardiogram. An automated selection of either the chest or abdominal video signal was determined by choosing the signal with a greater autocorrelation value. The breathing rate was then determined by selecting the dominant peak in the power spectrum. To evaluate the performance of the proposed methods, data were collected from 11 healthy subjects. The breathing ranges spanned both low and high frequencies (6-60 breaths/min), and the results show that the average median errors from the reflectance imaging on the chest and the abdominal walls based on choosing the maximum spectral peak were 1.43% and 1.62%, respectively. Similarly, HR estimates were also found to be accurate.

Tidal Volume and Instantaneous Respiration Rate Estimation using a Volumetric Surrogate Signal Acquired via a Smartphone Camera

Two parameters that a breathing status monitor should provide include tidal volume ( V_T) and respiration rate (RR). Recently, we implemented an optical monitoring approach that tracks chest wall movements directly on a smartphone. In this paper, we explore the use of such noncontact optical monitoring to obtain a volumetric surrogate signal, via analysis of intensity changes in the video channels caused by the chest wall movements during breathing, in order to provide not only average RR but also information about V_T and to track RR at each time instant (IRR). The algorithm, implemented on an Android smartphone, is used to analyze the video information from the smartphone's camera and provide in real time the chest movement signal from N = 15 healthy volunteers, each breathing at V_T ranging from 300 mL to 3 L. These measurements are performed separately for each volunteer. Simultaneous recording of volume signals from a spirometer is regarded as reference. A highly linear relationship between peak-to-peak amplitude of the smartphone-acquired chest movement signal and spirometer V_T is found ( r² = 0.951 ±0.042, mean ± SD). After calibration on a subject-by-subject basis, no statistically significant bias is found in terms of V_T estimation; the 95% limits of agreement are -0.348 to 0.376 L, and the root-mean-square error (RMSE) was 0.182 ±0.107 L. In terms of IRR estimation, a highly linear relation between smartphone estimates and the spirometer reference was found ( r² = 0.999 ±0.002). The bias, 95% limits of agreement, and RMSE are -0.024 breaths-per-minute (bpm), -0.850 to 0.802 bpm, and 0.414 ±0.178 bpm, respectively. These promising results show the feasibility of developing an inexpensive and portable breathing monitor, which could provide information about IRR as well as V_T, when calibrated on an individual basis, using smartphones. Further studies are required to enable practical implementation of the proposed approach.

Measurement of heart rate variability using off-the-shelf smart phones

Background

The cardiac parameters, such as heart rate (HR) and heart rate variability (HRV), are very important physiological data for daily healthcare. Recently, the camera-based photoplethysmography techniques have been proposed for HR measurement. These techniques allow us to estimate the HR contactlessly with low-cost camera. However, the previous works showed limit success for estimating HRV because the R–R intervals, the primary data for HRV calculation, are sensitive to noise and artifacts.

Methods

This paper proposed a non-contact method to extract the blood volume pulse signal using a chrominance-based method followed by a proposed CWT-based denoising technique. The R–R intervals can then be obtained by finding the peaks in the denoised signal. In this paper, we taped 12 video clips using the frontal camera of a smart phone with different scenarios to make comparisons among our method and the other alternatives using the absolute errors between the estimated HRV metrics and the ones obtained by an ECG-accurate chest band.

Results

As shown in experiments, our algorithm can greatly reduce absolute errors of HRV metrics comparing with the related works using RGB color signals. The mean of absolute errors of HRV metrics from our method is only 3.53 ms for the static-subject video clips.

Conclusions

The proposed camera-based method is able to produce reliable HRV metrics which are close to the ones measured by contact devices under different conditions. Thus, our method can be used for remote health monitoring in a convenient and comfortable way.

Introducing Contactless Blood Pressure Assessment Using a High Speed Video Camera

Recent studies demonstrated that blood pressure (BP) can be estimated using pulse transit time (PTT). For PTT calculation, photoplethysmogram (PPG) is usually used to detect a time lag in pulse wave propagation which is correlated with BP. Until now, PTT and PPG were registered using a set of body-worn sensors. In this study a new methodology is introduced allowing contactless registration of PTT and PPG using high speed camera resulting in corresponding image-based PTT (iPTT) and image-based PPG (iPPG) generation. The iPTT value can be potentially utilized for blood pressure estimation however extent of correlation between iPTT and BP is unknown. The goal of this preliminary feasibility study was to introduce the methodology for contactless generation of iPPG and iPTT and to make initial estimation of the extent of correlation between iPTT and BP "in vivo." A short cycling exercise was used to generate BP changes in healthy adult volunteers in three consecutive visits. BP was measured by a verified BP monitor simultaneously with iPTT registration at three exercise points: rest, exercise peak, and recovery. iPPG was simultaneously registered at two body locations during the exercise using high speed camera at 420 frames per second. iPTT was calculated as a time lag between pulse waves obtained as two iPPG's registered from simultaneous recoding of head and palm areas. The average inter-person correlation between PTT and iPTT was 0.85 ± 0.08. The range of inter-person correlations between PTT and iPTT was from 0.70 to 0.95 (p < 0.05). The average inter-person coefficient of correlation between SBP and iPTT was -0.80 ± 0.12. The range of correlations between systolic BP and iPTT was from 0.632 to 0.960 with p < 0.05 for most of the participants. Preliminary data indicated that a high speed camera can be potentially utilized for unobtrusive contactless monitoring of abrupt blood pressure changes in a variety of settings. The initial prototype system was able to successfully generate approximation of pulse transit time and showed high intra-individual correlation between iPTT and BP. Further investigation of the proposed approach is warranted.

Accurate heart rate estimation from camera recording via MUSIC algorithm

In this paper, we propose an algorithm to extract heart rate frequency from video camera using the Multiple SIgnal Classification (MUSIC) algorithm. This leads to improved accuracy of the estimated heart rate frequency in cases the performance is limited by the number of samples and frame rate. Monitoring vital signs remotely can be exploited for both non-contact physiological and psychological diagnosis. The color variation recorded by ordinary cameras is used for heart rate monitoring. The orthogonality between signal space and noise space is used to find more accurate heart rate frequency in comparison with traditional methods. It is shown via experimental results that the limitation of previous methods can be overcome by using subspace methods.

Physiological Modalities for Relaxation Skill Transfer in Biofeedback Games

We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach consists of monitoring the user's physiology during gameplay and adapting the game using a positive feedback loop that rewards relaxing behaviors and penalizes states of high arousal. We evaluate the approach using a casual game under three biofeedback modalities: electrodermal activity, heart rate variability, and breathing rate. The three biosignals can be measured noninvasively with wearable sensors, and represent different degrees of voluntary control and selectivity toward arousal. We conducted an experiment trial with 25 participants to compare the three modalities against a standard treatment (deep breathing) and a control condition (the game without biofeedback). Our results indicate that breathing-based game biofeedback is more effective in inducing relaxation during treatment than the other four groups. Participants in this group also showed greater retention of the relaxation skills (without biofeedback) during a subsequent stressor.

A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation

In this paper, we propose a conceptually novel algorithm, namely "Spatial Subspace Rotation" (2SR), that improves the robustness of remote photoplethysmography. Based on the assumption of 1) spatially redundant pixel-sensors of a camera, and 2) a well-defined skin mask, our core idea is to estimate a spatial subspace of skin-pixels and measure its temporal rotation for pulse extraction, which does not require skin-tone or pulse-related priors in contrast to existing algorithms. The proposed algorithm is thoroughly assessed on a benchmark dataset containing 54 videos, which includes challenges of various skin-tones, body-motions in complex illuminance conditions, and pulse-rate recovery after exercise. The experimental results show that given a well-defined skin mask, 2SR outperforms the popular ICA-based approach and two state-of-the-art algorithms (CHROM and PBV). When comparing the pulse frequency spectrum, 2SR improves on average the SNR of ICA by 2.22 dB, CHROM by 1.56 dB, and PBV by 1.95 dB. When comparing the instant pulse-rate, 2SR improves on average the Pearson correlation and precision of ICA by 47% and 65%, CHROM by 22% and 23%, and PBV by 21% and 39%. ANOVA confirms the significant improvement of 2SR in peak-to-peak accuracy. The proposed 2SR algorithm is very simple to use and extend, i.e., the implementation only requires a few lines MATLAB code.

Estimation of heart rate variability using a compact radiofrequency motion sensor

Physiological indices that reflect autonomic nervous activity are considered useful for monitoring peoples' health on a daily basis. A number of such indices are derived from heart rate variability, which is obtained by a radiofrequency (RF) motion sensor without making physical contact with the user's body. However, the bulkiness of RF motion sensors used in previous studies makes them unsuitable for home use. In this study, a new method to measure heart rate variability using a compact RF motion sensor that is sufficiently small to fit in a user's shirt pocket is proposed. To extract a heart rate related component from the sensor signal, an algorithm that optimizes a digital filter based on the power spectral density of the signal is proposed. The signals of the RF motion sensor were measured for 29 subjects during the resting state and their heart rate variability was estimated from the measured signals using the proposed method and a conventional method. A correlation coefficient between true heart rate and heart rate estimated from the proposed method was 0.69. Further, the experimental results showed the viability of the RF sensor for monitoring autonomic nervous activity. However, some improvements such as controlling the direction of sensing were necessary for stable measurement.

Influence of a skin status on the light interaction with dermis

We present experimental evidence that the parameters of green light remitted from a human tissue in-vivo strongly depend on skin contact status. In case when the skin is free of any contact, simultaneous recording of imaging photoplethysmogram (iPPG) and electrocardiogram revealed that contactless iPPG fails in correct estimates of the heart rate in almost half of the cases. Meanwhile, the number of successful correlations between ECG and iPPG is significantly increased when the skin is in contact with a glass plate. These observations are in line with the recently proposed model in which pulsatile arteries deform the connective-tissue components of the dermis thus resulting in temporal modulation of the capillary density interacting with slightly penetrating light.

Dynamic heart rate estimation using principal component analysis

In this paper, facial images from various video sequences are used to obtain a heart rate reading. In this study, a video camera is used to capture the facial images of eight subjects whose heart rates vary dynamically, between 81 and 153 BPM. Principal component analysis (PCA) is used to recover the blood volume pulses (BVP) which can be used for the heart rate estimation. An important consideration for accuracy of the dynamic heart rate estimation is to determine the shortest video duration that realizes it. This video duration is chosen when the six principal components (PC) are least correlated amongst them. When this is achieved, the first PC is used to obtain the heart rate. The results obtained from the proposed method are compared to the readings obtained from the Polar heart rate monitor. Experimental results show the proposed method is able to estimate the dynamic heart rate readings using less computational requirements when compared to the existing method. The mean absolute error and the standard deviation of the absolute errors between experimental readings and actual readings are 2.18 BPM and 1.71 BPM respectively.

Remote measurement of cognitive stress via heart rate variability

Remote detection of cognitive load has many powerful applications, such as measuring stress in the workplace. Cognitive tasks have an impact on breathing and heart rate variability (HRV). We show that changes in physiological parameters during cognitive stress can be captured remotely (at a distance of 3m) using a digital camera. A study (n=10) was conducted with participants at rest and under cognitive stress. A novel five band digital camera was used to capture videos of the face of the participant. Significantly higher normalized low frequency HRV components and breathing rates were measured in the stress condition when compared to the rest condition. Heart rates were not significantly different between the two conditions. We built a person-independent classifier to predict cognitive stress based on the remotely detected physiological parameters (heart rate, breathing rate and heart rate variability). The accuracy of the model was 85% (35% greater than chance).

Detection of Nocturnal Slow Wave Sleep Based on Cardiorespiratory Activity in Healthy Adults

Human slow wave sleep (SWS) during bedtime is paramount for energy conservation and memory consolidation. This study aims at automatically detecting SWS from nocturnal sleep using cardiorespiratory signals that can be acquired with unobtrusive sensors in a home-based scenario. From the signals, time-dependent features are extracted for continuous 30-s epochs. To reduce the measuring noise, body motion artifacts, and/or within-subject variability in physiology conveyed by the features, and thus, enhance the detection performance, we propose to smooth the features over each night using a spline fitting method. In addition, it was found that the changes in cardiorespiratory activity precede the transitions between SWS and the other sleep stages (non-SWS). To this matter, a novel scheme is proposed that performs the SWS detection for each epoch using the feature values prior to that epoch. Experiments were conducted with a large dataset of 325 overnight polysomnography (PSG) recordings using a linear discriminant classifier and tenfold cross validation. Features were selected with a correlation-based method. Results show that the performance in classifying SWS and non-SWS can be significantly improved when smoothing the features and using the preceding feature values of 5-min earlier. We achieved a Cohen's Kappa coefficient of 0.57 (at an accuracy of 88.8%) using only six selected features for 257 recordings with a minimum of 30-min overnight SWS that were considered representative of their habitual sleeping pattern at home. These features included the standard deviation, low-frequency spectral power, and detrended fluctuation of heartbeat intervals as well as the variations of respiratory frequency and upper and lower respiratory envelopes. A marked drop in Kappa to 0.21 was observed for the other nights with SWS time of less than 30 min, which were found to more likely occur in elderly. This will be the future challenge in cardiorespiratory-based SWS detection.

Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-Wavelength Imaging System

We present a noncontact method to monitor blood oxygen saturation (SpO2). The method uses a CMOS camera with a trigger control to allow recording of photoplethysmography (PPG) signals alternatively at two particular wavelengths, and determines the SpO2 from the measured ratios of the pulsatile to the nonpulsatile components of the PPG signals at these wavelengths. The signal-to-noise ratio (SNR) of the SpO2 value depends on the choice of the wavelengths. We found that the combination of orange (λ = 611 nm) and near infrared (λ = 880 nm) provides the best SNR for the noncontact video-based detection method. This combination is different from that used in traditional contact-based SpO 2 measurement since the PPG signal strengths and camera quantum efficiencies at these wavelengths are more amenable to SpO2 measurement using a noncontact method. We also conducted a small pilot study to validate the noncontact method over an SpO2 range of 83%-98%. This study results are consistent with those measured using a reference contact SpO2 device ( r = 0.936, ). The presented method is particularly suitable for tracking one's health and wellness at home under free-living conditions, and for those who cannot use traditional contact-based PPG devices.

Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging

Photoplethysmography (PPG) is a noninvasive optical technique for detecting microvascular blood volume changes in tissues. Its ease of use, low cost and convenience make it an attractive area of research in the biomedical and clinical communities. Nevertheless, its single spot monitoring and the need to apply a PPG sensor directly to the skin limit its practicality in situations such as perfusion mapping and healing assessments or when free movement is required. The introduction of fast digital cameras into clinical imaging monitoring and diagnosis systems, the desire to reduce the physical restrictions, and the possible new insights that might come from perfusion imaging and mapping inspired the evolution of the conventional PPG technology to imaging PPG (IPPG). IPPG is a noncontact method that can detect heart-generated pulse waves by means of peripheral blood perfusion measurements. Since its inception, IPPG has attracted significant public interest and provided opportunities to improve personal healthcare. This study presents an overview of the wide range of IPPG systems currently being introduced along with examples of their application in various physiological assessments. We believe that the widespread acceptance of IPPG is happening, and it will dramatically accelerate the promotion of this healthcare model in the near future.

Heart rate monitoring in ultra-high-field MRI using frequency information obtained from video signals of the human skin compared to electrocardiography and pulse oximetry

Videos of the human skin contain subtle color variations associated with the blood volume pulse. This remote photoplethysmography signal can be used for heart rate monitoring and represents an alternative to signals obtained from contact-based hardware. We developed an algorithm that estimates the heart rate in real-time from photoplethysmography signals and evaluate its performance in the context of ultra-high-field magnetic resonance imaging. We compare its accuracy to heart rate values estimated from electrocardiography and finger pulse oximetry triggers, obtained from MR vendor-provided hardware. For eight subjects, two experiments are conducted with the patient table outside and inside a 7 Tesla scanner. During both 5 min setups, heart rates from the algorithm and contact-based methods are stored. Their comparison suggests technical feasibility of the contactless method but that it is inferior in accuracy compared to contact-based hardware and that low heart rates (≤50 beats per minute) and adequate illumination are major challenges for practical feasibility.

Noncontact measurement of cardiac beat by using active stereo with waved-grid pattern projection

We propose a method to observe cardiac beat from 3D shape information of body surface by using the active stereo with waved-grid pattern projection, and report preliminary experiments to evaluate validities of the proposed method. By comparing results of our method with those of electrocardiogram (ECG), we confirmed sufficient correspondences between peak intervals of depth changes between contiguous frames measured by the active stereo and R-R intervals measured by ECG. We proposed the visualization of the spatial distribution of depth change plotted on the 3D shape of chest surface. We confirm that the spatial phase difference, which is caused by heart pump ability, appears in the 3-D shape change of chest surface.

Toward Ubiquitous Blood Pressure Monitoring via Pulse Transit Time: Theory and Practice

Ubiquitous blood pressure (BP) monitoring is needed to improve hypertension detection and control and is becoming feasible due to recent technological advances such as in wearable sensing. Pulse transit time (PTT) represents a well-known potential approach for ubiquitous BP monitoring. The goal of this review is to facilitate the achievement of reliable ubiquitous BP monitoring via PTT. We explain the conventional BP measurement methods and their limitations; present models to summarize the theory of the PTT-BP relationship; outline the approach while pinpointing the key challenges; overview the previous work toward putting the theory to practice; make suggestions for best practice and future research; and discuss realistic expectations for the approach.

Dynamic heart rate measurements from video sequences

This paper shows how dynamic heart rate measurements that are typically obtained from sensors mounted near to the heart can also be obtained from video sequences. In this study, two experiments are carried out where a video camera captures the facial images of the seven subjects. The first experiment involves the measurement of subjects' increasing heart rates (79 to 150 beats per minute (BPM)) while cycling whereas the second involves falling heart beats (153 to 88 BPM). In this study, independent component analysis (ICA) is combined with mutual information to ensure accuracy is not compromised in the use of short video duration. While both experiments are going on measures of heartbeat using the Polar heart rate monitor is also taken to compare with the findings of the proposed method. Overall experimental results show the proposed method can be used to measure dynamic heart rates where the root mean square error (RMSE) and the correlation coefficient are 1.88 BPM and 0.99 respectively.

Technology platforms for remote monitoring of vital signs in the new era of telemedicine

Driven by healthcare cost and home healthcare need, the development of remote monitoring technologies is poised to improve and revolutionize healthcare delivery and accessibility. This paper reviews the recent progress in the field of remote monitoring technologies that may have the potential to become the basic platforms for telemedicine. In particular, key techniques and devices for monitoring cardiorespiratory activity, blood pressure and blood glucose concentration are summarized and discussed. In addition, the US FDA approved remote vital signs monitoring devices currently available on the market are presented.

A novel method based on two cameras for accurate estimation of arterial oxygen saturation

BACKGROUND

Photoplethysmographic imaging (PPGi) that is based on camera allows acquiring photoplethysmogram and measuring physiological parameters such as pulse rate, respiration rate and perfusion level. It has also shown potential for estimation of arterial oxygen saturation (SaO2). However, there are some technical limitations such as optical shunting, different camera sensitivity to different light spectra, different AC-to-DC ratios (the peak-to-peak amplitude to baseline ratio) of the PPGi signal for different portions of the sensor surface area, the low sampling rate and the inconsistency of contact force between the fingertip and camera lens.

METHODS

In this paper, we take full account of the above-mentioned design challenges and present an accurate SaO2 estimation method based on two cameras. The hardware system we used consisted of an FPGA development board (XC6SLX150T-3FGG676 from Xilinx), with connected to it two commercial cameras and an SD card. The two cameras were placed back to back, one camera acquired PPGi signal from the right index fingertip under 660 nm light illumination while the other camera acquired PPGi signal from the thumb fingertip using an 800 nm light illumination. The both PPGi signals were captured simultaneously, recorded in a text file on the SD card and processed offline using MATLAB®. The calculation of SaO2 was based on the principle of pulse oximetry. The AC-to-DC ratio was acquired by the ratio of powers of AC and DC components of the PPGi signal in the time-frequency domain using the smoothed pseudo Wigner-Ville distribution. The calibration curve required for SaO2 measurement was obtained by linear regression analysis.

RESULTS

The results of our estimation method from 12 subjects showed a high correlation and accuracy with those of conventional pulse oximetry for the range from 90 to 100%.

CONCLUSIONS

Our method is suitable for mobile applications implemented in smartphones, which could allow SaO2 measurement in a pervasive environment.

Estimation of respiratory rate from photoplethysmographic imaging videos compared to pulse oximetry

We present a study evaluating two respiratory rate estimation algorithms using videos obtained from placing a finger on the camera lens of a mobile phone. The two algorithms, based on Smart Fusion and empirical mode decomposition (EMD), consist of previously developed signal processing methods to detect features and extract respiratory induced variations in photoplethysmographic signals to estimate respiratory rate. With custom-built software on an Android phone, photoplethysmographic imaging videos were recorded from 19 healthy adults while breathing spontaneously at respiratory rates between 6 to 32 breaths/min. Signals from two pulse oximeters were simultaneously recorded to compare the algorithms' performance using mobile phone data and clinical data. Capnometry was recorded to obtain reference respiratory rates. Two hundred seventy-two recordings were analyzed. The Smart Fusion algorithm reported 39 recordings with insufficient respiratory information from the photoplethysmographic imaging data. Of the 232 remaining recordings, a root mean square error (RMSE) of 6 breaths/min was obtained. The RMSE for the pulse oximeter data was lower at 2.3 breaths/min. RMSE for the EMD method was higher throughout all data sources as, unlike the Smart Fusion, the EMD method did not screen for inconsistent results. The study showed that it is feasible to estimate respiratory rates by placing a finger on a mobile phone camera, but that it becomes increasingly challenging at respiratory rates greater than 20 breaths/min, independent of data source or algorithm tested.

DistancePPG: Robust non-contact vital signs monitoring using a camera

Vital signs such as pulse rate and breathing rate are currently measured using contact probes. But, non-contact methods for measuring vital signs are desirable both in hospital settings (e.g. in NICU) and for ubiquitous in-situ health tracking (e.g. on mobile phone and computers with webcams). Recently, camera-based non-contact vital sign monitoring have been shown to be feasible. However, camera-based vital sign monitoring is challenging for people with darker skin tone, under low lighting conditions, and/or during movement of an individual in front of the camera. In this paper, we propose distancePPG, a new camera-based vital sign estimation algorithm which addresses these challenges. DistancePPG proposes a new method of combining skin-color change signals from different tracked regions of the face using a weighted average, where the weights depend on the blood perfusion and incident light intensity in the region, to improve the signal-to-noise ratio (SNR) of camera-based estimate. One of our key contributions is a new automatic method for determining the weights based only on the video recording of the subject. The gains in SNR of camera-based PPG estimated using distancePPG translate into reduction of the error in vital sign estimation, and thus expand the scope of camera-based vital sign monitoring to potentially challenging scenarios. Further, a dataset will be released, comprising of synchronized video recordings of face and pulse oximeter based ground truth recordings from the earlobe for people with different skin tones, under different lighting conditions and for various motion scenarios.

Individual recovery profiles in basketball players

In the sport context, recovery has been characterized as a multifactor process (physiological, psychological, behavioral, social, etc.). This study takes a multidisciplinary approach to find psychophysiological markers of the stress-recovery process. It aims to determine how athletes' specific recovery actions relate to their perceptions of recovery, and Heart Rate Variability (HRV). A total of 196 assessments were analyzed from 6 players on a men's professional basketball team within the Liga LEB Oro basketball federation (2012/2013 season). Perceptions of recovery, recovery strategies, and HRV were recorded. The results show a pattern of individual differences in behavior related to athletes' recovery actions and HRV profiles throughout the season (p < .05). Moreover, we observed that each player had different recovery needs. In light of these results, we suggest an individualistic approach to evaluating and monitoring recovery to attend more accurately to each player's recovery needs.

Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time

We present optical imaging-based methods to measure vital physiological signals, including breathing frequency (BF), exhalation flow rate, heart rate (HR), and pulse transit time (PTT). The breathing pattern tracking was based on the detection of body movement associated with breathing using a differential signal processing approach. A motion-tracking algorithm was implemented to correct random body movements that were unrelated to breathing. The heartbeat pattern was obtained from the color change in selected region of interest (ROI) near the subject's mouth, and the PTT was determined by analyzing pulse patterns at different body parts of the subject. The measured BF, exhaled volume flow rate and HR are consistent with those measured simultaneously with reference technologies (r = 0.98, for HR; r = 0.93, for breathing rate), and the measured PTT difference (30-40 ms between mouth and palm) is comparable to the results obtained with other techniques in the literature. The imaging-based methods are suitable for tracking vital physiological parameters under free-living condition and this is the first demonstration of using noncontact method to obtain PTT difference and exhalation flow rate.

A machine learning approach to improve contactless heart rate monitoring using a webcam

Unobtrusive, contactless recordings of physiological signals are very important for many health and human-computer interaction applications. Most current systems require sensors which intrusively touch the user's skin. Recent advances in contact-free physiological signals open the door to many new types of applications. This technology promises to measure heart rate (HR) and respiration using video only. The effectiveness of this technology, its limitations, and ways of overcoming them deserves particular attention. In this paper, we evaluate this technique for measuring HR in a controlled situation, in a naturalistic computer interaction session, and in an exercise situation. For comparison, HR was measured simultaneously using an electrocardiography device during all sessions. The results replicated the published results in controlled situations, but show that they cannot yet be considered as a valid measure of HR in naturalistic human-computer interaction. We propose a machine learning approach to improve the accuracy of HR detection in naturalistic measurements. The results demonstrate that the root mean squared error is reduced from 43.76 to 3.64 beats/min using the proposed method.

Ambient and Unobtrusive Cardiorespiratory Monitoring Techniques

Monitoring vital signs through unobtrusive means is a goal which has attracted a lot of attention in the past decade. This review provides a systematic and comprehensive review over the current state of the field of ambient and unobtrusive cardiorespiratory monitoring. To this end, nine different sensing modalities which have been in the focus of current research activities are covered: capacitive electrocardiography, seismo- and ballistocardiography, reflective photoplethysmography (PPG) and PPG imaging, thermography, methods relying on laser or radar for distance-based measurements, video motion analysis, as well as methods using high-frequency electromagnetic fields. Current trends in these subfields are reviewed. Moreover, we systematically analyze similarities and differences between these methods with respect to the physiological and physical effects they sense as well as the resulting implications. Finally, future research trends for the field as a whole are identified.

A compact and hand-held infection-screening system for use in rapid medical inspection at airport quarantine stations: system design and preliminary validation

To conduct mass screening and thereby reduce the spread of infection, a compact (13.5 cm × 8.5 cm × 2.5 cm), highly-mobile and hand-held infection-screening system was developed for rapid medical inspection in mass gathering places such as airports. The system is capable of non-contact vital-sign monitoring using two integrated sensors: a 24-GHz microwave radar for measuring heart and respiration rates and a thermopile array for capturing facial temperature. Subsequently, the system detects infected individuals using a linear discriminant function (LDA) from the derived vital-signs data. The system was tested on 10 subjects under two conditions (resting as normal and exercising as pseudo-infected, i.e. a 10-min bicycle ergometer at 100 W exercise); the normal and pseudo-infected conditions were classified successfully via LDA for all subjects (p < 0.01; classification error rate < 5%). The proposed non-contact system can be applied for preventing secondary exposure of medical doctors at the outbreak of highly pathogenic infectious diseases such as the Ebola virus.

Unobtrusive sensing and wearable devices for health informatics

The aging population, prevalence of chronic diseases, and outbreaks of infectious diseases are some of the major challenges of our present-day society. To address these unmet healthcare needs, especially for the early prediction and treatment of major diseases, health informatics, which deals with the acquisition, transmission, processing, storage, retrieval, and use of health information, has emerged as an active area of interdisciplinary research. In particular, acquisition of health-related information by unobtrusive sensing and wearable technologies is considered as a cornerstone in health informatics. Sensors can be weaved or integrated into clothing, accessories, and the living environment, such that health information can be acquired seamlessly and pervasively in daily living. Sensors can even be designed as stick-on electronic tattoos or directly printed onto human skin to enable long-term health monitoring. This paper aims to provide an overview of four emerging unobtrusive and wearable technologies, which are essential to the realization of pervasive health information acquisition, including: 1) unobtrusive sensing methods, 2) smart textile technology, 3) flexible-stretchable-printable electronics, and 4) sensor fusion, and then to identify some future directions of research.

Extraction of heart rate variability from smartphone photoplethysmograms

Heart rate variability (HRV) is a useful clinical tool for autonomic function assessment and cardiovascular diseases diagnosis. It is traditionally calculated from a dedicated medical electrocardiograph (ECG). In this paper, we demonstrate that HRV can also be extracted from photoplethysmograms (PPG) obtained by the camera of a smartphone. Sixteen HRV parameters, including time-domain, frequency-domain, and nonlinear parameters, were calculated from PPG captured by a smartphone for 30 healthy subjects and were compared with those derived from ECG. The statistical results showed that 14 parameters (AVNN, SDNN, CV, RMSSD, SDSD, TP, VLF, LF, HF, LF/HF, nLF, nHF, SD1, and SD2) from PPG were highly correlated (r > 0.7, P < 0.001) with those from ECG, and 7 parameters (AVNN, TP, VLF, LF, HF, nLF, and nHF) from PPG were in good agreement with those from ECG within the acceptable limits. In addition, five different algorithms to detect the characteristic points of PPG wave were also investigated: peak point (PP), valley point (VP), maximum first derivative (M1D), maximum second derivative (M2D), and tangent intersection (TI). The results showed that M2D and TI algorithms had the best performance. These results suggest that the smartphone might be used for HRV measurement.

Respiratory rate assessment from photoplethysmographic imaging

We present a study investigating the suitability of a respiratory rate estimation algorithm applied to photoplethysmographic imaging on a mobile phone. The algorithm consists of a cascade of previously developed signal processing methods to detect features and extract respiratory induced variations in photoplethysmogram signals to estimate respiratory rate. With custom-built software on an Android phone (Camera Oximeter), contact photoplethysmographic imaging videos were recorded using the integrated camera from 19 healthy adults breathing spontaneously at respiratory rates between 6 and 40 breaths/min. Capnometry was simultaneously recorded to obtain reference respiratory rates. Two hundred and ninety-eight Camera Oximeter recordings were available for analysis. The algorithm detected 22 recordings with poor photoplethysmogram quality and 46 recordings with insufficient respiratory information. Of the 232 remaining recordings, a root mean square error of 5.9 breaths/min and a median absolute error of 2.3 breaths/min was obtained. The study showed that it is feasible to estimate respiratory rates by placing a finger on a mobile phone camera, but that it becomes increasingly challenging at respiratory rates higher than 20 breaths/min.

Motion Robust Remote-PPG in Infrared

Current state-of-the-art remote photoplethysmography (rPPG) algorithms are capable of extracting a clean pulse signal in ambient light conditions using a regular color camera, even when subjects move significantly. In this study, we investigate the feasibility of rPPG in the (near)-infrared spectrum, which broadens the scope of applications for rPPG. Two camera setups are investigated: one setup consisting of three monochrome cameras with different optical filters, and one setup consisting of a single RGB camera with a visible light blocking filter. Simulation results predict the monochrome setup to be more motion robust, but this simulation neglects parallax. To verify this, a challenging benchmark dataset consisting of 30 videos is created with various motion scenarios and skin tones. Experiments show that both camera setups are capable of accurate pulse extraction in all motion scenarios, with an average SNR of +6.45 and +7.26 dB, respectively. The single camera setup proves to be superior in scenarios involving scaling, likely due to parallax of the multicamera setup. To further improve motion robustness of the RGB camera, dedicated LED illumination with two distinct wavelengths is proposed and verified. This paper demonstrates that accurate rPPG measurements in infrared are feasible, even with severe subject motion.

Non-contact estimation of heart rate and oxygen saturation using ambient light

We propose a robust method for automated computation of heart rate (HR) from digital color video recordings of the human face. In order to extract photoplethysmographic signals, two orthogonal vectors of RGB color space are used. We used a dual tree complex wavelet transform based denoising algorithm to reduce artifacts (e.g. artificial lighting, movement, etc.). Most of the previous work on skin color based HR estimation performed experiments with healthy volunteers and focused to solve motion artifacts. In addition to healthy volunteers we performed experiments with child patients in pediatric intensive care units. In order to investigate the possible factors that affect the non-contact HR monitoring in a clinical environment, we studied the relation between hemoglobin levels and HR estimation errors. Low hemoglobin causes underestimation of HR. Nevertheless, we conclude that our method can provide acceptable accuracy to estimate mean HR of patients in a clinical environment, where the measurements can be performed remotely. In addition to mean heart rate estimation, we performed experiments to estimate oxygen saturation. We observed strong correlations between our SpO2 estimations and the commercial oximeter readings.

Exploiting spatial redundancy of image sensor for motion robust rPPG

Remote photoplethysmography (rPPG) techniques can measure cardiac activity by detecting pulse-induced color variations on human skin using an RGB camera. State-of-the-art rPPG methods are sensitive to subject body motions (e.g., motion-induced color distortions). This study proposes a novel framework to improve the motion robustness of rPPG. The basic idea of this paper originates from the observation that a camera can simultaneously sample multiple skin regions in parallel, and each of them can be treated as an independent sensor for pulse measurement. The spatial redundancy of an image sensor can thus be exploited to distinguish the pulse signal from motion-induced noise. To this end, the pixel-based rPPG sensors are constructed to estimate a robust pulse signal using motion-compensated pixel-to-pixel pulse extraction, spatial pruning, and temporal filtering. The evaluation of this strategy is not based on a full clinical trial, but on 36 challenging benchmark videos consisting of subjects that differ in gender, skin types, and performed motion categories. Experimental results show that the proposed method improves the SNR of the state-of-the-art rPPG technique from 3.34 to 6.76 dB, and the agreement ( ±1.96σ) with instantaneous reference pulse rate from 55% to 80% correct. ANOVA with post hoc comparison shows that the improvement on motion robustness is significant. The rPPG method developed in this study has a performance that is very close to that of the contact-based sensor under realistic situations, while its computational efficiency allows real-time processing on an off-the-shelf computer.

Detection of atrial fibrillation using contactless facial video monitoring

BACKGROUND

It is estimated that 33.5 million people in the world have developed atrial fibrillation (AF), and an estimated 30% of patients with AF are unaware of their diagnosis (silent AF).

OBJECTIVE

The purpose of this study was to test a new technology for contactless detection of AF based on facial video recordings.

METHODS

The proposed technique uses a camera to record an individual's face and extract the subtle beat-to-beat variations of skin color reflecting the cardiac pulsatile signal. In a group of adults referred for electrical cardioversion, we recorded the ECG and the video of the subjects' face before and after electrical cardioversion. We extracted the beat-to-beat pulse rates expressed as pulses per minute (ppm) from the videoplethysmographic (VPG) signal acquired using a standard web camera. We introduce a novel quantifier of pulse variability called the pulse harmonic strength (PHS) and report its ability to detect the presence of AF.

RESULTS

Eleven subjects (8 male; age 65 ± 6 years) were included in the study. The VPG and ECG-based rates were statistically different between the AF and sinus rhythm periods: 72 ± 9 ppm vs 57 ± 7 ppm (P < .0001) for VPG and 80 ± 17 bpm vs 56 ± 7 bpm (P < .0001) for ECG signals. Among the 407 epochs of 15 seconds of synchronized ECG and VPG signals, PHS was associated with a 20% detection error rate, and the error rates of the automatic ECG-based measurements ranged between 17% and 29%.

CONCLUSION

Our preliminary results support the concept that contactless video-based monitoring of the human face for detection of abnormal pulse variability due to AF is feasible.

Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam

We introduce a new framework for detecting mental workload changes using video frames obtained from a low-cost webcam. Image processing in addition to a continuous wavelet transform filtering method were developed and applied to remove major artifacts and trends on raw webcam photoplethysmographic signals. The measurements are performed on human faces. To induce stress, we have employed a computerized and interactive Stroop color word test on a set composed by twelve participants. The electrodermal activity of the participants was recorded and compared to the mental workload curve assessed by merging two parameters derived from the pulse rate variability and photoplethysmographic amplitude fluctuations, which reflect peripheral vasoconstriction changes. The results exhibit strong correlation between the two measurement techniques. This study offers further support for the applicability of mental workload detection by remote and low-cost means, providing an alternative to conventional contact techniques.

Potential for health screening using long-term cardiovascular parameters measured by finger volume-oscillometry: pilot comparative evaluation in regular and sleep-deprived activities

We explored the potential of health screening based on the long-term measurement of cardiovascular parameters using the finger volume-oscillometric technique. An automated instrument made simultaneous measurements of key cardiovascular parameters, including blood pressure, pulse pressure, heart rate, normalized pulse volume as an index of α-adrenalin-mediated sympathetic activity, and finger arterial elasticity. These were derived from finger photo-plethysmographic signals during application of cuff pressure. To assess the feasibility of achieving a screening function, measurements were made in ten healthy volunteers during 10 days of day-to-day living (normal condition), and carried out several times at a fixed time every day. During successive 10-day measurements, a 30-hour period of total sleep deprivation was introduced as a physiological challenge (abnormal condition). A linear discriminant analysis of the data was conducted to determine whether these two conditions could be discriminated. Periodic data collection was performed rapidly and easily, and the %-correct classifications of normal and abnormal conditions were 78.2% and 77.5%, respectively. This ability of the method to discriminate between regular and sleep-deprived activities demonstrates its potential for healthcare screening during day-to-day living. Further investigations using larger age and gender groups of subjects including patients with cardiovascular diseases under real-life situations are required.

Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera

We present a new method for measuring photoplethysmogram signals remotely using ambient light and a digital camera that allows for accurate recovery of the waveform morphology (from a distance of 3 m). In particular, we show that the peak-to-peak time between the systolic peak and diastolic peak/inflection can be automatically recovered using the second-order derivative of the remotely measured waveform. We compare measurements from the face with those captured using a contact fingertip sensor and show high agreement in peak and interval timings. Furthermore, we show that results can be significantly improved using orange, green, and cyan color channels compared to the tradition red, green, and blue channel combination. The absolute error in interbeat intervals was 26 ms and the absolute error in mean systolic-diastolic peak-to-peak times was 12 ms. The mean systolic-diastolic peak-to-peak times measured using the contact sensor and the camera were highly correlated, ρ = 0.94 (p 0.001). The results were obtained with a camera frame-rate of only 30 Hz. This technology has significant potential for advancing healthcare.

Effective CPR Procedure With Real Time Evaluation and Feedback Using Smartphones

Timely cardio pulmonary resuscitation (CPR) can mean the difference between life and death. A trained person may not be available at emergency sites to give CPR. Normally, a 9-1-1 operator gives verbal instructions over the phone to a person giving CPR. In this paper, we discuss the use of smartphones to assist in administering CPR more efficiently and accurately. The two important CPR parameters are the frequency and depth of compressions. In this paper, we used smartphones to calculate these factors and to give real-time guidance to improve CPR. In addition, we used an application to measure oxygen saturation in blood. If blood oxygen saturation falls below an acceptable threshold, the person giving CPR can be asked to do mouth-to-mouth breathing. The 9-1-1 operator receives this information real time and can further guide the person giving CPR. Our experiments show accuracy >90% for compression frequency, depth, and oxygen saturation.

Improvements in remote cardiopulmonary measurement using a five band digital camera

Remote measurement of the blood volume pulse via photoplethysmography (PPG) using digital cameras and ambient light has great potential for healthcare and affective computing. However, traditional RGB cameras have limited frequency resolution. We present results of PPG measurements from a novel five band camera and show that alternate frequency bands, in particular an orange band, allowed physiological measurements much more highly correlated with an FDA approved contact PPG sensor. In a study with participants (n = 10) at rest and under stress, correlations of over 0.92 (p 0.01) were obtained for heart rate, breathing rate, and heart rate variability measurements. In addition, the remotely measured heart rate variability spectrograms closely matched those from the contact approach. The best results were obtained using a combination of cyan, green, and orange (CGO) bands; incorporating red and blue channel observations did not improve performance. In short, RGB is not optimal for this problem: CGO is better. Incorporating alternative color channel sensors should not increase the cost of such cameras dramatically.