Motion Robust Remote-PPG in Infrared

Remote Vital Sensing in Clinical Veterinary Medicine: A Comprehensive Review of Recent Advances, Accomplishments, Challenges, and Future Perspectives

Remote vital sensing in veterinary medicine is a relatively new area of practice, which involves the acquisition of data without invasion of the body cavities of live animals. This paper aims to review several technologies in remote vital sensing: infrared thermography, remote photoplethysmography (rPPG), radar, wearable sensors, and computer vision and machine learning. In each of these technologies, we outline its concepts, uses, strengths, and limitations in multiple animal species, and its potential to reshape health surveillance, welfare evaluation, and clinical medicine in animals. The review also provides information about the problems associated with applying these technologies, including species differences, external conditions, and the question of the reliability and classification of these technologies. Additional topics discussed in this review include future developments such as the use of artificial intelligence, combining different sensing methods, and creating monitoring solutions tailored to specific animal species. This contribution gives a clear understanding of the status and future possibilities of remote vital sensing in veterinary applications and stresses the importance of that technology for the development of the veterinary field in terms of animal health and science.

Are Activity Wrist-Worn Devices Accurate for Determining Heart Rate during Intense Exercise?

The market for wrist-worn devices is growing at previously unheard-of speeds. A consequence of their fast commercialization is a lack of adequate studies testing their accuracy on varied populations and pursuits. To provide an understanding of wearable sensors for sports medicine, the present study examined heart rate (HR) measurements of four popular wrist-worn devices, the (Fitbit Charge (FB), Apple Watch (AW), Tomtom runner Cardio (TT), and Samsung G2 (G2)), and compared them with gold standard measurements derived by continuous electrocardiogram examination (ECG). Eight athletes participated in a comparative study undergoing maximal stress testing on a cycle ergometer or a treadmill. We analyzed 1,286 simultaneous HR data pairs between the tested devices and the ECG. The four devices were reasonably accurate at the lowest activity level. However, at higher levels of exercise intensity the FB and G2 tended to underestimate HR values during intense physical effort, while the TT and AW devices were fairly reliable. Our results suggest that HR estimations should be considered cautiously at specific intensities. Indeed, an effective intervention is required to register accurate HR readings at high-intensity levels (above 150 bpm). It is important to consider that even though none of these devices are certified or sold as medical or safety devices, researchers must nonetheless evaluate wrist-worn wearable technology in order to fully understand how HR affects psychological and physical health, especially under conditions of more intense exercise.

Contactless Camera-Based Sleep Staging: The HealthBed Study

Polysomnography (PSG) remains the gold standard for sleep monitoring but is obtrusive in nature. Advances in camera sensor technology and data analysis techniques enable contactless monitoring of heart rate variability (HRV). In turn, this may allow remote assessment of sleep stages, as different HRV metrics indirectly reflect the expression of sleep stages. We evaluated a camera-based remote photoplethysmography (PPG) setup to perform automated classification of sleep stages in near darkness. Based on the contactless measurement of pulse rate variability, we use a previously developed HRV-based algorithm for 3 and 4-class sleep stage classification. Performance was evaluated on data of 46 healthy participants obtained from simultaneous overnight recording of PSG and camera-based remote PPG. To validate the results and for benchmarking purposes, the same algorithm was used to classify sleep stages based on the corresponding ECG data. Compared to manually scored PSG, the remote PPG-based algorithm achieved moderate agreement on both 3 class (Wake-N1/N2/N3-REM) and 4 class (Wake-N1/N2-N3-REM) classification, with average κ of 0.58 and 0.49 and accuracy of 81% and 68%, respectively. This is in range with other performance metrics reported on sensing technologies for wearable sleep staging, showing the potential of video-based non-contact sleep staging.

ECG-COVID: An end-to-end deep model based on electrocardiogram for COVID-19 detection

The early and accurate detection of COVID-19 is vital nowadays to avoid the vast and rapid spread of this virus and ease lockdown restrictions. As a result, researchers developed methods to diagnose COVID-19. However, these methods have several limitations. Therefore, presenting new methods is essential to improve the diagnosis of COVID-19. Recently, investigation of the electrocardiogram (ECG) signals becoming an easy way to detect COVID-19 since the ECG process is non-invasive and easy to use. Therefore, we proposed in this paper a novel end-to-end deep learning model (ECG-COVID) based on ECG for COVID-19 detection. We employed several deep Convolutional Neural Networks (CNNs) on a dataset of 1109 ECG images, which is built for screening the perception of COVID-19 and cardiac patients. After that, we selected the most efficient model as our model for evaluation. The proposed model is end-to-end where the input ECG images are fed directly to the model for the final decision without using any additional stages. The proposed method achieved an average accuracy of 98.81%, Precision of 98.8%, Sensitivity of 98.8% and, F1-score of 98.81% for COVID-19 detection. As cases of corona continue to rise and hospitalizations continue again, hospitals may find our study helpful when dealing with these patients who did not get significantly worse.

Contactless SpO2 with an RGB camera: experimental proof of calibrated SpO2

Camera-based blood oxygen saturation (SpO2) monitoring allows reliable measurements without touching the skin and is therefore very attractive when there is a risk of cross-infection, in case of fragile skin, and/or to improve the clinical workflow. Despite promising results, productization of the technology is hampered by the unavailability of adequate hardware, especially a camera, which can capture the optimal wavelengths for SpO2 measurements in the red near-infrared region. A regular color (RGB) camera is attractive because of its availability, but also poses several risks and challenges which affect the accuracy of the measurement. To mitigate the most important risks, we propose to add low-cost commercial off-the-shelf (COTS) components to the setup. We executed two studies with this setup: one at a hypoxia lab with SpO2 values in the range 70 - 100% with the purpose to determine the calibration model, and the other study on volunteers to investigate the accuracy for different spot-check scenarios. The proposed processing pipeline includes face tracking and a robust method to estimate the ratio of relative amplitudes of the photoplethysmographic waveforms. Results show that the error is smaller than 4 percent points for realistic screening scenarios where the subject is seated, either with or without head support and/or ambient light.

Blood Pressure Measurement: From Cuff-Based to Contactless Monitoring

Blood pressure (BP) determines whether a person has hypertension and offers implications as to whether he or she could be affected by cardiovascular disease. Cuff-based sphygmomanometers have traditionally provided both accuracy and reliability, but they require bulky equipment and relevant skills to obtain precise measurements. BP measurement from photoplethysmography (PPG) signals has become a promising alternative for convenient and unobtrusive BP monitoring. Moreover, the recent developments in remote photoplethysmography (rPPG) algorithms have enabled new innovations for contactless BP measurement. This paper illustrates the evolution of BP measurement techniques from the biophysical theory, through the development of contact-based BP measurement from PPG signals, and to the modern innovations of contactless BP measurement from rPPG signals. We consolidate knowledge from a diverse background of academic research to highlight the importance of multi-feature analysis for improving measurement accuracy. We conclude with the ongoing challenges, opportunities, and possible future directions in this emerging field of research.

Contactless Heartbeat Measurement Using Speckle Vibrometry

Monitoring of heart rate in patients in the general ward is necessary to assess the clinical situation of the patient. Currently, this is done via spot-checks on pulse rate manually or on heart rate using Electrocardiogram (ECG) by nurses. More frequent measurements would allow early detection of adverse cardiac events. In this work, we investigate a contactless measurement setup combined with a signal processing pipeline, which is based on speckle vibrometry (SV), to perform contactless heart rate monitoring of human subjects in a supine position, mimicking a resting scenario in the general ward. Our results demonstrate the feasibility of extracting heart rate with SV through varying textile thicknesses (i.e., 8 mm, 32 mm and 64 mm), with an error smaller than 3 beats per minute on average compared to the ground-truth heart rate derived from ECG.

Contactless Vital Signs Acquisition Using Video Photoplethysmography, Motion Analysis and Passive Infrared Thermography Devices During Emergency Department Walk-In Triage in Pandemic Conditions

BACKGROUND

Contactless vital signs (VS) measurement with video photoplethysmography (vPPG), motion analysis (MA), and passive infrared thermometry (pIR) has shown promise.

OBJECTIVES

To compare conventional (contact-based) and experimental contactless VS measurement approaches for emergency department (ED) walk-in triage in pandemic conditions.

METHODS

Patients' heart rates (HR), respiratory rates (RR), and temperatures were measured with cardiorespiratory monitor and vPPG, manual count and MA, and contact thermometers and pIR, respectively.

RESULTS

There were 475 walk-in ED patients studied (95% of eligible). Subjects were 35.2 ± 20.8 years old (range 4 days‒95 years); 52% female, 0.2% transgender; had Fitzpatrick skin type of 2.3 ± 1.4 (range 1‒6), Emergency Severity Index of 3.0 ± 0.6 (range 2‒5), and contact temperature of 36.83°C (range 35.89-39.4°C) (98.3°F [96.6‒103°F]). Pediatric HR and RR data were excluded from analysis due to research challenges associated with pandemic workflow. For a 30-s, unprimed "Triage" window in 377 adult patients, vPPG-MA acquired 377 (100%) HR measurements featuring a mean difference with cardiorespiratory monitor HR of 5.9 ± 12.8 beats/min (R = 0.6833) and 252 (66.8%) RR measurements featuring a mean difference with manual RR of -0.4 ± 2.6 beats/min (R = 0.8128). Subjects' Emergency Severity Index components based on conventional VS and contactless VS matched for 83.8% (HR) and 89.3% (RR). Filtering out vPPG-MA measurements with low algorithmic confidence reduced VS acquired while improving correlation with conventional measurements. The mean difference between contact and pIR temperatures was 0.83 ± 0.67°C (range -1.16-3.5°C) (1.5 ± 1.2°F [range -2.1-6.3°F]); pIR fever detection improved with post hoc adjustment for mean bias.

CONCLUSION

Contactless VS acquisition demonstrated good agreement with contact methods during adult walk-in ED patient triage in pandemic conditions; clinical applications will need further study.

Mobile Robotic Platform for Contactless Vital Sign Monitoring

The COVID-19 pandemic has accelerated methods to facilitate contactless evaluation of patients in hospital settings. By minimizing in-person contact with individuals who may have COVID-19, healthcare workers can prevent disease transmission and conserve personal protective equipment. Obtaining vital signs is a ubiquitous task that is commonly done in person by healthcare workers. To eliminate the need for in-person contact for vital sign measurement in the hospital setting, we developed Dr. Spot, a mobile quadruped robotic system. The system includes IR and RGB cameras for vital sign monitoring and a tablet computer for face-to-face medical interviewing. Dr. Spot is teleoperated by trained clinical staff to simultaneously measure the skin temperature, respiratory rate, and heart rate while maintaining social distancing from patients and without removing their mask. To enable accurate, contactless measurements on a mobile system without a static black body as reference, we propose novel methods for skin temperature compensation and respiratory rate measurement at various distances between the subject and the cameras, up to 5 m. Without compensation, the skin temperature MAE is 1.3°C. Using the proposed compensation method, the skin temperature MAE is reduced to 0.3°C. The respiratory rate method can provide continuous monitoring with a MAE of 1.6 BPM in 30 s or rapid screening with a MAE of 2.1 BPM in 10 s. For the heart rate estimation, our system is able to achieve a MAE less than 8 BPM in 10 s measured in arbitrary indoor light conditions at any distance below 2 m.

Skin Pigmentation Influence on Pulse Oximetry Accuracy: A Systematic Review and Bibliometric Analysis

Nowadays, pulse oximetry has become the standard in primary and intensive care units, especially as a triage tool during the current COVID-19 pandemic. Hence, a deeper understanding of the measurement errors that can affect precise readings is a key element in clinical decision-making. Several factors may influence the accuracy of pulse oximetry, such as skin color, body temperature, altitude, or patient movement. The skin pigmentation effect on pulse oximetry accuracy has long been studied reporting some contradictory conclusions. Recent studies have shown a positive bias in oxygen saturation measurements in patients with darkly pigmented skin, particularly under low saturation conditions. This review aims to study the literature that assesses the influence of skin pigmentation on the accuracy of these devices. We employed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement to conduct a systematic review retrospectively since February 2022 using WOS, PubMed, and Scopus databases. We found 99 unique references, of which only 41 satisfied the established inclusion criteria. A bibliometric and scientometrics approach was performed to examine the outcomes of an exhaustive survey of the thematic content and trending topics.

Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera

We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS.

Using High-Fidelity Avatars to Advance Camera-based Cardiac Pulse Measurement

Non-contact physiological measurement has the potential to provide low-cost, non-invasive health monitoring. However, machine vision approaches are often limited by the availability and diversity of annotated video datasets resulting in poor generalization to complex real-life conditions. To address these challenges, this work proposes the use of synthetic avatars that display facial blood flow changes and allow for systematic generation of samples under a wide variety of conditions. Our results show that training on both simulated and real video data can lead to performance gains under challenging conditions. We show strong performance on three large benchmark datasets and improved robustness to skin type and motion. These results highlight the promise of synthetic data for training camera-based pulse measurement; however, further research and validation is needed to establish whether synthetic data alone could be sufficient for training models.

Contactless facial video recording with deep learning models for the detection of atrial fibrillation

Atrial fibrillation (AF) is often asymptomatic and paroxysmal. Screening and monitoring are needed especially for people at high risk. This study sought to use camera-based remote photoplethysmography (rPPG) with a deep convolutional neural network (DCNN) learning model for AF detection. All participants were classified into groups of AF, normal sinus rhythm (NSR) and other abnormality based on 12-lead ECG. They then underwent facial video recording for 10 min with rPPG signals extracted and segmented into 30-s clips as inputs of the training of DCNN models. Using voting algorithm, the participant would be predicted as AF if > 50% of their rPPG segments were determined as AF rhythm by the model. Of the 453 participants (mean age, 69.3 ± 13.0 years, women, 46%), a total of 7320 segments (1969 AF, 1604 NSR & 3747others) were analyzed by DCNN models. The accuracy rate of rPPG with deep learning model for discriminating AF from NSR and other abnormalities was 90.0% and 97.1% in 30-s and 10-min recording, respectively. This contactless, camera-based rPPG technique with a deep-learning model achieved significantly high accuracy to discriminate AF from non-AF and may enable a feasible way for a large-scale screening or monitoring in the future.

Evaluation of Remote Photoplethysmography Measurement Conditions toward Telemedicine Applications

Camera-based remote photoplethysmography (rPPG) is a low-cost and casual non-contact heart rate measurement method suitable for telemedicine. Several factors affect the accuracy of measuring the heart rate and heart rate variability (HRV) using rPPG despite HRV being an important indicator for healthcare monitoring. This study aimed to investigate the appropriate setup for precise HRV measurements using rPPG while considering the effects of possible factors including illumination, direction of the light, frame rate of the camera, and body motion. In the lighting conditions experiment, the smallest mean absolute R–R interval (RRI) error was obtained when light greater than 500 lux was cast from the front (among the following conditions—illuminance: 100, 300, 500, and 700 lux; directions: front, top, and front and top). In addition, the RRI and HRV were measured with sufficient accuracy at frame rates above 30 fps. The accuracy of the HRV measurement was greatly reduced when the body motion was not constrained; thus, it is necessary to limit the body motion, especially the head motion, in an actual telemedicine situation. The results of this study can act as guidelines for setting up the shooting environment and camera settings for rPPG use in telemedicine.

Camera-based heart rate estimation for hospitalized newborns in the presence of motion artifacts

Background

Heart rate (HR) is an important vital sign for evaluating the physiological condition of a newborn infant. Recently, for measuring HR, novel RGB camera-based non-contact techniques have demonstrated their specific superiority compared with other techniques, such as dopplers and thermal cameras. However, they still suffered poor robustness in infants’ HR measurements due to frequent body movement.

Methods

This paper introduces a framework to improve the robustness of infants’ HR measurements by solving motion artifact problems. Our solution is based on the following steps: morphology-based filtering, region-of-interest (ROI) dividing, Eulerian video magnification and majority voting. In particular, ROI dividing improves ROI information utilization. The majority voting scheme improves the statistical robustness by choosing the HR with the highest probability. Additionally, we determined the dividing parameter that leads to the most accurate HR measurements. In order to examine the performance of the proposed method, we collected 4 hours of videos and recorded the corresponding electrocardiogram (ECG) of 9 hospitalized neonates under two different conditions—rest still and visible movements.

Results

Experimental results indicate a promising performance: the mean absolute error during rest still and visible movements are 3.39 beats per minute (BPM) and 4.34 BPM, respectively, which improves at least 2.00 and 1.88 BPM compared with previous works. The Bland-Altman plots also show the remarkable consistency of our results and the HR derived from the ground-truth ECG.

Conclusions

To the best of our knowledge, this is the first study aimed at improving the robustness of neonatal HR measurement under motion artifacts using an RGB camera. The preliminary results have shown the promising prospects of the proposed method, which hopefully reduce neonatal mortality in hospitals.

Non-Contact Measurement of Empathy Based on Micro-Movement Synchronization

Tracking consumer empathy is one of the biggest challenges for advertisers. Although numerous studies have shown that consumers’ empathy affects purchasing, there are few quantitative and unobtrusive methods for assessing whether the viewer is sharing congruent emotions with the advertisement. This study suggested a non-contact method for measuring empathy by evaluating the synchronization of micro-movements between consumers and people within the media. Thirty participants viewed 24 advertisements classified as either empathy or non-empathy advertisements. For each viewing, we recorded the facial data and subjective empathy scores. We recorded the facial micro-movements, which reflect the ballistocardiography (BCG) motion, through the carotid artery remotely using a camera without any sensory attachment to the participant. Synchronization in cardiovascular measures (e.g., heart rate) is known to indicate higher levels of empathy. We found that through cross-entropy analysis, the more similar the micro-movements between the participant and the person in the advertisement, the higher the participant’s empathy scores for the advertisement. The study suggests that non-contact BCG methods can be utilized in cases where sensor attachment is ineffective (e.g., measuring empathy between the viewer and the media content) and can be a complementary method to subjective empathy scales.

Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera

This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.

Oxygen Saturation Behavior by Pulse Oximetry in Female Athletes: Breaking Myths

The myths surrounding women’s participation in sport have been reflected in respiratory physiology. This study aims to demonstrate that continuous monitoring of blood oxygen saturation during a maximal exercise test in female athletes is highly correlated with the determination of the second ventilatory threshold (VT2) or anaerobic threshold (AnT). The measurements were performed using a pulse oximeter during a maximum effort test on a treadmill on a population of 27 healthy female athletes. A common behavior of the oxygen saturation evolution during the incremental exercise test characterized by a decrease in saturation before the aerobic threshold (AeT) followed by a second significant drop was observed. Decreases in peripheral oxygen saturation during physical exertion have been related to the athlete’s physical fitness condition. However, this drop should not be a limiting factor in women’s physical performance. We found statistically significant correlations between the maximum oxygen uptake and the appearance of the ventilatory thresholds (VT1 and VT2), the desaturation time, the total test time, and between the desaturation time and the VT2. We observed a relationship between the desaturation time and the VT2 appearance. Indeed, a linear regression model between the desaturation time and the VT2 appearance can predict 80% of the values in our sample. Besides, we suggest that pulse oximetry is a simple, fairly accurate, and non-invasive technique for studying the physical condition of athletes who perform physical exertion.

Non-Contact Heart Rate Estimation via Adaptive RGB/NIR Signal Fusion

We propose a non-contact heart rate (HR) estimation method that is robust to various situations, such as bright, low-light, and varying illumination scenes. We utilize a camera that records red, green, and blue (RGB) and near-infrared (NIR) information to capture the subtle skin color changes induced by the cardiac pulse of a person. The key novelty of our method is the adaptive fusion of RGB and NIR signals for HR estimation based on the analysis of background illumination variations. RGB signals are suitable indicators for HR estimation in bright scenes. Conversely, NIR signals are more reliable than RGB signals in scenes with more complex illumination, as they can be captured independently of the changes in background illumination. By measuring the correlations between the lights reflected from the background and facial regions, we adaptively utilize RGB and NIR observations for HR estimation. The experiments demonstrate the effectiveness of the proposed method.

Noncontact Monitoring of Heart Rate and Heart Rate Variability in Geriatric Patients Using Photoplethysmography Imaging

OBJECTIVE

Geriatric patients, especially those with dementia or in a delirious state, do not accept conventional contact-based monitoring. Therefore, we propose to measure heart rate (HR) and heart rate variability (HRV) of geriatric patients in a noncontact and unobtrusive way using photoplethysmography imaging (PPGI).

METHODS

PPGI video sequences were recorded from 10 geriatric patients and 10 healthy elderly people using a monochrome camera operating in the near-infrared spectrum and a colour camera operating in the visible spectrum. PPGI waveforms were extracted from both cameras using superpixel-based regions of interests (ROI). A classifier based on bagged trees was trained to automatically select artefact-free ROIs for HR estimation. HRV was calculated in the time-domain and frequency-domain.

RESULTS

an RMSE of 1.03 bpm and a correlation of 0.8 with the reference was achieved using the NIR camera for HR estimation. Using the RGB camera, RMSE and correlation improved to 0.48 bpm and 0.95, respectively. Correlation for HRV in the frequency-domain (LF/HF-ratio) was 0.50 using the NIR camera and 0.70 using the RGB camera.

CONCLUSION

We were able to demonstrate that PPGI is very suitable to measure HR and HRV in geriatric patients. We strongly believe that PPGI will become clinically relevant in monitoring of geriatric patients.

SIGNIFICANCE

we are the first group to measure both HR and HRV in awake geriatric patients using PPGI. Moreover, we systematically evaluate the effects of the spectrum (near-infrared vs. visible), ROI, and additional motion artefact reduction algorithms on the accuracy of estimated HR and HRV.

Camera-Based Vital Signs Monitoring During Sleep - A Proof of Concept Study

Polysomnography (PSG) is the current gold standard for the diagnosis of sleep disorders. However, this multi-parametric sleep monitoring tool also has some drawbacks, e.g. it limits the patient's mobility during the night and it requires the patient to come to a specialized sleep clinic or hospital to attach the sensors. Unobtrusive techniques for the detection of sleep disorders such as sleep apnea are therefore gaining increasing interest. Remote photoplethysmography using video is a technique which enables contactless detection of hemodynamic information. Promising results in near-infrared have been reported for the monitoring of sleep-relevant physiological parameters pulse rate, respiration and blood oxygen saturation. In this study we validate a contactless monitoring system on eight patients with a high likelihood of relevant obstructive sleep apnea, which are enrolled for a sleep study at a specialized sleep center. The dataset includes 46.5 hours of video recordings, full polysomnography and metadata. The camera can detect pulse and respiratory rate within 2 beats/breaths per minute accuracy 92% and 91% of the time, respectively. Estimated blood oxygen values are within 4 percentage points of the finger-oximeter 89% of the time. These results demonstrate the potential of a camera as a convenient diagnostic tool for sleep apnea, and sleep disorders in general.

Smart automated heart health monitoring using photoplethysmography signal classification

This paper proposes a smart, automated heart health-monitoring (SAHM) device using a single photoplethysmography (PPG) sensor that can monitor cardiac health. The SAHM uses an Orthogonal Matching Pursuit (OMP)-based classifier along with low-rank motion artifact removal as a pre-processing stage. Major contributions of the proposed SAHM device over existing state-of-the-art technologies include these factors: (i) the detection algorithm works with robust features extracted from a single PPG sensor; (ii) the motion compensation algorithm for the PPG signal can make the device wearable; and (iii) the real-time analysis of PPG input and sharing through the Internet. The proposed low-cost, compact and user-friendly PPG device can also be prototyped easily. The SAHM system was tested on three different datasets, and detailed performance analysis was carried out to show and prove the efficiency of the proposed algorithm.

Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia

Introduction

Technological advances have allowed for the estimation of physiological indicators from video data. FaceReader™ is an automated facial analysis software that has been used widely in studies of facial expressions of emotion and was recently updated to allow for the estimation of heart rate (HR) using remote photoplethysmography (rPPG). We investigated FaceReader™-based heart rate and pain expression estimations in older adults in relation to manual coding by experts.

Methods

Using a video dataset of older adult patients with and without dementia, we assessed the relationship between FaceReader’s™ HR estimations against a well-established Video Magnification (VM) algorithm during baseline and pain conditions. Furthermore, we examined the correspondence between the Facial Action Coding System (FACS)-based pain scores obtained through FaceReader™ and manual coding.

Results

FaceReader’s™ HR estimations were correlated with VM algorithm in baseline and pain conditions. Non-verbal FaceReader™ pain scores and manual coding were also highly correlated despite discrepancies between the FaceReader™ and manual coding in the absolute value of scores based on pain-related facial action coding of the events preceding and following the pain response.

Conclusions

Compared to expert manual FACS coding and optimized VM algorithm, FaceReader™ showed good results in estimating HR values and non-verbal pain scores.

Identification and Tracking of Physiological Parameters from Skin using Video Photoplethysmography

In recent years, there has been a growing interest in video Photoplethysmography (vPPG), a technique able to estimate cardiovascular parameters from video recordings of the skin. Despite the growing interest in vPPG technology, there are still problems in extracting the correct waveform of blood volume pulse, mainly due to real world artifacts, such as changes in light condition and movement artifacts. Another important issue is the correct definition of skin against background. Therefore, we propose an algorithm of skin detection that is able to recognize skin pixels solid to variations of luminosity. We recorded the signals of interest during an experimental protocol designed to provide thermal stimulation and observe the resulting Autonomic Nervous System changes. Experimental data were gathered from 10 young healthy subjects (age: 21±2 years). Video recordings are processed using a band-pass filter and then an automatic algorithm of peak detection is applied to detect the pulse wave peaks, then used to estimate heart rate variability (HRV). The efficiency and stability of the algorithm are compared against finger-PPG waveforms. Preliminary results show an overall statistical agreement between time and frequency domain indexes. However, further efforts are required to improve the estimation of frequency components, particularly during rest.

Continuous-Spectrum Infrared Illuminator for Camera-PPG in Darkness

Many camera-based remote photoplethysmography (PPG) applications require sensing in near infrared (NIR). The performance of PPG systems benefits from multi-wavelength processing. The illumination source in such system is explored in this paper. We demonstrate that multiple narrow-band LEDs have inferior color homogeneity compared to broadband light sources. Therefore, we consider the broadband option based on phosphor material excited by LEDs. A first prototype was realized and its details are discussed. It was tested within a remote-PPG monitoring scenario in darkness and the full system demonstrates robust pulse-rate measurement. Given its accuracy in pulse rate extraction, the proposed illumination principle is considered a valuable asset for large-scale NIR-PPG applications as it enables multi-wavelength processing, lightweight set-ups with relatively low-power infrared light sources.

Modified RGB Cameras for Infrared Remote-PPG

Multi-wavelength cameras play an essential role in remote photoplethysmography (PPG). Whereas these are readily available for visible light, this is not the case for near infrared (NIR). We propose to modify existing RGB cameras to make them suited for NIR-PPG. In particular, we exploit the spectral leakage of the RGB channels in infrared in combination with a narrow dual-band optical filter. Such camera modification is simple, cost-effective, easy to implement, and it is shown to attain a pulse-rate extraction performance comparable to that of multiple narrow-band NIR cameras.

Using a Motion Capture System as Reference for Motion Tracking in Photoplethysmography Imaging

Photoplethysmography Imaging (PPGI) is a camera-based and non-contact technology for measurement of physiological signals. It has been shown that important physiological parameters such as heart rate, heart rate variability and respiratory rate can be derived from PPGI. However, as is the case with most non-contact measurement techniques, motion artefacts present a major challenge. Various algorithms for application to both the 2D PPGI video frames as well as the resulting 1D PPGI waveforms have been developed in order to enhance robustness against motion. In this paper, we focus on the aspect of feature point tracking in the 2D PPGI video sequences. We present an experimental setup, where we used a motion capture system in order to obtain a reference for motion during the recording of PPGI video sequences. In a laboratory experiment, PPGI video sequences were recorded from ten healthy volunteers, who were asked to perform various movements during the recording. The KLT tracking algorithm was applied to the recorded sequences and results compared with the reference values from the motion capture system. The results indicate, that tracking of measurement regions in PPGI video sequences is only one element towards motion robust PPGI. In most scenarios, tracking is not sufficiently precise, requiring further processing of the PPGI waveforms in order to reduce motion artefacts in PPGI signals. These indications were confirmed by further analysis when we looked into the effects of tracking on PPGI heart rate extraction.

Feasibility of assessing ultra-short-term pulse rate variability from video recordings

Objectives

Remote photoplethysmography (rPPG) is a promising non-contact measurement technique for assessing numerous physiological parameters: pulse rate, pulse rate variability (PRV), respiratory rate, pulse wave velocity, blood saturation, blood pressure, etc. To justify its use in ultra-short-term (UST) PRV analysis, which is of great benefit for several healthcare applications, the agreement between rPPG- and PPG-derived UST-PRV metrics was studied.

Approach

Three time-domain metrics—standard deviation of normal-to-normal (NN) intervals (SDNN), root mean square of successive NN interval differences (RMSSD), and the percentage of adjacent NN intervals that differ from each other by more than 50 ms (pNN50)—were extracted from 56 video recordings in a publicly available data set. The selected metrics were calculated on the basis of three groups of 10 s recordings and their average, two groups of 30 s recordings and their average, and a group of 60 s recordings taken from the full-length recordings and then compared with metrics derived from the corresponding reference (PPG) pulse waveform signals by using correlation and effect size parameters, and Bland–Altman plots.

Main results

The results show there is stronger agreement as the recording length increases for SDNN and RMSSD, yet there is no significant change for pNN50. The agreement parameters reach r = 0.841 (p < 0.001), r = 0.529 (p < 0.001), and r = 0.657 (p < 0.001), estimated median bias −1.52, −2.28 ms and −1.95% and a small effect size for SDNN, RMSSD, and pNN50 derived from the 60 s recordings, respectively.

Significance

Remote photoplethysmography-derived UST-PRV metrics manage to capture UST-PRV metrics derived from reference (PPG) recordings well. This feature is highly desirable in numerous applications for the assessment of one’s health and well-being. In future research, the validity of rPPG-derived UST-PRV metrics compared to the gold standard electrocardiography recordings is to be assessed.

Remote heart rate monitoring - Assessment of the Facereader rPPg by Noldus

Remote photoplethysmography (rPPG) allows contactless monitoring of human cardiac activity through a video camera. In this study, we assessed the accuracy and precision for heart rate measurements of the only consumer product available on the market, namely the FacereaderTM rPPG by Noldus, with respect to a gold standard electrocardiograph. Twenty-four healthy participants were asked to sit in front of a computer screen and alternate two periods of rest with two stress tests (i.e. Go/No-Go task), while their heart rate was simultaneously acquired for 20 minutes using the ECG criterion measure and the FacereaderTM rPPG. Results show that the FacereaderTM rPPG tends to overestimate lower heart rates and underestimate higher heart rates compared to the ECG. The Facereader™ rPPG revealed a mean bias of 9.8 bpm, the 95% limits of agreement (LoA) ranged from almost -30 up to +50 bpm. These results suggest that whilst the rPPG FacereaderTM technology has potential for contactless heart rate monitoring, its predictions are inaccurate for higher heart rates, with unacceptable precision across the entire range, rendering its estimates unreliable for monitoring individuals.

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation

Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HRVIPL-HR is available at: ), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the state-of-the-art methods on both the public-domain and our VIPL-HR databases.

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Remote pulse rate measurement from facial video has gained particular attention over the last few years. Research exhibits significant advancements and demonstrates that common video cameras correspond to reliable devices that can be employed to measure a large set of biomedical parameters without any contact with the subject. A new framework for measuring and mapping pulse rate from video is presented in this pilot study. The method, which relies on convolutional 3D networks, is fully automatic and does not require any special image preprocessing. In addition, the network ensures concurrent mapping by producing a prediction for each local group of pixels. A particular training procedure that employs only synthetic data is proposed. Preliminary results demonstrate that this convolutional 3D network can effectively extract pulse rate from video without the need for any processing of frames. The trained model was compared with other state-of-the-art methods on public data. Results exhibit significant agreement between estimated and ground-truth measurements: the root mean square error computed from pulse rate values assessed with the convolutional 3D network is equal to 8.64 bpm, which is superior to 10 bpm for the other state-of-the-art methods. The robustness of the method to natural motion and increases in performance correspond to the two main avenues that will be considered in future works.

Discriminative Signatures for Remote-PPG

Near-infrared (NIR) remote photoplethysmography (PPG) promises attractive applications in darkness, as it involves unobtrusive, invisible light. However, since the PPG strength (AC/DC) is much lower in the NIR spectrum than in the RGB spectrum, robust vital signs monitoring is more challenging. In this paper, we propose a new PPG-extraction method, DIScriminative signature based extraction (DIS), to significantly improve the pulse-rate measurement in NIR. Our core idea is to use both the color signals containing blood absorption variations and additional disturbance signals as input for PPG extraction. By defining a discriminative signature, we use one-step least-squares regression (joint optimization) to retrieve the pulsatile component from color signals and suppress disturbance signals simultaneously. A large-scale lab experiment, recorded in NIR with heavy body motions, shows the significant improvement of DIS over the state-of-the-art method, whereas its principle is simple and generally applicable.

Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review

OBJECTIVE

Over the last 15 years, developments in camera technology have coincided with increased availability and affordability. This has led to an increasing interest in using these technologies in healthcare settings. Image-based monitoring methods potentially allow multiple vital signs to be measured concurrently using a non-contact sensor. We have undertaken a systematic review of the current availability and performance of these monitoring methods.

APPROACH

A multiple database search was conducted using MEDLINE, Embase, CINAHL, Cochrane Library, OpenGrey, IEEE Xplore Library and ACM Digital Library to July 2018. We included studies comparing image-based heart rate, respiratory rate, oxygen saturation and blood pressure monitoring methods against one or more validated reference device(s). Each included study was assessed using the modified GRRAS criteria for reporting bias.

MAIN RESULTS

Of 30 279 identified studies, 161 were included in the final analysis. Twenty studies (20/161, 12%) were carried out on patients in clinical settings, while the remainder were conducted in academic settings using healthy volunteer populations. The 18-40 age group was best represented across the identified studies. One hundred and twenty studies (120/161, 75%) estimated heart rate, followed by 62 studies (62/161, 39%) estimating respiratory rate. Fewer studies focused on oxygen saturation (11/161, 7%) or blood pressure (6/161, 4%) estimation. Fifty-one heart rate studies (51/120, 43%) and 24 respiratory rate studies (24/62, 39%) used Bland-Altman analysis to report their results. Of the heart rate studies, 28 studies (28/51, 55%) showed agreement within industry standards of [Formula: see text]5 beats per minute. Only two studies achieved this within clinical settings. Of the respiratory rate studies, 13 studies (13/24, 54%) showed agreement within industry standards of [Formula: see text]3 breaths per minute, but only one study achieved this in a clinical setting. Statistical analysis was heterogeneous across studies with frequent inappropriate use of correlation. The majority of studies (99/161, 61%) monitored subjects for under 5 min. Three studies (3/161, 2%) monitored subjects for over 60 min, all of which were conducted in hospital settings.

SIGNIFICANCE

Heart rate and respiratory rate monitoring using video images is currently possible and performs within clinically acceptable limits under experimental conditions. Camera-derived estimates were less accurate in the proportion of studies conducted in clinical settings. We would encourage thorough reporting of the population studied, details of clinically relevant aspects of methodology, and the use of appropriate statistical methods in future studies. Systematic review registration: PROSPERO CRD42016029167 Protocol: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0615-3.

Influence of acquisition frame-rate and video compression techniques on pulse-rate variability estimation from vPPG signal

In this paper, common time- and frequency-domain variability indexes obtained by pulse rate variability (PRV) series extracted from video-photoplethysmographic signal (vPPG) were compared with heart rate variability (HRV) parameters calculated from synchronized ECG signals. The dual focus of this study was to analyze the effect of different video acquisition frame-rates starting from 60 frames-per-second (fps) down to 7.5 fps and different video compression techniques using both lossless and lossy codecs on PRV parameters estimation. Video recordings were acquired through an off-the-shelf GigE Sony XCG-C30C camera on 60 young, healthy subjects (age 23±4 years) in the supine position. A fully automated, signal extraction method based on the Kanade-Lucas-Tomasi (KLT) algorithm for regions of interest (ROI) detection and tracking, in combination with a zero-phase principal component analysis (ZCA) signal separation technique was employed to convert the video frames sequence to a pulsatile signal. The frame-rate degradation was simulated on video recordings by directly sub-sampling the ROI tracking and signal extraction modules, to correctly mimic videos recorded at a lower speed. The compression of the videos was configured to avoid any frame rejection caused by codec quality leveling, FFV1 codec was used for lossless compression and H.264 with variable quality parameter as lossy codec. The results showed that a reduced frame-rate leads to inaccurate tracking of ROIs, increased time-jitter in the signals dynamics and local peak displacements, which degrades the performances in all the PRV parameters. The root mean square of successive differences (RMSSD) and the proportion of successive differences greater than 50 ms (PNN50) indexes in time-domain and the low frequency (LF) and high frequency (HF) power in frequency domain were the parameters which highly degraded with frame-rate reduction. Such a degradation can be partially mitigated by up-sampling the measured signal at a higher frequency (namely 60 Hz). Concerning the video compression, the results showed that compression techniques are suitable for the storage of vPPG recordings, although lossless or intra-frame compression are to be preferred over inter-frame compression methods. FFV1 performances are very close to the uncompressed (UNC) version with less than 45% disk size. H.264 showed a degradation of the PRV estimation directly correlated with the increase of the compression ratio.

Cardiovascular assessment by imaging photoplethysmography - a review

Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.

Optimum Wavelengths in the Near Infrared for Imaging Photoplethysmography

OBJECTIVE

The purpose of this contribution is to determine the ideal near infrared wavelength bands for monochromatic and dual-band remote heartbeat detection using imaging photoplethysmography (iPPG) of the forehead.

METHODS

Experimental data of 38 healthy volunteers has been recorded and analyzed. For the data acquisition, a fast hyperspectral imager has been used. A new combination approach has been implemented that computes the quotient of the bands and, therefore, reduces motion artifacts.

RESULTS

With this dual-band method excellent results (1.67 beats per minute mean deviation from electrocardiogram measurements for 73 recordings) have been obtained using a simple algorithm to analyze images at 799 and 861 nm.

CONCLUSION

It can be concluded that excellent imaging photoplethysmography measurements can be performed at low cost using conventional silicon-based image sensors with invisible light in the near infrared region.

SIGNIFICANCE

This approach is a contribution to the development of non-contact heart rate measurement systems that can be used for medical diagnosis or other applications.

InPhysible: Camouflage Against Video-Based Physiological Measurement

Imaging photoplethysmography (iPPG) is a powerful set of methods for measuring physiological signals from video. Recent advances have shown that a low-cost webcam can be used to measure heart rate, blood flow, respiration, blood oxygen levels and stress. While these methods have many beneficial applications, the unobtrusive and ubiquitous nature of the sensors risk exposing people to unwanted measurement. We present InPhysible the first camouflage system against video- based physiological measurement. The infra-red system can be embedded into any pair of glasses, or other headwear, and disrupts the measurement of the iPPG signal while being imperceptible by the human eye. Our system is flexible and can simulate realistic pulse signals to hinder heart rate measurement. In this paper we present the design of our prototype and a user study validating its efficacy. Finally, we discuss the limitations and implications for data privacy and security.

Remote Heart Rate Measurement from RGB-NIR Video Based on Spatial and Spectral Face Patch Selection

In this paper, we propose a novel heart rate (HR) estimation method using simultaneously recorded RGB and near-infrared (NIR) face videos. The key idea of our method is to automatically select suitable face patches for HR estimation in both spatial and spectral domains. The spatial and spectral face patch selection enables us to robustly estimate HR under various situations, including scenes under which existing RGB camera-based methods fail to accurately estimate HR. For a challenging scene in low light and with light fluctuations, our method can successfully estimate HR for all 20 subjects $( \pm 3$ beats per minute), while the RGB camera-based methods succeed only for 25% of the subjects.

Estimating carotid pulse and breathing rate from near-infrared video of the neck

OBJECTIVE

Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can be easily analyzed and/or require expensive and custom hardware to perform the measurements.

APPROACH

This work introduces a low-cost method to measure subtle motions associated with the carotid pulse and breathing movement from the neck using near-infrared (NIR) video imaging. A skin reflection model of the neck was established to provide a theoretical foundation for the method. In particular, the method relies on template matching for neck detection, principal component analysis for feature extraction, and hidden Markov models for data smoothing.

MAIN RESULTS

We compared the estimated HR and BR measures with ones provided by an FDA-cleared device in a 12-participant laboratory study: the estimates achieved a mean absolute error of 0.36 beats per minute and 0.24 breaths per minute under both bright and dark lighting.

SIGNIFICANCE

This work advances the possibilities of non-contact physiological measurement in real-life conditions in which environmental illumination is limited and in which the face of the person is not readily available or needs to be protected. Due to the increasing availability of NIR imaging devices, the described methods are readily scalable.

Unobtrusive Vital Sign Monitoring in Automotive Environments-A Review

This review provides an overview of unobtrusive monitoring techniques that could be used to monitor some of the human vital signs (i.e., heart activity, breathing activity, temperature and potentially oxygen saturation) in a car seat. It will be shown that many techniques actually measure mechanical displacement, either on the body surface and/or inside the body. However, there are also techniques like capacitive electrocardiogram or bioimpedance that reflect electrical activity or passive electrical properties or thermal properties (infrared thermography). In addition, photopleythysmographic methods depend on optical properties (like scattering and absorption) of biological tissues and-mainly-blood. As all unobtrusive sensing modalities are always fragile and at risk of being contaminated by disturbances (like motion, rapidly changing environmental conditions, triboelectricity), the scope of the paper includes a survey on redundant sensor arrangements. Finally, this review also provides an overview of automotive demonstrators for vital sign monitoring.

Using imaging photoplethysmography for heart rate estimation in non-human primates

For humans and for non-human primates heart rate is a reliable indicator of an individual's current physiological state, with applications ranging from health checks to experimental studies of cognitive and emotional state. In humans, changes in the optical properties of the skin tissue correlated with cardiac cycles (imaging photoplethysmogram, iPPG) allow non-contact estimation of heart rate by its proxy, pulse rate. Yet, there is no established simple and non-invasive technique for pulse rate measurements in awake and behaving animals. Using iPPG, we here demonstrate that pulse rate in rhesus monkeys can be accurately estimated from facial videos. We computed iPPGs from eight color facial videos of four awake head-stabilized rhesus monkeys. Pulse rate estimated from iPPGs was in good agreement with reference data from a contact pulse-oximeter: the error of pulse rate estimation was below 5% of the individual average pulse rate in 83% of the epochs; the error was below 10% for 98% of the epochs. We conclude that iPPG allows non-invasive and non-contact estimation of pulse rate in non-human primates, which is useful for physiological studies and can be used toward welfare-assessment of non-human primates in research.

Lock-in technique for extraction of pulse rates and associated confidence levels from video

We investigate the practical applicability of video photoplethysmography (VPPG) to extract heart rates of subjects using noncontact color video recordings of human faces collected under typical indoor laboratory conditions using commercial video cameras. Videos were processed following three previously described simple VPPG algorithms to produce a time-varying plethysmographic signal. These time signals were then analyzed using, to the best of our knowledge, a novel, lock-in algorithm that was developed to extract the pulsatile frequency component. A protocol to associate confidence estimates for the extracted heart rates for each video stream is presented. Results indicate that the difference between heart rates extracted using the lock-in technique and gold-standard measurements, for videos with high-confidence metrics, was less than 4 beats per minute. Constraints on video acquisition and processing, including natural subject motion and the total duration of video recorded required for evaluating these confidence metrics, are discussed.

Video pulse rate variability analysis in stationary and motion conditions

BACKGROUND

In the last few years, some studies have measured heart rate (HR) or heart rate variability (HRV) parameters using a video camera. This technique focuses on the measurement of the small changes in skin colour caused by blood perfusion. To date, most of these works have obtained HRV parameters in stationary conditions, and there are practically no studies that obtain these parameters in motion scenarios and by conducting an in-depth statistical analysis.

METHODS

In this study, a video pulse rate variability (PRV) analysis is conducted by measuring the pulse-to-pulse (PP) intervals in stationary and motion conditions. Firstly, given the importance of the sampling rate in a PRV analysis and the low frame rate of commercial cameras, we carried out an analysis of two models to evaluate their performance in the measurements. We propose a selective tracking method using the Viola-Jones and KLT algorithms, with the aim of carrying out a robust video PRV analysis in stationary and motion conditions. Data and results of the proposed method are contrasted with those reported in the state of the art.

RESULTS

The webcam achieved better results in the performance analysis of video cameras. In stationary conditions, high correlation values were obtained in PRV parameters with results above 0.9. The PP time series achieved an RMSE (mean ± standard deviation) of 19.45 ± 5.52 ms (1.70 ± 0.75 bpm). In the motion analysis, most of the PRV parameters also achieved good correlation results, but with lower values as regards stationary conditions. The PP time series presented an RMSE of 21.56 ± 6.41 ms (1.79 ± 0.63 bpm).

CONCLUSIONS

The statistical analysis showed good agreement between the reference system and the proposed method. In stationary conditions, the results of PRV parameters were improved by our method in comparison with data reported in related works. An overall comparative analysis of PRV parameters in motion conditions was more limited due to the lack of studies or studies containing insufficient data analysis. Based on the results, the proposed method could provide a low-cost, contactless and reliable alternative for measuring HR or PRV parameters in non-clinical environments.

Unobtrusive assessment of neonatal sleep state based on heart rate variability retrieved from electrocardiography used for regular patient monitoring

As an approach of unobtrusive assessment of neonatal sleep state we aimed at an automated sleep state coding based only on heart rate variability obtained from electrocardiography used for regular patient monitoring. We analyzed active and quiet sleep states of preterm infants between 30 and 37weeks postmenstrual age. To determine the sleep states we used a nonlinear kernel support vector machine for sleep state separation based on known heart rate variability features. We used unweighted and weighted misclassification penalties for the imbalanced distribution between sleep states. The validation was performed with leave-one-out-cross-validation based on the annotations of three independent observers. We analyzed the classifier performance with receiver operating curves leading to a maximum mean value for the area under the curve of 0.87. Using this sleep state separation methods, we show that automated active and quiet sleep state separation based on heart rate variability in preterm infants is feasible.

A Real-Time Contactless Pulse Rate and Motion Status Monitoring System Based on Complexion Tracking

Subject movement and a dark environment will increase the difficulty of image-based contactless pulse rate detection. In this paper, we detected the subject’s motion status based on complexion tracking and proposed a motion index (MI) to filter motion artifacts in order to increase pulse rate measurement accuracy. Additionally, we integrated the near infrared (NIR) LEDs with the adopted sensor and proposed an effective method to measure the pulse rate in a dark environment. To achieve real-time data processing, the proposed framework is constructed on a Field Programmable Gate Array (FPGA) platform. Next, the instant pulse rate and motion status are transmitted to a smartphone for remote monitoring. The experiment results showed the error of the pulse rate detection to be within −3.44 to +4.53 bpm under sufficient ambient light and −2.96 to + 4.24 bpm for night mode detection, when the moving speed is higher than 14.45 cm/s. These results demonstrate that the proposed method can improve the robustness of image-based contactless pulse rate detection despite subject movement and a dark environment.

New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring

Finger-oximeters are ubiquitously used for patient monitoring in hospitals worldwide. Recently, remote measurement of arterial blood oxygenation (SpO₂) with a camera has been demonstrated. Both contact and remote measurements, however, require the subject to remain static for accurate SpO₂ values. This is due to the use of the common ratio-of-ratios measurement principle that measures the relative pulsatility at different wavelengths. Since the amplitudes are small, they are easily corrupted by motion-induced variations. We introduce a new principle that allows accurate remote measurements even during significant subject motion. We demonstrate the main advantage of the principle, i.e. that the optimal signature remains the same even when the SNR of the PPG signal drops significantly due to motion or limited measurement area. The evaluation uses recordings with breath-holding events, which induce hypoxemia in healthy moving subjects. The events lead to clinically relevant SpO₂ levels in the range 80–100%. The new principle is shown to greatly outperform current remote ratio-of-ratios based methods. The mean-absolute SpO₂-error (MAE) is about 2 percentage-points during head movements, where the benchmark method shows a MAE of 24 percentage-points. Consequently, we claim ours to be the first method to reliably measure SpO₂ remotely during significant subject motion.

Robust respiration detection from remote photoplethysmography

Continuous monitoring of respiration is essential for early detection of critical illness. Current methods require sensors attached to the body and/or are not robust to subject motion. Alternative camera-based solutions have been presented using motion vectors and remote photoplethysmography. In this work, we present a non-contact camera-based method to detect respiration, which can operate in both visible and dark lighting conditions by detecting the respiratory-induced colour differences of the skin. We make use of the close similarity between skin colour variations caused by the beating of the heart and those caused by respiration, leading to a much improved signal quality compared to single-channel approaches. Essentially, we propose to find the linear combination of colour channels which suppresses the distortions best in a frequency band including pulse rate, and subsequently we use this same linear combination to extract the respiratory signal in a lower frequency band. Evaluation results obtained from recordings on healthy subjects which perform challenging scenarios, including motion, show that respiration can be accurately detected over the entire range of respiratory frequencies, with a correlation coefficient of 0.96 in visible light and 0.98 in infrared, compared to 0.86 with the best-performing non-contact benchmark algorithm. Furthermore, evaluation on a set of videos recorded in a Neonatal Intensive Care Unit (NICU) shows that this technique looks promising as a future alternative to current contact-sensors showing a correlation coefficient of 0.87.