A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation

A Comparative Study of Principled rPPG-Based Pulse Rate Tracking Algorithms for Fitness Activities

Performance improvements obtained by recent principled approaches for pulse rate (PR) estimation from face videos have typically been achieved by adding or modifying certain modules within a reconfigurable system. Yet, evaluations of such remote photoplethysmography (rPPG) are usually performed only at the system level. To better understand each module's contribution and facilitate future research in explainable learning and artificial intelligence for physiological monitoring, this paper conducts a comparative study of video-based, principled PR tracking algorithms, with a focus on challenging fitness scenarios. A review of the progress achieved over the last decade and a half in this field is utilized to construct the major processing modules of a reconfigurable remote pulse rate sensing system. Experiments are conducted on two challenging datasets-an internal collection of 25 videos of two Asian males exercising on stationary-bike, elliptical, and treadmill machines and 34 videos from a public ECG fitness database of 14 men and 3 women exercising on elliptical and stationary-bike machines. The signal-to-noise ratio (SNR), Pearson's correlation coefficient, error count ratio, error rate, and root mean squared error are used for performance evaluation. The top-performing configuration produces respective values of 0.8 dB, 0.86, 9%, 1.7%, and 3.3 beats per minute (bpm) for the internal dataset and 1.3 dB, 0.77, 28.6%, 6.0%, and 8.1 bpm for the ECG Fitness dataset, achieving significant improvements over alternative configurations. Our results suggest a synergistic effect between pulse color mapping and adaptive motion filtering, as well as the importance of a robust frequency tracking algorithm for PR estimation in low SNR settings.

ConDiff-rPPG: Robust Remote Physiological Measurement to Heterogeneous Occlusions

Remote photoplethysmography (rPPG) is a contactless technique that facilitates the measurement of physiological signals and cardiac activities through facial video recordings. This approach holds tremendous potential for various applications. However, existing rPPG methods often did not account for different types of occlusions that commonly occur in real-world scenarios, such as temporary movement or actions of humans in videos or dust on camera. The failure to address these occlusions can compromise the accuracy of rPPG algorithms. To address this issue, we proposed a novel Condiff-rPPG to improve the robustness of rPPG measurement facing various occlusions. First, we compressed the damaged face video into a spatio-temporal representation with several types of masks. Second, the diffusion model was designed to recover the missing information with observed values as a condition. Moreover, a novel low-rank decomposition regularization was proposed to eliminate background noise and maximize informative features. ConDiff-rPPG ensured consistency in optimization goals during the training process. Through extensive experiments, including intra- and cross-dataset evaluations, as well as ablation tests, we demonstrated the robustness and generalization ability of our proposed model.

Remote physiological signal recovery with efficient spatio-temporal modeling

Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies.

Privacy-Protected Contactless Sleep Parameters Measurement Using a Defocused Camera

Sleep monitoring plays a vital role in various scenarios such as hospitals and living-assisted homes, contributing to the prevention of sleep accidents as well as the assessment of sleep health. Contactless camera-based sleep monitoring is promising due to its user-friendly nature and rich visual semantics. However, the privacy concern of video cameras limits their applications in sleep monitoring. In this paper, we explored the opportunity of using a defocused camera that does not allow identification of the monitored subject when measuring sleep-related parameters, as face detection and recognition are impossible on optically blurred images. We proposed a novel privacy-protected sleep parameters measurement framework, including a physiological measurement branch and a semantic analysis branch based on ResNet-18. Four important sleep parameters are measured: heart rate (HR), respiration rate (RR), sleep posture, and movement. The results of HR, RR, and movement have strong correlations with the reference (HR: R = 0.9076; RR: R = 0.9734; Movement: R = 0.9946). The overall mean absolute errors (MAE) for HR and RR are 5.2 bpm and 1.5 bpm respectively. The measurement of HR and RR achieve reliable estimation coverage of 72.1% and 93.6%, respectively. The sleep posture detection achieves an overall accuracy of 94.5%. Experimental results show that the defocused camera is promising for sleep monitoring as it fundamentally eliminates the privacy issue while still allowing the measurement of multiple parameters that are essential for sleep health informatics.

Efficient detection of driver fatigue state based on all-weather illumination scenarios

Among the causes of the annually traffic accidents, driving fatigue is the main culprit. In consequence, it is of great practical significance to carry out the research of driving fatigue detection and early warning system. However, there are still two problems in the latest methods of driving fatigue detection: one is that a single information cannot precisely reflect the actual state of the driver in different fatigue phases, another one is the detection effect is not very well or even difficult to detect under abnormal illumination. In this paper, the multi-task cascaded convolutional networks (MTCNN) and infrared-based remote photo-plethysmography (rPPG) theory are used to extract the driver's facial and physiological information, and the multi-modal specific fatigue information is deeply excavated, and the multi-modal feature fusion model is constructed to comprehensively analyze the driver's fatigue variation tendency. Aiming at the matter of low detection accuracy under abnormal illumination, the multi-modal features extracted from visible light images and infrared images are fused by multi-loss reconstruction (MLR) module, and the driving fatigue detection module is established which is based on Bi-LSTM model by utilizing fatigue timing. The experiments were validated under all-weather illumination scenarios and were carried out on the datasets NTHU-DDD, UTA-RLDDD and FAHD. The results show that the multi-modal driving fatigue detection model has better performance than the single-modal model, and the accuracy is improved by 8.1%. In the abnormal illumination such as strong and weak light, the accuracy of the method can reach 91.7% at the highest and 83.6% at the lowest. Meanwhile, in the normal illumination, it can reach 93.2%.

Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement

In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.

Non-contact Heart Rate and Respiratory Rate Estimation from Videos of the Neck

Heart Rate (HR) and Respiratory Rate (RR) estimation constitutes a crucial part of non-contact assessment of cardiovascular disease, which has been a leading cause of death worldwide. This paper proposes a novel HR and RR estimation algorithm based on RGB videos recorded by a smartphone camera. Instead of requiring facial videos, this novel algorithm demonstrates the ability to estimate HR and RR using only a video of the human neck. This novel algorithm captures cardiac as well as respiratory activity via detecting skin displacement by only analyzing the Laplacian pyramid of each frame of the video. Its performance was evaluated by applying it to the videos of neck of 80 participants and comparing it to existing methods, demonstrating the superior performance of the proposed algorithm.

PhysioSens1D-NET: A 1D Convolution Network for Extracting Heart Rate from Facial Videos

Non-contact heart rate (HR) monitoring from video streams is the most established approach to unobtrusive vitals monitoring. A multitude of classical signal processing algorithms and cutting-edge deep learning models have been developed for non-contact HR extraction. Classical signal processing algorithms excel in real-time application, even on low-end CPUs, while deep learning models offer higher accuracy at the cost of computational complexity. In this study, we introduce PhysioSens1DNET- a novel 1D convolutional neural network, that deliver both computational efficiency and accurate HR measures. In contrast to classical rPPG algorithms like ICA, POS, CHROM, PBV, LGI, and GREEN, the PhysioSens1D-NET demonstrates significant improvements, achieving reductions in Mean Absolute Error (MAE) by 91.4%, 72.5%, 70.7%, 93.1%, 76.7%, and 95.1%, respectively. When compared to state-of-the-art deep learning models, including DeepPhys, EfficientNet, PhysNet, and TS-CAN, our 1D-NET exhibits comparable performance. A performance analysis on low specification CPU's, indicated that PhysioSens1DNET outperforms deep learning models, showcasing a considerable speed advantage-being 180 times faster than the bestperforming DL model. Furthermore, our 1D-NET aligns closely with classical algorithms with a computational time of only 2.3 ms.

Integrating RGB-thermal image sensors for non-contact automatic respiration rate monitoring

Respiration rate (RR) holds significance as a human health indicator. Presently, the conventional RR monitoring system requires direct physical contact, which may cause discomfort and pain. Therefore, this paper proposes a non-contact RR monitoring system integrating RGB and thermal imaging through RGB-thermal image alignment. The proposed method employs an advanced image processing algorithm for automatic region of interest (ROI) selection. The experimental results demonstrated a close correlation and a lower error rate between measured thermal, measured RGB, and reference data. In summary, the proposed non-contact system emerges as a promising alternative to conventional contact-based approaches without the associated discomfort and pain.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

An overview of current assessment techniques for evaluating cutaneous perfusion in reconstructive surgery

This article provides a comprehensive analysis of modern techniques used in the assessment of cutaneous flaps in reconstructive surgery. It emphasizes the importance of preoperative planning and intra- and perioperative assessment of flap perfusion to ensure successful outcomes. Despite technological advancements, direct clinical assessment remains the gold standard. We categorized assessment techniques into non-invasive and invasive modalities, discussing their strengths and weaknesses. Non-invasive methods, such as acoustic Doppler sonography, near-infrared spectroscopy, hyperspectral imaging thermal imaging, and remote-photoplethysmography, offer accessibility and safety but may sacrifice specificity. Invasive techniques, including contrast-enhanced ultrasound, computed tomography angiography, near-infrared fluorescence angiography with indocyanine green, and implantable Doppler probe, provide high accuracy but introduce additional risks. We emphasize the need for a tailored decision-making process based on specific clinical scenarios, patient characteristics, procedural requirements, and surgeon expertise. It also discusses potential future advancements in flap assessment, including the integration of artificial intelligence and emerging technologies.

Preserving shape details of pulse signals for video-based blood pressure estimation

In recent years, imaging photoplethysmograph (iPPG) pulse signals have been widely used in the research of non-contact blood pressure (BP) estimation, in which BP estimation based on pulse features is the main research direction. Pulse features are directly related to the shape of pulse signals while iPPG pulse signals are easily disturbed during the extraction process. To mitigate the impact of pulse feature distortion on BP estimation, it is necessary to eliminate interference while retaining valuable shape details in the iPPG pulse signal. Contact photoplethysmograph (cPPG) pulse signals measured at rest can be considered as the undisturbed reference signal. Transforming the iPPG pulse signal to the corresponding cPPG pulse signal is a method to ensure the effectiveness of shape details. However, achieving the required shape accuracy through direct transformation from iPPG to the corresponding cPPG pulse signals is challenging. We propose a method to mitigate this challenge by replacing the reference signal with an average cardiac cycle (ACC) signal, which can approximately represent the shape information of all cardiac cycles in a short time. A neural network using multi-scale convolution and self-attention mechanisms is developed for this transformation. Our method demonstrates a significant improvement in the maximal information coefficient (MIC) between pulse features and BP values, indicating a stronger correlation. Moreover, pulse signals transformed by our method exhibit enhanced performance in BP estimation using different model types. Experiments are conducted on a real-world database with 491 subjects in the hospital, averaging 60 years of age.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Face2PPG: An Unsupervised Pipeline for Blood Volume Pulse Extraction From Faces

Photoplethysmography (PPG) signals have become a key technology in many fields, such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract remote PPG signals (rPPG) from the face robustly, reliably, and configurably. We identify and evaluate the possible choices in the critical steps of unsupervised rPPG methodologies. We assess a state-of-the-art processing pipeline in six different datasets, incorporating important corrections in the methodology that ensure reproducible and fair comparisons. In addition, we extend the pipeline by proposing three novel ideas; 1) a new method to stabilize the detected face based on a rigid mesh normalization; 2) a new method to dynamically select the different regions in the face that provide the best raw signals, and 3) a new RGB to rPPG transformation method, called Orthogonal Matrix Image Transformation (OMIT) based on QR decomposition, that increases robustness against compression artifacts. We show that all three changes introduce noticeable improvements in retrieving rPPG signals from faces, obtaining state-of-the-art results compared with unsupervised, non-learning-based methodologies and, in some databases, very close to supervised, learning-based methods. We perform a comparative study to quantify the contribution of each proposed idea. In addition, we depict a series of observations that could help in future implementations.

Non-contact high precision pulse-rate monitoring system for moving subjects in different motion states

Remote photoplethysmography (rPPG) enables contact-free monitoring of the pulse rate by using a color camera. The fundamental limitation is that motion artifacts and changes in ambient light conditions greatly affect the accuracy of pulse-rate monitoring. We propose use of a high-speed camera and a motion suppression algorithm with high computational efficiency. This system incorporates a number of major improvements including reproduction of pulse wave details, high-precision pulse-rate monitoring of moving subjects, and excellent scene scalability. A series of quantization methods were used to evaluate the effect of different frame rates and different algorithms in pulse-rate monitoring of moving subjects. The experimental results show that use of 180-fps video and a Plane-Orthogonal-to-Skin (POS) algorithm can produce high-precision pulse-rate monitoring results with mean absolute error can be less than 5 bpm and the relative accuracy reaching 94.5%. Thus, it has significant potential to improve personal health care and intelligent health monitoring.

Robust Heart Rate Variability Measurement from Facial Videos

Remote Photoplethysmography (rPPG) is a contactless method that enables the detection of various physiological signals from facial videos. rPPG utilizes a digital camera to detect subtle changes in skin color to measure vital signs such as heart rate variability (HRV), an important biomarker related to the autonomous nervous system. This paper presents a novel contactless HRV extraction algorithm, WaveHRV, based on the Wavelet Scattering Transform technique, followed by adaptive bandpass filtering and inter-beat-interval (IBI) analysis. Furthermore, a novel method is introduced to preprocess noisy contact-based PPG signals. WaveHRV is bench-marked against existing algorithms and public datasets. Our results show that WaveHRV is promising and achieves the lowest mean absolute error (MAE) of 10.5 ms and 6.15 ms for RMSSD and SDNN on the UBFCrPPG dataset.

Heart Rate Estimation from Facial Image Sequences of a Dual-Modality RGB-NIR Camera

This paper presents an RGB-NIR (Near Infrared) dual-modality technique to analyze the remote photoplethysmogram (rPPG) signal and hence estimate the heart rate (in beats per minute), from a facial image sequence. Our main innovative contribution is the introduction of several denoising techniques such as Modified Amplitude Selective Filtering (MASF), Wavelet Decomposition (WD), and Robust Principal Component Analysis (RPCA), which take advantage of RGB and NIR band characteristics to uncover the rPPG signals effectively through this Independent Component Analysis (ICA)-based algorithm. Two datasets, of which one is the public PURE dataset and the other is the CCUHR dataset built with a popular Intel RealSense D435 RGB-D camera, are adopted in our experiments. Facial video sequences in the two datasets are diverse in nature with normal brightness, under-illumination (i.e., dark), and facial motion. Experimental results show that the proposed method has reached competitive accuracies among the state-of-the-art methods even at a shorter video length. For example, our method achieves MAE = 4.45 bpm (beats per minute) and RMSE = 6.18 bpm for RGB-NIR videos of 10 and 20 s in the CCUHR dataset and MAE = 3.24 bpm and RMSE = 4.1 bpm for RGB videos of 60-s in the PURE dataset. Our system has the advantages of accessible and affordable hardware, simple and fast computations, and wide realistic applications.

Pulse rate variability estimation method based on imaging-photoplethysmography and application to telepsychiatry

The worldwide adoption of telehealth services may benefit people who otherwise would not be able to access mental health support. In this paper, we present a novel algorithm to obtain reliable pulse and respiration signals from non-contact facial image sequence analysis. The proposed algorithm involved a skin pixel extraction method in the image processing part and signal reconstruction using the spectral information of RGB signal in the signal processing part. The algorithm was tested on 15 healthy subjects in a laboratory setting. The results show that the proposed algorithm can accurately monitor respiration rate (RR), pulse rate (PR), and pulse rate variability (PRV) in rest conditions.Clinical Relevance- The main achievement of this study is enabling non-contact PR and RR signal extraction from facial image sequences, which has potential for future use and support for psychiatrists in telepsychiatry.

The Darker the Skin, the Greater the Disparity? Why a Reliance on Visible Injuries Fosters Health, Legal, and Racial Disparities in Domestic Violence Complaints Involving Strangulation

The reliance on external injuries for justice is misguided given that assault injuries may be less visible among victims of color due to increased melanin in the skin. To date, however, less is known whether racial/ethnic disparities extend to officers' identification of signs of nonfatal strangulation (NFS). The current study estimates the extent of NFS indicators identified by officers who completed a standardized strangulation assessment in 133 family violence complaints. Breathing difficulties were the most common symptoms identified by officers (98%), followed by external signs (89%), and symptoms of impeded blood circulation (87%). Compared to cases involving White/Asian survivors, officers were less likely to identify external injuries on Black survivors' neck, chin, and chest/shoulders. While racial/ethnic differences did not emerge for symptoms of disrupted airflow, Hispanic survivors were twice as likely to report losing control of bodily functions. Implications for policy and practice are discussed.

X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation

Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR (MAE = 4.10 ; RMSE = 5.32 ; r = 0.85), UBFC-rPPG (MAE = 4.99 ; RMSE = 6.26 ; r = 0.67), MAHNOB-HCI (MAE = 3.17 ; RMSE = 3.93 ; r = 0.88).

Optimal digital filter selection for remote photoplethysmography (rPPG) signal conditioning

Remote photoplethysmography (rPPG) using camera-based imaging has shown excellent potential recently in vital signs monitoring due to its contactless nature. However, the optimum filter selection for pre-processing rPPG data in signal conditioning is still not straightforward. The best algorithm selection improves the signal-to-noise ratio (SNR) and therefore improves the accuracy of the recognition and classification of vital signs. We recorded more than 300 temporal rPPG signals where the noise was not motion-induced. Then, we investigated the best digital filter in pre-processing temporal rPPG data and compared the performances of 10 filters with 10 orders each (i.e., a total of 100 filters). The performances are assessed using a signal quality metric on three levels. The quality of the raw signals was classified under three categories; Q1 being the best and Q3 being the worst. The results are presented in SNR scores, which show that the Chebyshev II orders of 2nd, 4th, and 6th perform the best for denoising rPPG signals.

Heart rate estimation network from facial videos using spatiotemporal feature image

Remote health monitoring has become quite inevitable after SARS-CoV-2 pandemic and continues to be accepted as a measure of healthcare in future too. However, contact-less measurement of vital sign, like Heart Rate(HR) is quite difficult to measure because, the amplitude of physiological signal is very weak and can be easily degraded due to noise. The various sources of noise are head movements, variation in illumination or acquisition devices. In this paper, a video-based noise-less cardiopulmonary measurement is proposed. 3D videos are converted to 2D Spatio-Temporal Images (STI), which suppresses noise while preserving temporal information of Remote Photoplethysmography(rPPG) signal. The proposed model projects a new motion representation to CNN derived using wavelets, which enables estimation of HR under heterogeneous lighting condition and continuous motion. STI is formed by the concatenation of feature vectors obtained after wavelet decomposition of subsequent frames. STI is provided as input to CNN for mapping the corresponding HR values. The proposed approach utilizes the ability of CNN to visualize patterns. Proposed approach yields better results in terms of estimation of HR on four benchmark dataset such as MAHNOB-HCI, MMSE-HR, UBFC-rPPG and VIPL-HR.

Effects of illuminance intensity on the green channel of remote photoplethysmography (rPPG) signals

Point-of-care remote photoplethysmography (rPPG) devices that utilize low-cost RGB cameras have drawn considerable attention due to their convenience in contactless and non-invasive vital signs monitoring. In rPPG, sufficient lighting conditions are essential for obtaining accurate diagnostics by observing the complete signal morphology. The effects of illuminance intensity and light source settings play a significant role in rPPG assessment quality, and it was previously observed that different lighting schemes result in different signal quality and morphology. This study presents a quantitative empirical analysis where the quality and morphology of rPPG signals were assessed under different light settings. Participants' faces were exposed to the white LED spotlight, first when the sources were installed directly behind the video camera, and then when the sources were installed in a cross-polarized scheme. Hence, the effect of specular reflectance on rPPG signals could be observed in an increasing projection. The signal qualities were analyzed in each intensity level using a signal-to-noise (SNR) ratio metric. In 3 of 7 participants, placing the video camera on the same level as the light source led to signal quality loss of up to 3 dB for the range 30-60 Lux. In addition, two fundamental morphological features were analyzed, and the derivative-related feature was found to be increasing with illuminance intensity in 6 of 7 participants.

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

A real-time heart rate estimation framework based on a facial video while wearing a mask

BACKGROUND: The imaging photoplethysmography (iPPG) method is a non-invasive, non-contact measurement method that uses a camera to detect physiological indicators. On the other hand, wearing a mask has become essential today when COVID-19 is rampant, which has become a new challenge for heart rate (HR) estimation from facial videos recorded by a camera. OBJECTIVE: The aim is to propose an iPPG-based method that can accurately estimate HR with or without a mask. METHODS: First, the facial regions of interest (ROI) were divided into two sub-ROIs, and the original signal was obtained through spatial averaging with different weights according to the result of judging whether wearing a mask or not, and the CDF, which emphasizes the main component signal, was combined with the improved POS suitable for real-time HR estimation to obtain the noise-removed BVP signal. RESULTS: For self-collected data while wearing a mask, MAE, RMSE, and ACC were 1.09 bpm, 1.44 bpm, and 99.08%, respectively. CONCLUSION: Experimental results show that the proposed framework can estimate HR stably in real-time in both cases of wearing a mask or not. This study expands the application range of HR estimation based on facial videos and has very practical value in real-time HR estimation in daily life.

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Objective. Efficient non-contact heart rate (HR) measurement from facial video has received much attention in health monitoring. Past methods relied on prior knowledge and an unproven hypothesis to extract remote photoplethysmography (rPPG) signals, e.g. manually designed regions of interest (ROIs) and the skin reflection model.Approach. This paper presents a short-time end to end HR estimation framework based on facial features and temporal relationships of video frames. In the proposed method, a deep 3D multi-scale network with cross-layer residual structure is designed to construct an autoencoder and extract robust rPPG features. Then, a spatial-temporal fusion mechanism is proposed to help the network focus on features related to rPPG signals. Both shallow and fused 3D spatial-temporal features are distilled to suppress redundant information in the complex environment. Finally, a data augmentation strategy is presented to solve the problem of uneven distribution of HR in existing datasets.Main results. The experimental results on four face-rPPG datasets show that our method overperforms the state-of-the-art methods and requires fewer video frames. Compared with the previous best results, the proposed method improves the root mean square error (RMSE) by 5.9%, 3.4% and 21.4% on the OBF dataset (intra-test), COHFACE dataset (intra-test) and UBFC dataset (cross-test), respectively.Significance. Our method achieves good results on diverse datasets (i.e. highly compressed video, low-resolution and illumination variation), demonstrating that our method can extract stable rPPG signals in short time.

RobustPPG: camera-based robust heart rate estimation using motion cancellation

Camera-based heart rate measurement is becoming an attractive option as a non-contact modality for continuous remote health and engagement monitoring. However, reliable heart rate extraction from camera-based measurement is challenging in realistic scenarios, especially when the subject is moving. In this work, we develop a motion-robust algorithm, labeled RobustPPG, for extracting photoplethysmography signals (PPG) from face video and estimating the heart rate. Our key innovation is to explicitly model and generate motion distortions due to the movements of the person's face. We use inverse rendering to obtain the 3D shape and albedo of the face and environment lighting from video frames and then render the human face for each frame. The rendered face is similar to the original face but does not contain the heart rate signal; facial movements alone cause pixel intensity variation in the generated video frames. Finally, we use the generated motion distortion to filter the motion-induced measurements. We demonstrate that our approach performs better than the state-of-the-art methods in extracting a clean blood volume signal with over 2 dB signal quality improvement and 30% improvement in RMSE of estimated heart rate in intense motion scenarios.

Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey

Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.

Multitask Siamese Network for Remote Photoplethysmography and Respiration Estimation

Heart and respiration rates represent important vital signs for the assessment of a person’s health condition. To estimate these vital signs accurately, we propose a multitask Siamese network model (MTS) that combines the advantages of the Siamese network and the multitask learning architecture. The MTS model was trained by the images of the cheek including nose and mouth and forehead areas while sharing the same parameters between the Siamese networks, in order to extract the features about the heart and respiratory information. The proposed model was constructed with a small number of parameters and was able to yield a high vital-sign-prediction accuracy, comparable to that obtained from the single-task learning model; furthermore, the proposed model outperformed the conventional multitask learning model. As a result, we can simultaneously predict the heart and respiratory signals with the MTS model, while the number of parameters was reduced by 16 times with the mean average errors of heart and respiration rates being 2.84 and 4.21. Owing to its light weight, it would be advantageous to implement the vital-sign-monitoring model in an edge device such as a mobile phone or small-sized portable devices.

Skin tissue perfusion mapping triggered by an audio-(de)modulated reference signal

Spatial mapping of skin perfusion provides essential information about physiological processes that are often hidden from the eyes of the examining physician. The perfusion map quality depends on several key factors, such as the camera system type, frame rate, sensitivity, or signal-to-noise ratio. When investigating physiological parameters, the reference signal allows for increasing the spatial resolution of the photoplethysmography imaging (PPGI) system. On the other hand, it increases the system complexity and the synchronization prerequisites. Our solution is a hardware device that modulates the reference biosignal into the audio frequency band. This signal is connected to the mic input of a digital camera or a smartphone, enabling the transformation of such a device into a PPGI measurement system even in the case of compressed video recording using lock-in amplification technique. It also brings the possibility of synchronous recording of PPGI and another reference signal such as conventional photoplethysmogram or electrocardiogram.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications. The emergence of novel coronavirus pneumonia COVID-19 has attracted worldwide attentions. Contact photoplethysmography (cPPG) methods need to contact the detection equipment with the patient, which may accelerate the spread of the epidemic. In the future, the non-contact heart rate detection will be an urgent need. However, existing heart rate measuring methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and wearing a mask). In this paper, we proposed a method of heart rate detection based on eye location of region of interest (ROI) to solve the problem of missing information when wearing masks. Besides, a model to filter outliers based on residual network was conceived first by us and the better heart rate measurement accuracy was generated. To validate our method, we also created a mask dataset. The results demonstrated that after using our method for correcting the heart rate (HR) value measured with the traditional method, the accuracy reaches 4.65 bpm, which is 0.42 bpm higher than that without correction.

Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism

Remote photoplethysmography (rPPG) is a video-based non-contact heart rate measurement technology. It is a fact that most existing rPPG methods fail to deal with the spatiotemporal features of the video, which is significant for the extraction of the rPPG signal. In this paper, we propose a 3D central difference convolutional network (CDCA-rPPGNet) to measure heart rate, with an attention mechanism to combine spatial and temporal features. First, we crop and stitch the region of interest together through facial landmarks. Next, the high-quality regions of interest are fed to CDCA-rPPGNet based on a central difference convolution, which can enhance the spatiotemporal representation and capture rich relevant time contexts by collecting time difference information. In addition, we integrate the attention module into the neural network, aiming to strengthen the ability of the neural network to extract video channels and spatial features, so as to obtain more accurate rPPG signals. In summary, the three main contributions of this paper are as follows: (1) the proposed network base on central difference convolution could better capture the subtle color changes to recover the rPPG signals; (2) the proposed ROI extraction method provides high-quality input to the network; (3) the attention module is used to strengthen the ability of the network to extract features. Extensive experiments are conducted on two public datasets—the PURE dataset and the UBFC-rPPG dataset. In terms of the experiment results, our proposed method achieves 0.46 MAE (bpm), 0.90 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the PURE dataset, and 0.60 MAE (bpm), 1.38 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the UBFC dataset, which proves the effectiveness of our proposed approach.

pyVHR: a Python framework for remote photoplethysmography

Remote photoplethysmography (rPPG) aspires to automatically estimate heart rate (HR) variability from videos in realistic environments. A number of effective methods relying on data-driven, model-based and statistical approaches have emerged in the past two decades. They exhibit increasing ability to estimate the blood volume pulse (BVP) signal upon which BPMs (Beats per Minute) can be estimated. Furthermore, learning-based rPPG methods have been recently proposed. The present pyVHR framework represents a multi-stage pipeline covering the whole process for extracting and analyzing HR fluctuations. It is designed for both theoretical studies and practical applications in contexts where wearable sensors are inconvenient to use. Namely, pyVHR supports either the development, assessment and statistical analysis of novel rPPG methods, either traditional or learning-based, or simply the sound comparison of well-established methods on multiple datasets. It is built up on accelerated Python libraries for video and signal processing as well as equipped with parallel/accelerated ad-hoc procedures paving the way to online processing on a GPU. The whole accelerated process can be safely run in real-time for 30 fps HD videos with an average speedup of around 5. This paper is shaped in the form of a gentle tutorial presentation of the framework.

AND-rPPG: A novel denoising-rPPG network for improving remote heart rate estimation

Heart rate (HR) estimation is an essential physiological parameter in the field of biomedical imaging. Remote Photoplethysmography (r-PPG) is a pathbreaking development in this field wherein the PPG signal is extracted from non-contact face videos. In the COVID-19 pandemic, rPPG plays a vital role for doctors and patients to perform telehealthcare. Existing rPPG methods provide incorrect HR estimation when face video contains facial deformations induced by facial expression. These methods process the entire face and utilize the same knowledge to mitigate different noises. It limits the performance of these methods because different facial expressions induce different noise characteristics depending on the facial region. Another limitation is that these methods neglect the facial expression for denoising even though it is the prominent noise source in temporal signals. These issues are mitigated in this paper by proposing a novel HR estimation method AND-rPPG, that is, A Novel Denoising-rPPG. We initiate the utilization of Action Units (AUs) for denoising temporal signals. Our denoising network models the temporal signals better than sequential architectures and mitigate the AUs-based (or face expression-based) noises effectively. The experiments performed on publicly available datasets reveal that our proposed method outperforms state-of-the-art HR estimation methods, and our denoising model can be easily integrated with existing methods to improve their HR estimation.

Assessment of ROI Selection for Facial Video-Based rPPG

In general, facial image-based remote photoplethysmography (rPPG) methods use color-based and patch-based region-of-interest (ROI) selection methods to estimate the blood volume pulse (BVP) and beats per minute (BPM). Anatomically, the thickness of the skin is not uniform in all areas of the face, so the same diffuse reflection information cannot be obtained in each area. In recent years, various studies have presented experimental results for their ROIs but did not provide a valid rationale for the proposed regions. In this paper, to see the effect of skin thickness on the accuracy of the rPPG algorithm, we conducted an experiment on 39 anatomically divided facial regions. Experiments were performed with seven algorithms (CHROM, GREEN, ICA, PBV, POS, SSR, and LGI) using the UBFC-rPPG and LGI-PPGI datasets considering 29 selected regions and two adjusted regions out of 39 anatomically classified regions. We proposed a BVP similarity evaluation metric to find a region with high accuracy. We conducted additional experiments on the TOP-5 regions and BOT-5 regions and presented the validity of the proposed ROIs. The TOP-5 regions showed relatively high accuracy compared to the previous algorithm’s ROI, suggesting that the anatomical characteristics of the ROI should be considered when developing a facial image-based rPPG algorithm.

Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera

This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.

VidAF: A Motion-Robust Model for Screening Atrial Fibrillation from Facial Videos

Atrial fibrillation (AF) is the most common arrhythmia, but an estimated 30% of patients with AF are unaware of their conditions. The purpose of this work is to design a model for AF screening from facial videos, with a focus on addressing typical motion disturbances in our real life, such as head movements and expression changes. This model detects a pulse signal from the skin color changes in a facial video by a convolution neural network, incorporating a phase-driven attention mechanism to suppress motion signals in the space domain. It then encodes the pulse signal into discriminative features for AF classification by a coding neural network, using a de-noise coding strategy to improve the robustness of the features to motion signals in the time domain. The proposed model was tested on a dataset containing 1200 samples of 100 AF patients and 100 non-AF subjects. Experimental results demonstrated that VidAF had significant robustness to facial motions, predicting clean pulse signals with the mean absolute error of inter-pulse intervals less than 100 milliseconds. Besides, the model achieved promising performance in AF identification, showing an accuracy of more than 90% in multiple challenging scenarios. VidAF provides a more convenient and cost-effective approach for opportunistic AF screening in the community.

Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda

Heart rate (HR) is one of the essential vital signs used to indicate the physiological health of the human body. While traditional HR monitors usually require contact with skin, remote photoplethysmography (rPPG) enables contactless HR monitoring by capturing subtle light changes of skin through a video camera. Given the vast potential of this technology in the future of digital healthcare, remote monitoring of physiological signals has gained significant traction in the research community. In recent years, the success of deep learning (DL) methods for image and video analysis has inspired researchers to apply such techniques to various parts of the remote physiological signal extraction pipeline. In this paper, we discuss several recent advances of DL-based methods specifically for remote HR measurement, categorizing them based on model architecture and application. We further detail relevant real-world applications of remote physiological monitoring and summarize various common resources used to accelerate related research progress. Lastly, we analyze the implications of research findings and discuss research gaps to guide future explorations.

A New Framework for Robust Heart Rate Measurement Based on the Head Motion State Estimation

It is of great significance in managing human health, preventing and curing diseases such as heart disease to measure and monitor the physiological parameters accurately and robustly. However, imaging photoplethysmography (iPPG) can be easily affected by the ambient illumination variations or the subject's motions. In this paper, therefore, a novel framework of heart rate (HR) measurement robust to both illumination and motion artefacts is proposed, which combines the projection-plane-switching-based iPPG method (2PS) with the singular spectrum analysis (SSA). Based on the estimation of the head motion state, one reasonable projection plane is firstly determined, the temporally normalized red-green-blue signals are projected onto the plane and a pulse signal is obtained by alpha-tuning. After that, singular spectrum analysis (SSA) is applied to the obtained pulse signal and the normalized B-channel signal of the facial region of interest (ROI) to remove the artefacts remained in the pulse signal. For the self-collected database and the public PURE database, Bland-Altman plots show that the proposed 2PS-SSA has better agreement than the five compared methods, where the mean biases are 0.59 beat per minute (bpm) and 0.034 bpm, with 95% limits from -2.59 bpm to 3.78 bpm and from -1.97 bpm to 2.04 bpm, respectively.

Recent Advances in Atherosclerotic Disease Screening Using Pervasive Healthcare

Atherosclerosis screening helps the medical model transform from therapeutic medicine to preventive medicine by assessing degree of atherosclerosis prior to the occurrence of fatal vascular events. Pervasive screening emphasizes atherosclerotic monitoring with easy access, quick process, and advanced computing. In this work, we introduced five cutting-edge pervasive technologies including imaging photoplethysmography (iPPG), laser Doppler, radio frequency (RF), thermal imaging (TI), optical fiber sensing and piezoelectric sensor. IPPG measures physiological parameters by using video images that record the subtle skin color changes consistent with cardiac-synchronous blood volume changes in subcutaneous arteries and capillaries. Laser Doppler obtained the information on blood flow by analyzing the spectral components of backscattered light from the illuminated tissues surface. RF is based on Doppler shift caused by the periodic movement of the chest wall induced by respiration and heartbeat. TI measures vital signs by detecting electromagnetic radiation emitted by blood flow. The working principle of optical fiber sensor is to detect the change of light properties caused by the interaction between the measured physiological parameter and the entering light. Piezoelectric sensors are based on the piezoelectric effect of dielectrics. All these pervasive technologies are noninvasive, mobile, and can detect physiological parameters related to atherosclerosis screening.

A Heart Rate Monitoring Framework for Real-World Drivers Using Remote Photoplethysmography

Remote photoplethysmography (rPPG) is an unobtrusive solution to heart rate monitoring in drivers. However, disturbances that occur during driving such as driver behavior, motion artifacts, and illuminance variation complicate the monitoring of heart rate. Faced with disturbance, one commonly used assumption is heart rate periodicity (or spectrum sparsity). Several methods improve stability at the expense of tracking sensitivity for heart rate variation. Based on statistical signal processing (SSP) and Monte Carlo simulations, the outlier probability is derived and ADaptive spectral filter banks (AD) is proposed as a new algorithm which provides an explicable tuning option for spectral filter banks to strike a balance between robustness and sensitivity in remote monitoring for driving scenarios. Moreover, we construct a driving database containing over 23 hours of data to verify the proposed algorithm. The influence on rPPG from driver habits (both amateurs and professionals), vehicle types (compact cars and buses), and routes are also evaluated. In comparison to state-of-the-art rPPG for driving scenarios, the mean absolute error in the Passengers, Compact Cars, and Buses scenarios is 3.43, 7.85, and 5.02 beats per minute, respectively. Moreover, AD also won the top third place in the first challenge on remote physiological signal sensing (RePSS) with relative low computational complexity.

Contact-Free Pulse Signal Extraction from Human Face Videos: A Review and New Optimized Filtering Approach

In this chapter, we review methods for video-based heart monitoring, from classical signal processing approaches to modern deep learning methods. In addition, we propose a new method for learning an optimal filter that can overcome many of the problems that can affect classical approaches, such as light reflection and subject's movements, at a fraction of the training cost of deep learning approaches. Following the usual procedures for region of interest extraction and tracking, robust skin color estimation and signal pre-processing, we introduce a least-squares error optimal filter, learnt using an established training dataset to estimate the photoplethysmographic (PPG) signal more accurately from the measured color changes over time. This method not only improves the accuracy of heart rate measurement but also resulted in the extraction of a cleaner pulse signal, which could be integrated into many other useful applications such as human biometric recognition or recognition of emotional state. The method was tested on the DEAP dataset and showed improved performance over the best previous classical method on that dataset. The results obtained show that our proposed contact-free heart rate measurement method has significantly improved on existing methods.

Modulation Model of the Photoplethysmography Signal for Vital Sign Extraction

This paper introduces an amplitude and frequency modulation (AM-FM) model to characterize the photoplethysmography (PPG) signal. The model indicates that the PPG signal spectrum contains one dominant frequency component - the heart rate (HR), which is guarded by two weaker frequency components on both sides; the distance from the dominant component to the guard components represents the respiratory rate (RR). Based on this model, an efficient algorithm is proposed to estimate both HR and RR by searching for the dominant frequency component and two guard components. The proposed method is performed in the frequency domain to estimate RR, which is more robust to additive noise than the prior art based on temporal features. Experiments were conducted on two types of PPG signals collected with a contact sensor (an oximeter) and a contactless visible imaging sensor (a color camera), respectively. The PPG signal from the contactless sensor is much noisier than the signal from the contact sensor. The experimental results demonstrate the effectiveness of the proposed algorithm, including under relatively noisy scenarios.

Classification of Hemodynamics Scenarios from a Public Radar Dataset Using a Deep Learning Approach

Contact-free sensors offer important advantages compared to traditional wearables. Radio-frequency sensors (e.g., radars) offer the means to monitor cardiorespiratory activity of people without compromising their privacy, however, only limited information can be obtained via movement, traditionally related to heart or breathing rate. We investigated whether five complex hemodynamics scenarios (resting, apnea simulation, Valsalva maneuver, tilt up and tilt down on a tilt table) can be classified directly from publicly available contact and radar input signals in an end-to-end deep learning approach. A series of robust k-fold cross-validation evaluation experiments were conducted in which neural network architectures and hyperparameters were optimized, and different data input modalities (contact, radar and fusion) and data types (time and frequency domain) were investigated. We achieved reasonably high accuracies of 88% for contact, 83% for radar and 88% for fusion of modalities. These results are valuable in showing large potential of radar sensing even for more complex scenarios going beyond just heart and breathing rate. Such contact-free sensing can be valuable for fast privacy-preserving hospital screenings and for cases where traditional werables are impossible to use.