Detail-preserving pulse wave extraction from facial videos using consumer-level camera

Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement

In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.

Preserving shape details of pulse signals for video-based blood pressure estimation

In recent years, imaging photoplethysmograph (iPPG) pulse signals have been widely used in the research of non-contact blood pressure (BP) estimation, in which BP estimation based on pulse features is the main research direction. Pulse features are directly related to the shape of pulse signals while iPPG pulse signals are easily disturbed during the extraction process. To mitigate the impact of pulse feature distortion on BP estimation, it is necessary to eliminate interference while retaining valuable shape details in the iPPG pulse signal. Contact photoplethysmograph (cPPG) pulse signals measured at rest can be considered as the undisturbed reference signal. Transforming the iPPG pulse signal to the corresponding cPPG pulse signal is a method to ensure the effectiveness of shape details. However, achieving the required shape accuracy through direct transformation from iPPG to the corresponding cPPG pulse signals is challenging. We propose a method to mitigate this challenge by replacing the reference signal with an average cardiac cycle (ACC) signal, which can approximately represent the shape information of all cardiac cycles in a short time. A neural network using multi-scale convolution and self-attention mechanisms is developed for this transformation. Our method demonstrates a significant improvement in the maximal information coefficient (MIC) between pulse features and BP values, indicating a stronger correlation. Moreover, pulse signals transformed by our method exhibit enhanced performance in BP estimation using different model types. Experiments are conducted on a real-world database with 491 subjects in the hospital, averaging 60 years of age.

Video plethysmography for contactless blood pressure and heart rate measurement in perioperative care

The purpose of this study was to evaluate the feasibility and accuracy of remote Video Plethysmography (VPPG) for contactless measurements of blood pressure (BP) and heart rate (HR) in adult surgical patients in a hospital setting. An iPad Pro was used to record a 1.5-minute facial video of the participant's face and VPPG was used to extract vital signs measurements. A standard medical device (Welch Allyn) was used for comparison to measure BP and HR. Trial registration: NCT05165381. Two-hundred-sixteen participants consented and completed the contactless BP and HR monitoring (mean age 54.1 ± 16.8 years, 58% male). The consent rate was 75% and VPPG was 99% successful in capturing BP and HR. VPPG predicted SBP, DBP, and HR with a measurement bias ± SD, -8.18 ± 16.44 mmHg, - 6.65 ± 9.59 mmHg, 0.09 ± 6.47 beats/min respectively. Pearson's correlation for all measurements between VPPG and standard medical device was significant. Correlation for SBP was moderate (0.48), DBP was weak (0.29), and HR was strong (0.85). Most patients were satisfied with the non-contact technology with an average rating of 8.7/10 and would recommend it for clinical use. VPPG was highly accurate in measuring HR, and is currently not accurate in measuring BP in surgical patients. The VPPG BP algorithm showed limitations in capturing individual variations in blood pressure, highlighting the need for further improvements to render it clinically effective across all ranges. Contactless vital signs monitoring was well-received and earned a high satisfaction score.

Non-contact heart rate estimation based on singular spectrum component reconstruction using low-rank matrix and autocorrelation

The remote photoplethysmography (rPPG) based on cameras, a technology for extracting pulse wave from videos, has been proved to be an effective heart rate (HR) monitoring method and has great potential in many fields; such as health monitoring. However, the change of facial color intensity caused by cardiovascular activities is weak. Environmental illumination changes and subjects' facial movements will produce irregular noise in rPPG signals, resulting in distortion of heart rate pulse signals and affecting the accuracy of heart rate measurement. Given the irregular noises such as motion artifacts and illumination changes in rPPG signals, this paper proposed a new method named LA-SSA. It combines low-rank sparse matrix decomposition and autocorrelation function with singular spectrum analysis (SSA). The low-rank sparse matrix decomposition is employed to globally optimize the components of the rPPG signal obtained by SSA, and some irregular noise is removed. Then, the autocorrelation function is used to optimize the global optimization results locally. The periodic components related to the heartbeat signal are selected, and the denoised rPPG signal is obtained by weighted reconstruction with a singular value ratio. The experiment using UBFC-RPPG and PURE database is performed to assess the performance of the method proposed in this paper. The average absolute error was 1.37 bpm, the 95% confidence interval was -7.56 bpm to 6.45 bpm, and the Pearson correlation coefficient was 98%, superior to most existing video-based heart rate extraction methods. Experimental results show that the proposed method can estimate HR effectively.

A real-time heart rate estimation framework based on a facial video while wearing a mask

BACKGROUND: The imaging photoplethysmography (iPPG) method is a non-invasive, non-contact measurement method that uses a camera to detect physiological indicators. On the other hand, wearing a mask has become essential today when COVID-19 is rampant, which has become a new challenge for heart rate (HR) estimation from facial videos recorded by a camera. OBJECTIVE: The aim is to propose an iPPG-based method that can accurately estimate HR with or without a mask. METHODS: First, the facial regions of interest (ROI) were divided into two sub-ROIs, and the original signal was obtained through spatial averaging with different weights according to the result of judging whether wearing a mask or not, and the CDF, which emphasizes the main component signal, was combined with the improved POS suitable for real-time HR estimation to obtain the noise-removed BVP signal. RESULTS: For self-collected data while wearing a mask, MAE, RMSE, and ACC were 1.09 bpm, 1.44 bpm, and 99.08%, respectively. CONCLUSION: Experimental results show that the proposed framework can estimate HR stably in real-time in both cases of wearing a mask or not. This study expands the application range of HR estimation based on facial videos and has very practical value in real-time HR estimation in daily life.

Heart rate estimation from facial videos with motion interference using T-SNE-based signal separation

Remote photoplethysmography (RPPG) can detect heart rate from facial videos in a non-contact way. However, head movement often affects its performance in the real world. In this paper, a novel anti-motion interference method named T-SNE-based signal separation (TSS) is proposed to solve this problem. TSS first decomposes the observed color traces into pulse-related vectors and noise vectors using the T-SNE algorithm. Then, it selects the vector with the most significant spectral peak as the pulse signal for heart rate measurement. The proposed method is tested on a self-collected dataset (17 males and 8 females) and two public datasets (UBFC-RPPG and VIPL-HR). Experimental results show that the proposed method outperforms state-of-the-art methods, especially on the videos containing head movements, improving the Pearson correlation coefficient by 5% compared with the best contrasting method. To summarize, this work significantly strengthens the motion robustness of RPPG, which makes a substantial contribution to the development of video-based heart rate detection.

Rational selection of RGB channels for disease classification based on IPPG technology

The green channel is usually selected as the optimal channel for vital signs monitoring in image photoplethysmography (IPPG) technology. However, some controversies arising from the different penetrability of skin tissue in visible light remain unresolved, i.e., making the optical and physiological information carried by the IPPG signals of the RGB channels inconsistent. This study clarifies that the optimal channels for different diseases are different when IPPG technology is used for disease classification. We further verified this conclusion in the classification model of heart disease and diabetes mellitus based on the random forest classification algorithm. The experimental results indicate that the green channel has a considerably excellent performance in classifying heart disease patients and the healthy with an average Accuracy value of 88.43% and an average F1score value of 93.72%. The optimal channel for classifying diabetes mellitus patients and the healthy is the red channel with an average Accuracy value of 82.12% and the average F1score value of 89.31%. Due to the limited penetration depth of the blue channel into the skin tissue, the blue channel is not as effective as the green and red channels as a disease classification channel. This investigation is of great significance to the development of IPPG technology and its application in disease classification.

VidAF: A Motion-Robust Model for Screening Atrial Fibrillation from Facial Videos

Atrial fibrillation (AF) is the most common arrhythmia, but an estimated 30% of patients with AF are unaware of their conditions. The purpose of this work is to design a model for AF screening from facial videos, with a focus on addressing typical motion disturbances in our real life, such as head movements and expression changes. This model detects a pulse signal from the skin color changes in a facial video by a convolution neural network, incorporating a phase-driven attention mechanism to suppress motion signals in the space domain. It then encodes the pulse signal into discriminative features for AF classification by a coding neural network, using a de-noise coding strategy to improve the robustness of the features to motion signals in the time domain. The proposed model was tested on a dataset containing 1200 samples of 100 AF patients and 100 non-AF subjects. Experimental results demonstrated that VidAF had significant robustness to facial motions, predicting clean pulse signals with the mean absolute error of inter-pulse intervals less than 100 milliseconds. Besides, the model achieved promising performance in AF identification, showing an accuracy of more than 90% in multiple challenging scenarios. VidAF provides a more convenient and cost-effective approach for opportunistic AF screening in the community.

Analysis and improvement of non-contact SpO2 extraction using an RGB webcam

Peripheral oxygen saturation (SpO2), a vital physiological sign employed in clinical care, is commonly obtained by using a contact pulse oximeter. With the rapid popularization of ordinary red-green-blue (RGB) webcams embedded in devices such as smartphones or laptops, there are broad application prospects for exploring techniques for non-contact SpO2 extraction using RGB webcams. However, many issues remain to be solved in the traditional webcam-based SpO2 extraction methods, such as the inherent low signal-to-noise ratio (SNR) of alternating current (AC) components of RGB signals and the potential defects in using RGB signals combination for SpO2 extraction. In this study, we conducted an in-depth examination of the existing research on webcam-based SpO2 extraction techniques, analyzed the practical problems in using them, and explored new ideas to solve the problems. Rather than roughly using the standard deviations (SD) of AC components for calculations, we performed blind source separation for AC components, and then used the energy coefficients retained in the mixed matrix to replace the variables required in the algorithm. Moreover, steady data was selected to compensate for the potential defects in using RGB signals combination. Through these efforts, the anti-noise capability of the algorithm was significantly enhanced, and the related defects were compensated for. The experimental results indicated that the proposed method produced reliable SpO2 estimation that could potentially-with further research-be used in real applications.

Effectiveness of consumer-grade contactless vital signs monitors: a systematic review and meta-analysis

The objective of this systematic review and meta-analysis was to analyze the effectiveness of contactless vital sign monitors that utilize a consumer-friendly camera versus medical grade instruments. A multiple database search was conducted from inception to September 2020. Inclusion criteria were as follows: studies that used a consumer-grade camera (smartphone/webcam) to examine contactless vital signs in adults; evaluated the non-contact device against a reference medical device; and used the participants’ face for measurement. Twenty-six studies were included in the review of which 16 were included in Pearson’s correlation and 14 studies were included in the Bland–Altman meta-analysis. Twenty-two studies measured heart rate (HR) (92%), three measured blood pressure (BP) (12%), and respiratory rate (RR) (12%). No study examined blood oxygen saturation (SpO₂). Most studies had a small sample size (≤ 30 participants) and were performed in a laboratory setting. Our meta-analysis found that consumer-grade contactless vital sign monitors were accurate in comparison to a medical device in measuring HR. Current contactless monitors have limitations such as motion, poor lighting, and lack of automatic face tracking. Currently available consumer-friendly contactless monitors measure HR accurately compared to standard medical devices. More studies are needed to assess the accuracy of contactless BP and RR monitors. Implementation of contactless vital sign monitors for clinical use will require validation in a larger population, in a clinical setting, and expanded to encompass other vital signs including BP, RR, and SpO₂.

Optical Technologies for Improving Healthcare in Low-Resource Settings: introduction to the feature issue

This feature issue of Biomedical Optics Express presents a cross-section of interesting and emerging work of relevance to optical technologies in low-resource settings. In particular, the technologies described here aim to address challenges to meeting healthcare needs in resource-constrained environments, including in rural and underserved areas. This collection of 18 papers includes papers on both optical system design and image analysis, with applications demonstrated for ex vivo and in vivo use. All together, these works portray the importance of global health research to the scientific community and the role that optics can play in addressing some of the world's most pressing healthcare challenges.