Living-Skin Detection Based on Spatio-Temporal Analysis of Structured Light Pattern

Living-skin detection is an important step for imaging photoplethysmography and biometric anti-spoofing. In this paper, we propose a new approach that exploits spatio-temporal characteristics of structured light patterns projected on the skin surface for living-skin detection. We observed that due to the interactions between laser photons and tissues inside a multi-layer skin structure, the frequency-domain sharpness feature of laser spots on skin and non-skin surfaces exhibits clear difference. Additionally, the subtle physiological motion of living-skin causes laser interference, leading to brightness fluctuations of laser spots projected on the skin surface. Based on these two observations, we designed a new living-skin detection algorithm to distinguish skin from non-skin using spatio-temporal features of structured laser spots. Experiments in the dark chamber and Neonatal Intensive Care Unit (NICU) demonstrated that the proposed setup and method performed well, achieving a precision of 85.32%, recall of 83.87%, and F1-score of 83.03% averaged over these two scenes. Compared to the approach that only leverages the property of multilayer skin structure, the hybrid approach obtains an averaged improvement of 8.18% in precision, 3.93% in recall, and 8.64% in F1-score. These results validate the efficacy of using frequency domain sharpness and brightness fluctuations to augment the features of living-skin tissues irradiated by structured light, providing a solid basis for structured light based physiological imaging.

Skin Depolarization for Living-skin Tissue Segmentation: A Quantitative Comparison with PPG Imaging

Conventional living-skin detection methods based on remote photoplethysmographic imaging (PPGI) are not suitable for face anti-spoofing nor skin segmentation for video health monitoring. Therefore, we refer to a novel algorithm based on an entirely new principle for these tasks, i.e., multi-spectral depolarization (MSD) of skin tissues. The effectiveness of MSD remains to be quantified in different application scenarios. In this paper, we conduct laboratory and clinical trials to make a detailed quantitative comparison between two approaches (MSD and PPGI) for skin segmentation. The laboratory result shows that MSD can distinguish the non-live human in the fraud video, whereas the PPGI method failed to identify the liveness. The clinical experiments show that MSD outperforms PPGI in skin pigmentation, with improvements of almost 20%, 15%, and 10% in precision, recall, and F1-score, respectively, highlighting the effectiveness of MSD.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

An overview of current assessment techniques for evaluating cutaneous perfusion in reconstructive surgery

This article provides a comprehensive analysis of modern techniques used in the assessment of cutaneous flaps in reconstructive surgery. It emphasizes the importance of preoperative planning and intra- and perioperative assessment of flap perfusion to ensure successful outcomes. Despite technological advancements, direct clinical assessment remains the gold standard. We categorized assessment techniques into non-invasive and invasive modalities, discussing their strengths and weaknesses. Non-invasive methods, such as acoustic Doppler sonography, near-infrared spectroscopy, hyperspectral imaging thermal imaging, and remote-photoplethysmography, offer accessibility and safety but may sacrifice specificity. Invasive techniques, including contrast-enhanced ultrasound, computed tomography angiography, near-infrared fluorescence angiography with indocyanine green, and implantable Doppler probe, provide high accuracy but introduce additional risks. We emphasize the need for a tailored decision-making process based on specific clinical scenarios, patient characteristics, procedural requirements, and surgeon expertise. It also discusses potential future advancements in flap assessment, including the integration of artificial intelligence and emerging technologies.

Living-skin Detection using Multi-layer Skin Property Perceived by the Structured Light

Living-skin detection has been used to prevent the attack of face fraud in a face recognition system. In this paper, we propose a new concept that exploits the multi-layer structure property of skin for living-skin detection. We observe a significant difference in the blur of the laser spot created by the structured light on the skin and non-skin due to the characteristic properties of laser photons in skin penetration and reflection. Based on this observation, we designed a new living-skin detection algorithm to differentiate skin and non-skin based on the blur detection of laser spots. The experimental results show that the proposed setup and method have a promising performance with an averaged precision of 96.7%, averaged recall of 82.2%, and averaged F1-score of 88.6% on a dataset of 20 adult subjects. This demonstrates the effectiveness of the new concept that uses multi-layer properties of skin tissues for living-skin detection, which may lead to new solutions for face anti-spoofing.

Facial Landmark based BMI Analysis for Pervasive Health Informatics

Body Mass Index (BMI), calculated based on the ratio between a person's height and weight, is a widely used metric for body weight or fatness. In this paper, we investigate the potential of face image-based BMI estimation using an RGB camera. We proposed a simple yet highly reproducible image processing framework that converts an input face image into a BMI value or obesity class (underweight, normal and overweight). In this framework, we explored the options of using 2D or 3D facial landmark models, view angle correction in 2D and 3D, different choices for facial feature extraction (landmark distances or coordinates), and different prediction models (regression or classification) based on shallow machine learning techniques. Our framework was thoroughly validated on two public datasets. The insights of this measurement are discussed, as well as the challenges and limitations, to increase the understanding for future improvement of camera-based BMI estimation. The source code of this study is available at https://github.com/hxfj/Facial-Landmark-based-BMI-Analysis.git.Clinical relevance- This contributes to simpler and more effective daily health management.

Fundamentals of Camera-PPG based Magnetic Resonance Imaging

In Magnetic Resonance Imaging (MRI), cardiac triggering that synchronizes data acquisition with cardiac contractions is an essential technique for acquiring high-quality images. Triggering is typically based on the Electrocardiogram (ECG) signal (e.g. R-peak). Since ECG acquisition involves extra workflow steps like electrode placement and ECG signals are usually disturbed by magnetic fields in high Magnetic Resonance (MR) systems, we explored camera-based photoplethysmography (PPG) as an alternative. We used the in-bore camera of a clinical MR system to investigate the feasibility and challenges of camera-based cardiac triggering. Data from ECG, finger oximeter and camera were synchronously collected. We found that that camera-PPG provides a higher availability of signal (and trigger) measurement, and the PPG signals measured from the forehead show considerably less delay w.r.t. the coupled ECG R-peak than the finger PPG signals in terms of trigger detection. The insights obtained in this study provide a basis for an envisioned system design phase.

Noncontact Physiological Measurement Using a Camera: A Technical Review and Future Directions

Using a camera as an optical sensor to monitor physiological parameters has garnered considerable research interest in biomedical engineering in recent decades. Researchers have explored the use of a camera for monitoring a variety of physiological waveforms, together with the vital signs carried by these waveforms. Most of the obtained waveforms are related to the human respiratory and cardiovascular systems, and in addition of being indicative of overall health, they can also detect early signs of certain diseases. While using a camera for noncontact physiological signal monitoring offers the advantages of low cost and operational ease, it also has the disadvantages such as vulnerability to motion and lack of burden-free calibration solutions in some use cases. This study presents an overview of the existing camera-based methods that have been reported in recent years. It introduces the physiological principles behind these methods, signal acquisition approaches, various types of acquired signals, data processing algorithms, and application scenarios of these methods. It also discusses the technological gaps between the camera-based methods and traditional medical techniques, which are mostly contact-based. Furthermore, we present the manner in which noncontact physiological signal monitoring use has been extended, particularly over the recent years, to more day-to-day aspects of individuals' lives, so as to go beyond the more conventional use case scenarios. We also report on the development of novel approaches that facilitate easier measurement of less often monitored and recorded physiological signals. These have the potential of ushering a host of new medical and lifestyle applications. We hope this study can provide useful information to the researchers in the noncontact physiological signal measurement community.

Impact of makeup on remote-PPG monitoring

Camera-based remote photoplethysmography (remote-PPG) enables contactless measurement of blood volume pulse from the human skin. Skin visibility is essential to remote-PPG as the camera needs to capture the light reflected from the skin that penetrates deep into skin tissues and carries blood pulsation information. The use of facial makeup may jeopardize this measurement by reducing the amount of light penetrating into and reflecting from the skin. In this paper, we conduct an empirical study to thoroughly investigate the impact of makeup on remote-PPG monitoring, in both the visible (RGB) and invisible (Near Infrared, NIR) lighting conditions. The experiment shows that makeup has negative influence on remote-PPG, which reduces the relative PPG strength (AC/DC) at different wavelengths and changes the normalized PPG signature across multiple wavelengths. It makes (i) the pulse-rate extraction more difficult in both the RGB and NIR, although NIR is less affected than RGB, and (ii) the blood oxygen saturation extraction in NIR impossible. To the best of our knowledge, this is the first work that systematically investigate the impact of makeup on camera-based remote-PPG monitoring.

Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review

Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.

A wavelet-based decomposition method for a robust extraction of pulse rate from video recordings

Background

Remote photoplethysmography (rPPG) is a promising optical method for non-contact assessment of pulse rate (PR) from video recordings. In order to implement the method in real-time applications, it is necessary for the rPPG algorithms to be capable of eliminating as many distortions from the pulse signal as possible.

Methods

In order to increase the degrees-of-freedom of the distortion elimination, the dimensionality of the RGB video signals is increased by the wavelet transform decomposition using the generalized Morse wavelet. The proposed Continuous-Wavelet-Transform-based Sub-Band rPPG method (SB-CWT) is evaluated on the 101 publicly available RGB facial video recordings and corresponding reference blood volume pulse (BVP) signals taken from the MMSE-HR database. The performance of the SB-CWT is compared with the performance of the state-of-the-art Sub-band rPPG (SB).

Results

Median signal-to-noise ratio (SNR) for the proposed SB-CWT ranges from 6.63 to 10.39 dB and for the SB from 4.23 to 6.24 dB. The agreement between the estimated PRs from rPPG pulse signals and the reference signals in terms of the coefficients of determination ranges from 0.81 to 0.91 for SB-CWT and from 0.41 to 0.47 for SB. All the correlation coefficients are statistically significant (p < 0.001). The Bland-Altman plots show that mean difference range from 5.37 to 1.82 BPM for SB-CWT and from 22.18 to 18.80 BPM for SB.

Discussion

The results show that the proposed SB-CWT outperforms SB in terms of SNR and the agreement between the estimated PRs from RGB video signals and PRs from the reference BVP signals.

Full video pulse extraction

This paper introduces a new method to automate heart-rate detection using remote photoplethysmography (rPPG). The method replaces the commonly used region of interest (RoI) detection and tracking, and does not require initialization. Instead, it combines a number of candidate pulse-signals computed in the parallel, each biased towards differently colored objects in the scene. The method is based on the observation that the temporally averaged colors of video objects (skin and background) are usually quite stable over time in typical application-driven scenarios, such as the monitoring of a subject sleeping in bed, or an infant in an incubator. The resulting system, called full video pulse extraction (FVP), allows the direct use of raw video streams for pulse extraction. Our benchmark set of diverse videos shows that FVP enables long-term sleep monitoring in visible light and in infrared, and works for adults and neonates. Although we only demonstrate the concept for heart-rate monitoring, we foresee the adaptation to a range of vital signs, thus benefiting the larger video health monitoring field.

Lock-in technique for extraction of pulse rates and associated confidence levels from video

We investigate the practical applicability of video photoplethysmography (VPPG) to extract heart rates of subjects using noncontact color video recordings of human faces collected under typical indoor laboratory conditions using commercial video cameras. Videos were processed following three previously described simple VPPG algorithms to produce a time-varying plethysmographic signal. These time signals were then analyzed using, to the best of our knowledge, a novel, lock-in algorithm that was developed to extract the pulsatile frequency component. A protocol to associate confidence estimates for the extracted heart rates for each video stream is presented. Results indicate that the difference between heart rates extracted using the lock-in technique and gold-standard measurements, for videos with high-confidence metrics, was less than 4 beats per minute. Constraints on video acquisition and processing, including natural subject motion and the total duration of video recorded required for evaluating these confidence metrics, are discussed.

Living-Skin Classification via Remote-PPG

Detecting living-skin tissue in a video on the basis of induced color changes due to blood pulsation is emerging for automatic region of interest localization in remote photoplethysmography (rPPG). However, the state-of-the-art method performing unsupervised living-skin detection in a video is rather time consuming, which is mainly due to the high complexity of its unsupervised online learning for pulse/noise separation. In this paper, we address this issue by proposing a fast living-skin classification method. Our basic idea is to transform the time-variant rPPG-signals into signal shape descriptors called "multiresolution iterative spectrum," where pulse and noise have different patterns enabling accurate binary classification. The proposed technique is a proof-of-concept that has only been validated in lab conditions but not in real clinical conditions. The benchmark, including synthetic and realistic (nonclinical) experiments, shows that it achieves a high detection accuracy better than the state-of-the-art method, and a high detection speed at hundreds of frames per second in MATLAB, enabling real-time living-skin detection.

Spectral-spatial fusion model for robust blood pulse waveform extraction in photoplethysmographic imaging

Photoplethysmographic imaging is an optical solution for non-contact cardiovascular monitoring from a distance. This camera-based technology enables physiological monitoring in situations where contact-based devices may be problematic or infeasible, such as ambulatory, sleep, and multi-individual monitoring. However, automatically extracting the blood pulse waveform signal is challenging due to the unknown mixture of relevant (pulsatile) and irrelevant pixels in the scene. Here, we propose a signal fusion framework, FusionPPG, for extracting a blood pulse waveform signal with strong temporal fidelity from a scene without requiring anatomical priors. The extraction problem is posed as a Bayesian least squares fusion problem, and solved using a novel probabilistic pulsatility model that incorporates both physiologically derived spectral and spatial waveform priors to identify pulsatility characteristics in the scene. Evaluation was performed on a 24-participant sample with various ages (9-60 years) and body compositions (fat% 30.0 ± 7.9, muscle% 40.4 ± 5.3, BMI 25.5 ± 5.2 kg·m^-2). Experimental results show stronger matching to the ground-truth blood pulse waveform signal compared to the FaceMeanPPG (p < 0.001) and DistancePPG (p < 0.001) methods. Heart rates predicted using FusionPPG correlated strongly with ground truth measurements (r² = 0.9952). A cardiac arrhythmia was visually identified in FusionPPG's waveform via temporal analysis.

A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation

In this paper, we propose a conceptually novel algorithm, namely "Spatial Subspace Rotation" (2SR), that improves the robustness of remote photoplethysmography. Based on the assumption of 1) spatially redundant pixel-sensors of a camera, and 2) a well-defined skin mask, our core idea is to estimate a spatial subspace of skin-pixels and measure its temporal rotation for pulse extraction, which does not require skin-tone or pulse-related priors in contrast to existing algorithms. The proposed algorithm is thoroughly assessed on a benchmark dataset containing 54 videos, which includes challenges of various skin-tones, body-motions in complex illuminance conditions, and pulse-rate recovery after exercise. The experimental results show that given a well-defined skin mask, 2SR outperforms the popular ICA-based approach and two state-of-the-art algorithms (CHROM and PBV). When comparing the pulse frequency spectrum, 2SR improves on average the SNR of ICA by 2.22 dB, CHROM by 1.56 dB, and PBV by 1.95 dB. When comparing the instant pulse-rate, 2SR improves on average the Pearson correlation and precision of ICA by 47% and 65%, CHROM by 22% and 23%, and PBV by 21% and 39%. ANOVA confirms the significant improvement of 2SR in peak-to-peak accuracy. The proposed 2SR algorithm is very simple to use and extend, i.e., the implementation only requires a few lines MATLAB code.