Cardiovascular assessment by imaging photoplethysmography - a review

Capturing the pulse: a state-of-the-art review on camera-based jugular vein assessment

Heart failure is associated with a rehospitalisation rate of up to 50% within six months. Elevated central venous pressure may serve as an early warning sign. While invasive procedures are used to measure central venous pressure for guiding treatment in hospital, this becomes impractical upon discharge. A non-invasive estimation technique exists, where the clinician visually inspects the pulsation of the jugular veins in the neck, but it is less reliable due to human limitations. Video and signal processing technologies may offer a high-fidelity alternative. This state-of-the-art review analyses existing literature on camera-based methods for jugular vein assessment. We summarize key design considerations and suggest avenues for future research. Our review highlights the neck as a rich imaging target beyond the jugular veins, capturing comprehensive cardiac signals, and outlines factors affecting signal quality and measurement accuracy. Addressing an often quoted limitation in the field, we also propose minimum reporting standards for future studies.

A Systematic Approach Focused on Machine Learning Models for Exploring the Landscape of Physiological Measurement and Estimation Using Photoplethysmography (PPG)

A non-invasive optical technique known as photoplethysmography (PPG) can be used to provide various physiological measurements and estimations. PPG can be used to assess cardiovascular disease (CVD). Hypertension is a primary risk factor for CVD and a major health problem worldwide. PPG is popular because of its important applications in the evaluation of cardiac activity, variations in venous blood volume, blood oxygen saturation, blood pressure and heart rate variability, etc. In this study, we provide a comprehensive analysis of the extraction of various physiological parameters using PPG waveforms. In addition, we focused on the role of machine learning (ML) models used for the estimation of blood pressure and hypertension classification based on PPG waveforms to make future research and innovation recommendations. This study will be helpful for researchers, scientists, and medical practitioners working on PPG waveforms for monitoring, screening, and diagnosis, as a comparative study or reference.

Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography

INTRODUCTION

Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA.

MATERIAL AND METHODS

The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works.

RESULTS

The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method.

CONCLUSIONS

Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.

Non-contact measurement of neck pulses achieved by imaging micro-motions in the neck skin

We report a method and system of micro-motion imaging (µMI) to realize non-contact measurement of neck pulses. The system employs a 16-bit camera to acquire videos of the neck skin, containing reflectance variation caused by the neck pulses. Regional amplitudes and phases of pulse-induced reflection variation are then obtained by applying a lock-in amplification algorithm to the acquired videos. Composite masks are then generated using the raw frame, amplitude and phase maps, which are then used to guide the extraction of carotid pulse (CP) and jugular vein pulse (JVP) waveforms. Experimental results sufficiently demonstrate the feasibility of our method to extract CP and JVP waves. Compared with conventional methods, the proposed strategy works in a non-contact, non-invasive and self-guidance manner without a need for manual identification to operate, which is important for patient compliance and measurement objectivity. Considering the close relationship between neck pulses and cardiovascular diseases, for example, CA stenosis, the proposed µMI system and method may be useful in the development of early screening tools for potential cardiovascular diseases.

Prediction and prevention system for Severe Acute Respiratory Syndrome CoronaVirus 2 infection by preempting the onset of a cough

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is fast becoming one of the most significant infections worldwide. Of all the causes of SARS-CoV-2 infection, airborne-droplet infection via coughing is the most common. Therefore, if predicting the onset of a cough and preventing infection were possible, it would have a globally positive impact. Here, we describe a new prediction and prevention system for SARS-CoV-2 infection. Usually, air is inhaled prior to coughing, and the cough, which contains droplets of the virus, then occurs during acute exhalation. Therefore, if we can predict the onset of a cough, we can prevent the spread of SARS-CoV-2. At Tohoku University, a diagnosis system for evaluating swallowing motions and peripheral circulation has already been developed, and our prediction system can be integrated into this system. Using three-dimensional human body imaging, we developed a prediction system for preempting the onset of a cough. If we can predict the onset a cough, we can prevent the spread of SARS-CoV-2 infection, by decreasing the shower of virally active airborne droplets. Here, we describe the newly developed prediction and prevention system for SARS-CoV-2 infection that preempts the onset of a cough.Clinical Relevance- If predicting the onset of a cough and preventing infection were possible, it would have a globally positive impact. Here, we describe the newly developed prediction and prevention system for SARS-CoV-2 infection.

Arterial pulsation modulates the optical attenuation coefficient of skin

Photoplethysmographic (PPG) signals arise from the modulation of light reflectivity on the skin due to changes of physiological origin. Imaging plethysmography (iPPG) is a video-based PPG method that can remotely monitor vital signs in a non-invasive manner. iPPG signals result from skin reflectivity modulation. The origin of such reflectivity modulation is still a subject of debate. Here, we have used optical coherence tomography (OCT) imaging to find whether iPPG signals may result from skin optical properties being directly or indirectly modulated by arterial transmural pressure propagation. The light intensity across the tissue was modeled through a simple exponential decay (Beer-Lambert law) to analyze in vivo the modulation of the optical attenuation coefficient of the skin by arterial pulsation. The OCT transversal images were acquired from a forearm of three subjects in a pilot study. The results show that the optical attenuation coefficient of skin changes at the same frequency as the arterial pulsation due to transmural pressure propagation (local ballistographic effect), but we cannot discard the contribution of global ballistographic effects.

Continuous intraoperative perfusion monitoring of free microvascular anastomosed fasciocutaneous flaps using remote photoplethysmography

Flap loss through limited perfusion remains a major complication in reconstructive surgery. Continuous monitoring of perfusion will facilitate early detection of insufficient perfusion. Remote or imaging photoplethysmography (rPPG/iPPG) as a non-contact, non-ionizing, and non-invasive monitoring technique provides objective and reproducible information on physiological parameters. The aim of this study is to establish rPPG for intra- and postoperative monitoring of flap perfusion in patients undergoing reconstruction with free fasciocutaneous flaps (FFCF). We developed a monitoring algorithm for flap perfusion, which was evaluated in 15 patients. For 14 patients, ischemia of the FFCF in the forearm and successful reperfusion of the implanted FFCF was quantified based on the local signal. One FFCF showed no perfusion after reperfusion and devitalized in the course. Intraoperative monitoring of perfusion with rPPG provides objective and reproducible results. Therefore, rPPG is a promising technology for standard flap perfusion monitoring on low costs without the need for additional monitoring devices.

Needs- and user-oriented development of contactless camera-based telemonitoring in heart disease-Results of an acceptance survey from the Home-based Healthcare Project (feasibility project)

Home-based telemonitoring in heart failure patients can reduce all-cause mortality and the relative risk of heart failure-related hospitalization compared to standard care. However, technology use depends, among other things, on user acceptance, making it important to include potential users early in development. In a home-based healthcare project (a feasibility project) a participatory approach was chosen in preparation for future development of contactless camera-based telemonitoring in heart disease patients. The project study patients (n = 18) were surveyed regarding acceptance and design expectations, and acceptance-enhancing measures and design suggestions were then drawn from the results. The study patients corresponded to the target group of potential future users. 83% of respondents showed high acceptance. 17% of those surveyed were more skeptical with moderate or low acceptance. The latter were female, mostly living alone, and without technical expertise. Low acceptance was associated with a higher expectation of effort and lower perception of self-efficacy and lower integratability into daily rhythms. For the design, the respondents found independent operation of the technology very important. Furthermore, concerns were expressed about the new measuring technology, e.g., anxiety about constant surveillance. The acceptance of a new generation of medical technology (contactless camera-based measuring technology) for telemonitoring is already quite high in the surveyed group of older users (60+). Specific user expectations concerning design should be considered during development to increase acceptance by potential users even more.

A real-time heart rate estimation framework based on a facial video while wearing a mask

BACKGROUND: The imaging photoplethysmography (iPPG) method is a non-invasive, non-contact measurement method that uses a camera to detect physiological indicators. On the other hand, wearing a mask has become essential today when COVID-19 is rampant, which has become a new challenge for heart rate (HR) estimation from facial videos recorded by a camera. OBJECTIVE: The aim is to propose an iPPG-based method that can accurately estimate HR with or without a mask. METHODS: First, the facial regions of interest (ROI) were divided into two sub-ROIs, and the original signal was obtained through spatial averaging with different weights according to the result of judging whether wearing a mask or not, and the CDF, which emphasizes the main component signal, was combined with the improved POS suitable for real-time HR estimation to obtain the noise-removed BVP signal. RESULTS: For self-collected data while wearing a mask, MAE, RMSE, and ACC were 1.09 bpm, 1.44 bpm, and 99.08%, respectively. CONCLUSION: Experimental results show that the proposed framework can estimate HR stably in real-time in both cases of wearing a mask or not. This study expands the application range of HR estimation based on facial videos and has very practical value in real-time HR estimation in daily life.

Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey

Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.

Intraoperative visualization and quantitative assessment of tissue perfusion by imaging photoplethysmography: comparison with ICG fluorescence angiography

Intraoperative monitoring of tissue perfusion is of great importance for optimizing surgery and reducing postoperative complications. To date, there is no standard procedure for assessing blood circulation in routine clinical practice. Over the past decade, indocyanine green (ICG) fluorescence angiography is most commonly used for intraoperative perfusion evaluation. Imaging photoplethysmography (iPPG) potentially enables contactless assessment of the blood supply to organs. However, no strong evidence of this potential has been provided so far. Here we report results of a comparative assessment of tissue perfusion obtained using custom-made iPPG and commercial ICG-fluorescence systems during eight different gastrointestinal surgeries. Both systems allow mapping the blood-supply distribution over organs. It was demonstrated for the first time that the quantitative assessment of blood perfusion by iPPG is in good agreement with that obtained by ICG-fluorescence imaging in all surgical cases under study. iPPG can become an objective quantitative monitoring system for tissue perfusion in the operating room due to its simplicity, low cost and no need for any agent injections.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO₂). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO₂, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO₂ are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO₂ is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO₂ monitoring.

Guidance for the Interpretation of Continual Cuffless Blood Pressure Data for the Diagnosis and Management of Hypertension

Hypertension remains the leading risk factor for death worldwide. Despite its prevalence, success of blood pressure (BP) management efforts remains elusive, and part of the difficulty lies in the tool still used to diagnose, measure, and treat hypertension: the sphygmomanometer introduced by Samuel Siegfried Karl von Basch in 1867. In recent years, there has been an explosion of devices attempting to provide estimates of BP without a cuff, overcoming many limitations of cuff-based BP monitors. Unfortunately, the differences in underlying technologies between traditional BP cuffs and newer cuffless devices, as well as hesitancy of changing a well-implemented standard, still generate understandable skepticism about and reluctance to adopt cuffless BP monitors in clinical practice. This guidance document aims to navigate the scientific and medical communities through the types of cuffless devices and present examples of robust BP data collection which are better representations of a person's true BP. It highlights the differences between data collected by cuffless and traditional cuff-based devices and provides an initial framework of interpretation of the new cuffless datasets using, as an example, a CE-marked continual cuffless BP device (Aktiia BP Monitor, Aktiia, Switzerland). Demonstration of novel BP metrics, which have the potential to change the paradigm of hypertension diagnosis and treatment, are now possible for the first time with cuffless BP monitors that provide continual readings over long periods. Widespread adoption of continual cuffless BP monitors in healthcare will require a collaborative and thoughtful process, acknowledging that the transition from a legacy to a novel medical technology will be slow. Finally, this guidance concludes with a call to action to international scientific and expert associations to include cuffless BP monitors in original scientific research and in future versions of guidelines and standards.

On the spatial phase distribution of cutaneous low-frequency perfusion oscillations

Distributed cutaneous tissue blood volume oscillations contain information on autonomic nervous system (ANS) regulation of cardiorespiratory activity as well as dominating thermoregulation. ANS associated with low-frequency oscillations can be quantified in terms of frequencies, amplitudes, and phase shifts. The relative order between these faculties may be disturbed by conditions colloquially termed ‘stress’. Photoplethysmography imaging, an optical non-invasive diagnostic technique provides information on cutaneous tissue perfusion in the temporal and spatial domains. Using the cold pressure test (CPT) in thirteen healthy volunteers as a well-studied experimental intervention, we present a method for evaluating phase shifts in low- and intermediate frequency bands in forehead cutaneous perfusion mapping. Phase shift changes were analysed in low- and intermediate frequency ranges from 0.05 Hz to 0.18 Hz. We observed that time waveforms increasingly desynchronised in various areas of the scanned area throughout measurements. An increase of IM band phase desynchronization observed throughout measurements was comparable in experimental and control group, suggesting a time effect possibly due to overshooting the optimal relaxation duration. CPT triggered an increase in the number of points phase-shifted to the reference that was specific to the low frequency range for phase-shift thresholds defined as π/4, 3π/8, and π/2 rad, respectively. Phase shifts in forehead blood oscillations may infer changes of vascular tone due to activity of various neural systems. We present an innovative method for the phase shift analysis of cutaneous tissue perfusion that appears promising to assess ANS change processes related to physical or psychological stress. More comprehensive studies are needed to further investigate the reliability and physiological significance of findings.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.

Camera-based assessment of cutaneous perfusion strength in a clinical setting

Objective. After skin flap transplants, perfusion strength monitoring is essential for the early detection of tissue perfusion disorders and thus to ensure the survival of skin flaps. Camera-based photoplethysmography (cbPPG) is a non-contact measurement method, using video cameras and ambient light, which provides spatially resolved information about tissue perfusion. It has not been researched yet whether the measurement depth of cbPPG, which is limited by the penetration depth of ambient light, is sufficient to reach pulsatile vessels and thus to measure the perfusion strength in regions that are relevant for skin flap transplants.Approach. We applied constant negative pressure (compared to ambient pressure) to the anterior thighs of 40 healthy subjects. Seven measurements (two before and five up to 90 minutes after the intervention) were acquired using an RGB video camera and photospectrometry simultaneously. We investigated the performance of different algorithmic approaches for perfusion strength assessment, including the signal-to-noise ratio (SNR), its logarithmic components logS and logN, amplitude maps, and the amplitude height of alternating and direct signal components.Main results. We found strong correlations of up tor=0.694 (p<0.001) between photospectrometric measurements and all cbPPG parameters except SNR when using the green color channel. The transfer of cbPPG signals to POS, CHROM, and O3C did not lead to systematic improvements. However, for direct signal components, the transformation to O3C led to correlations of up tor=0.744 (p<0.001) with photospectrometric measurements.Significance. Our results indicate that a camera-based perfusion strength assessment in tissue with deep-seated pulsatile vessels is possible.

Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera

This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.

Non-Contact Automatic Vital Signs Monitoring of Infants in a Neonatal Intensive Care Unit Based on Neural Networks

Infants with fragile skin are patients who would benefit from non-contact vital sign monitoring due to the avoidance of potentially harmful adhesive electrodes and cables. Non-contact vital signs monitoring has been studied in clinical settings in recent decades. However, studies on infants in the Neonatal Intensive Care Unit (NICU) are still limited. Therefore, we conducted a single-center study to remotely monitor the heart rate (HR) and respiratory rate (RR) of seven infants in NICU using a digital camera. The region of interest (ROI) was automatically selected using a convolutional neural network and signal decomposition was used to minimize the noise artefacts. The experimental results have been validated with the reference data obtained from an ECG monitor. They showed a strong correlation using the Pearson correlation coefficients (PCC) of 0.9864 and 0.9453 for HR and RR, respectively, and a lower error rate with RMSE 2.23 beats/min and 2.69 breaths/min between measured data and reference data. A Bland-Altman analysis of the data also presented a close correlation between measured data and reference data for both HR and RR. Therefore, this technique may be applicable in clinical environments as an economical, non-contact, and easily deployable monitoring system, and it also represents a potential application in home health monitoring.

Adaptive Gaussian Mixture Model Driven Level Set Segmentation for Remote Pulse Rate Detection

This paper presents an approach for pulse rate extraction from videos. The core of the presented approach is a novel method to segment and track a suitable region of interest (ROI). The proposed method combines level sets with subject-individual Gaussian Mixture Models to yield a time varying ROI. The ROI builds up from multiple homogeneous skin areas under constraints regarding the area and contour length of the ROI. Together with state of the art signal processing methods our approach yields an Mean Average Error (MAE) of 2.3 bpm, 1.4 bpm and 2.7 bpm on own data, the PURE database and the UBFC-rPPG database, respectively. Therewith, our method performs equal or better compared to widely used approaches (e.g. the KLT tracker instead of the proposed image processing yields an MAE of 2.6 bpm, 2.6 bpm and 4.4 bpm). Such results and the 2nd place with a MAE of 7.92 bpm in the 1st Challenge on Remote Physiological Signal Sensing prove the applicability of the proposed method. The taken approach, however, bears further potential for optimization in the context of photoplethysmography imaging and should be transferable to other segmentation tasks as well.

A Heart Rate Monitoring Framework for Real-World Drivers Using Remote Photoplethysmography

Remote photoplethysmography (rPPG) is an unobtrusive solution to heart rate monitoring in drivers. However, disturbances that occur during driving such as driver behavior, motion artifacts, and illuminance variation complicate the monitoring of heart rate. Faced with disturbance, one commonly used assumption is heart rate periodicity (or spectrum sparsity). Several methods improve stability at the expense of tracking sensitivity for heart rate variation. Based on statistical signal processing (SSP) and Monte Carlo simulations, the outlier probability is derived and ADaptive spectral filter banks (AD) is proposed as a new algorithm which provides an explicable tuning option for spectral filter banks to strike a balance between robustness and sensitivity in remote monitoring for driving scenarios. Moreover, we construct a driving database containing over 23 hours of data to verify the proposed algorithm. The influence on rPPG from driver habits (both amateurs and professionals), vehicle types (compact cars and buses), and routes are also evaluated. In comparison to state-of-the-art rPPG for driving scenarios, the mean absolute error in the Passengers, Compact Cars, and Buses scenarios is 3.43, 7.85, and 5.02 beats per minute, respectively. Moreover, AD also won the top third place in the first challenge on remote physiological signal sensing (RePSS) with relative low computational complexity.

Research on the combination of color channels in heart rate measurement based on photoplethysmography imaging

SIGNIFICANCE

The measurement of human vital signs based on photoplethysmography imaging (PPGI) can be severely affected by the interference of various factors in the measurement process; therefore, a lot of complex signal processing techniques are used to remove the influence of the interference.

AIM

We comprehensively analyze several methods for color channel combination in the color spaces currently used in PPGI and determine the combination method that can improve the quality of the pulse signal, which results in a modified plane-orthogonal-to-skin based method (POS).

APPROACH

Based on the analysis of the previous studies, 13 methods for color channel combination in the different color spaces, which can be seen as having potential abilities in measuring vital signs, were compared by employing the average value of signal-to-noise ratio (SNR) and the box-plot in the public databases UBFC-RPPG and PURE. In addition, the pulse signal was extracted through the dual-color space transformation (sRGB → intensity normalized RGB → YCbCr) and fine-tuning on the CbCr plane.

RESULTS

Among the 13 methods for color channel combination, the signal extracted by the Cb+Cr combination in the YCbCr color space includes the most pulse information. Furthermore, the average SNR of the modified POS for all the used databases is improved by 69.3% compared to POS.

CONCLUSIONS

The methods using prior knowledge are not only simple to calculate but can significantly increase the SNR, which will provide a great help in the practical use of vital sign measurements based on PPGI.

An iPPG-Based Device for Pervasive Monitoring of Multi-Dimensional Cardiovascular Hemodynamics

Hemodynamic activities, as an essential measure of physiological and psychological characteristics, can be used for cardiovascular and cerebrovascular disease detection. Photoplethysmography imaging (iPPG) can be applied for such purposes with non-contact advances, however, most cardiovascular hemodynamics of iPPG systems are developed for laboratory research, which limits the application in pervasive healthcare. In this study, a video-based facial iPPG detecting equipment was devised to provide multi-dimensional spatiotemporal hemodynamic pulsations for applications with high portability and self-monitoring requirements. A series of algorithms have also been developed for physiological indices such as heart rate and breath rate extraction, facial region analysis, and visualization of hemodynamic pulsation distribution. Results showed that the new device can provide a reliable measurement of a rich range of cardiovascular hemodynamics. Combined with the advanced computing techniques, the new non-contact iPPG system provides a promising solution for user-friendly pervasive healthcare.

Fast body part segmentation and tracking of neonatal video data using deep learning

Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates' body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 × 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications. Graphical Abstract This work presents the development of a customized, real-time capable Deep Learning architecture for segmenting of neonatal videos recorded in the intensive care unit. In addition to hand-annotated data, transfer learning is exploited to improve performance.

Assessment of a standalone photoplethysmography (PPG) algorithm for detection of atrial fibrillation on wristband-derived data

INTRODUCTION

Atrial fibrillation (AF) is the most common cardiac arrhythmia in the developed world. Using photoplethysmography (PPG) and software algorithms, AF can be detected with high accuracy using smartphone camera-derived data. However, reports of diagnostic accuracy of standalone algorithms using wristband-derived PPG data are sparse, while this provides a means to perform long-term AF screening and monitoring. This study evaluated the diagnostic accuracy of a well-known standalone algorithm using wristband-derived PPG data.

MATERIALS AND METHODS

Subjects recruited from a community senior care organization were instructed to wear the Wavelet PPG wristband on one arm and the Alivecor KardiaBand one-lead-ECG wristband on the other. Three consecutive measurements (duration per measurement: 60 s for PPG and 30 s for one-lead ECG) were performed with both devices, simultaneously. The PPG data were analyzed by the Fibricheck standalone algorithm and the ECG data by the Kardia algorithm. The results were compared to a reference standard (interpretation of the one-lead ECG by two independent cardiologists).

RESULTS

A total of 180 PPGs and one-lead ECGs were recorded in 60 subjects, with a mean age of 70±17. AF was identified in 6 (10%) of the users, two users (3%) were not classifiable by the PPG algorithm and 1 user (2%) was not classifiable by the one-lead ECG algorithm. The diagnostic performance (sensitivity/specificity/positive predictive value/negative predictive value/accuracy) on user level was 100/96/75/100/97% for the PPG wristband and 100/98/86/100/98% for the one-lead ECG wristband.

CONCLUSIONS

In a small real-world cohort of elderly people, the standalone Fibricheck AF algorithm can accurately detect AF using Wavelet wristband-derived PPG data. Results are comparable to the Alivecor Kardia one-lead ECG device, with an acceptable unclassifiable/bad quality rate. This opens the door for long-term AF screening and monitoring.

Biometric Signals Estimation Using Single Photon Camera and Deep Learning

The problem of performing remote biomedical measurements using just a video stream of a subject face is called remote photoplethysmography (rPPG). The aim of this work is to propose a novel method able to perform rPPG using single-photon avalanche diode (SPAD) cameras. These are extremely accurate cameras able to detect even a single photon and are already used in many other applications. Moreover, a novel method that mixes deep learning and traditional signal analysis is proposed in order to extract and study the pulse signal. Experimental results show that this system achieves accurate results in the estimation of biomedical information such as heart rate, respiration rate, and tachogram. Lastly, thanks to the adoption of the deep learning segmentation method and dependability checks, this method could be adopted in non-ideal working conditions—for example, in the presence of partial facial occlusions.

Comparison of Visible and Infrared Video Plethysmography Captured from Different Regions of the Human Face

Recently, video plethysmography (VPG) - a heart rate estimation technique using a video camera - has gained significant attention. Most studies of VPG have used a visible RGB camera; only a limited number of studies investigating near-infrared light (wavelength 750-2500 nm), which can be used even in a dark environment, have been performed. The purpose of this study was to investigate the differences between VPG data collected using visible light (VPG_VIS) or near-infrared light (VPG_NIR) from four facial areas (forehead, right cheek, left cheek, and nose). An experiment was conducted to obtain both VPG_VIS and VPG_NIR simultaneously by alternately irradiating the face with NIR and VIS lights. Experimental results showed that the root mean squared error of heart rate estimated using VPG_NIR was 1 bpm higher than that of VPG_VIS. However, contrary to our expectations, the power of the heartbeat-related component included in VPG_NIR was not reduced despite the absorbance of hemoglobin in the NIR light range being 1/100 of that in the VIS light range. This result supports the hypothesis that a main factor in the generation of VPG waves was change in the optical properties caused by blood vessels compressing the subcutaneous tissue and the venous bed. Additionally, the accuracy of the heart rate estimation using VPG tended to be high when the nose was set as the ROI. This result was likely associated with the anatomical structure of the nose.

Pulse decomposition analysis in photoplethysmography imaging

OBJECTIVE

Photoplethysmography imaging (PPGI) has gained immense attention over the last few years but only a few works have addressed morphological analysis so far. Pulse wave decomposition (PWD), i.e. the decomposition of a pulse wave by a varying number of kernels, allows for such analyses. This work investigates the applicability of PWD algorithms in the context of PPGI.

APPROACH

We used simulated and experimental data to compare various PWD algorithms from the literature regarding their robustness against noise and motion artifacts while preserving morphological information as well as regarding their ability to reveal physiological changes by PPGI.

MAIN RESULTS

Our experiments prove that algorithms that combine Gamma and Gaussian distributions outperform other choices. Further, algorithms with two kernels exhibit the highest robustness against noise and motion artifacts (improvement in [Formula: see text] of 14.09 %) while preserving the morphology similarly to algorithms using more kernels. Lastly, we showed that PWD can reveal physiological changes upon distal stimuli by PPGI.

SIGNIFICANCE

This work proves the feasibility of pulse decomposition analysis in PPGI, particularly for algorithms with a low number of kernels, and opens up novel applications for PPGI. Not only for PPGI but for future research on PWD in general, our findings have importance as they elucidate differences between PWD algorithms and emphasize the importance of using initial values. To support such future research, we have released the algorithms and simulated data to the public.

Association of remote imaging photoplethysmography and cutaneous perfusion in volunteers

Remote imaging photoplethysmography (iPPG) senses the cardiac pulse in outer skin layers and is responsive to mean arterial pressure and pulse pressure in critically ill patients. Whether iPPG is sufficiently sensitive to monitor cutaneous perfusion is not known. This study aimed at determining the response of iPPG to changes in cutaneous perfusion measured by  Laser speckle imaging (LSI). Thirty-seven volunteers were engaged in a cognitive test known to evoke autonomic nervous activity and a Heat test. Simultaneous measurements of iPPG and LSI were taken at baseline and during cutaneous perfusion challenges. A perfusion index (PI) was calculated to assess iPPG signal strength. The response of iPPG to the challenges and its relation to LSI were determined. PI of iPPG significantly increased in response to autonomic nervous stimuli and to the Heat test by 5.8% (p = 0.005) and 11.1% (p < 0.001), respectively. PI was associated with LSI measures of cutaneous perfusion throughout experiments (p < 0.001). iPPG responses to study task correlated with those of LSI (r = 0.62, p < 0.001) and were comparable among subjects. iPPG is sensitive to autonomic nervous activity in volunteers and is closely associated with cutaneous perfusion.

Photoplethysmography imaging:camera performance evaluation by means of an optoelectronic skin perfusion phantom

OBJECTIVE

Photoplethysmography imaging (PPGI) is a promising contactless camera-based method of non-invasive cardiovascular diagnostics. To achieve the best results, it is important to choose the most suitable camera for a specific application. The settings of the camera influence the quality of the detected signal.

APPROACH

The standard (European Machine Vision Association 2016 EMVA Standard 1288-Standard for Characterization of Image Sensors and Cameras pp 1-39 (available at: https://www.emva.org/wp-content/uploads/EMVA1288-3.1a.pdf)) for evaluating the imaging performance of machine vision cameras (MVC) helps at the initial decision of the sensor, but the camera should always be tested in terms of usability for a specific application. So far, PPGI lacks a standardized measurement scenario for evaluating the performance of individual cameras. For this, we realized a controllable optoelectronic phantom with artificial silicone skin allowing reproducible tests of cameras to verify their suitability for PPGI. The entire system is housed in a light-tight box. We tested an MVC, a digital single-lens reflex camera (DSLR) camera and a webcam. Each camera varies in used technology and price.

MAIN RESULTS

We simulated real PPGI measurement conditions simulating the ratio of pulse (AC) and non-pulse (DC) component of the photoplethysmographic signal and achieved AC/DC ratios of 0.5 % on average. An additional OLED panel ensures proper DC providing reproducible measurement conditions. We evaluated the signal morphological features, amplitude spectrum, signal-to-noise ratio (SNR) and spatially dependent changes of simulated subcutaneous perfusion. Here, the MVC proved to be the most suitable device. A DSLR is also suitable for PPGI, but a larger smoothing kernel is required to obtain a perfusion map. The webcam, as the weakest contender, proved to be very susceptible to any inhomogeneous illumination of the examined artificial skin surface. However, it is still able to detect cardiac rhythm.

SIGNIFICANCE

The result of our work is an optoelectronic phantom for reproducible testing of PPGI camera performance in terms of signal quality and measurement equipment costs.

Estimation of Absolute Blood Pressure Using Video Images Captured at Different Heights from the Heart

The risk of cardiovascular diseases is related to the absolute level of blood pressure as well as its fluctuation while sleeping or during daily activities. To assess the risk, a simpler method to monitor daily blood pressure is desirable. In recent years, there has been a focus on developing a method to obtain pulse waves from video images of the human body. This is a promising technique to acquire biometric information without contact. In this study, we propose a new method to estimate the absolute level of blood pressure by using two video images of human hands captured at different heights from the heart. We focus on the amplitude difference of pulse waves obtained from the video images and derive an equation to estimate blood pressure based on the relationship between the internal pressure and the cross-sectional area of the blood vessel. The accuracy of the estimation was evaluated using data obtained from 5 healthy subjects performing cycling exercises that change their blood pressure. The average value of the root mean square error between the real value and the estimated value was 25.7 mmHg, while that of correlation coefficient was 0.66. There were large individual differences, particularly in the estimation of the absolute value of blood pressure. This result suggests the need for individual correction of the compliance curve, which represents the relationship between the internal pressure and the cross-sectional area of the blood vessel.

Pulse decomposition analysis in camera-based photoplethysmography

Imaging photoplethysmography (iPPG) is an interesting alternative to laser speckle contrast imaging for the analysis of spatio-temporal patterns in the cutaneous microcirculation. Recent years have witnessed the development of sophisticated techniques for the non-invasive extraction of vascular-related features. These techniques, referred to as pulse decomposition algorithms (PDA), most often involve the analysis of photoplethysmographic waves. This study validated the use of a multi-Gaussian (PDA) for the automatic mapping of iPPG pulse waveforms acquired with a standard camera. We show that iPPG-based PDA can reveal differences in skin perfusion in response to cold stimuli. The study thus proves the potential for morphological analyses of the iPPG pulse waveform.

Noise Reduction Technique for Single-Color Video Plethysmography Using Singular Spectrum Analysis

Recently, a contactless method for measuring a biological signal using a video camera has garnered attention. Especially, video plethysmography, a technique for obtaining a pulse wave from a video, is useful for managing the health of people on a daily basis. However, any body movement of a person subjected to the measurement leads to the generation of irregular noise in video plethysmography and reduces the accuracy of the recorded biological information, e.g., heart rate, during the measurement. Blind source separation is a popular technique for eliminating noise from the results of video plethysmography comprising different multiple-color channels. However, it is difficult to apply this technique to a single-color video such as a near-infrared video. Herein, a new method that combines singular spectrum analysis with the circular autocorrelation function is introduced to eliminate irregular noise in single-color video plethysmography. Applying the proposed method on videos collected from 39 individuals improved the estimation accuracy of instantaneous heart rate by approximately 44% over a conventional method using a linear filter. Furthermore, the proposed method also enabled more precise estimations of the heart rate than that achieved using multi-color video plethysmography.

Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review

Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Remote pulse rate measurement from facial video has gained particular attention over the last few years. Research exhibits significant advancements and demonstrates that common video cameras correspond to reliable devices that can be employed to measure a large set of biomedical parameters without any contact with the subject. A new framework for measuring and mapping pulse rate from video is presented in this pilot study. The method, which relies on convolutional 3D networks, is fully automatic and does not require any special image preprocessing. In addition, the network ensures concurrent mapping by producing a prediction for each local group of pixels. A particular training procedure that employs only synthetic data is proposed. Preliminary results demonstrate that this convolutional 3D network can effectively extract pulse rate from video without the need for any processing of frames. The trained model was compared with other state-of-the-art methods on public data. Results exhibit significant agreement between estimated and ground-truth measurements: the root mean square error computed from pulse rate values assessed with the convolutional 3D network is equal to 8.64 bpm, which is superior to 10 bpm for the other state-of-the-art methods. The robustness of the method to natural motion and increases in performance correspond to the two main avenues that will be considered in future works.

Modeling photoplethysmographic signals in camera-based perfusion measurements: optoelectronic skin phantom

The remote acquisition of photoplethysmographic (PPG) signals via a video camera, also known as photoplethysmography imaging (PPGI), is not yet standardized. In general, PPGI is investigated with test persons in a laboratory setting. While these in-vivo tests have the advantage of generating real-life data, they suffer from the lack of repeatability and are comparatively effort-intensive because human subjects are required. Consequently, studying changes in signal morphology, for example, due to aging or pathological effects, is practically impossible. As a tool to study these effects, a hardware PPG simulator has been developed: this is a phantom which simulates and generates both 1D and locally resolved 2D optical PPG signals. Here, we demonstrate that it is possible to generate PPG-like signals with various signal morphologies by means of a purely optoelectronic setup, namely an LED array, and to analyze them by means of PPGI. Signals extracted via a camera show good agreement with simulated generated signals. In fact, the first phantom design is suitable to demonstrate this qualitatively.

A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

OBJECTIVES

Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting "outliers".

METHODS

We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed.

RESULTS

Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning.

CONCLUSION

Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.

Local Pulse Wave Velocity: Theory, Methods, Advancements, and Clinical Applications

Local pulse wave velocity (PWV) is evolving as one of the important determinants of arterial hemodynamics, localized vessel stiffening associated with several pathologies, and a host of other cardiovascular events. Although PWV was introduced over a century ago, only in recent decades, due to various technological advancements, has emphasis been directed toward its measurement from a single arterial section or from piecewise segments of a target arterial section. This emerging worldwide trend in the exploration of instrumental solutions for local PWV measurement has produced several invasive and noninvasive methods. As of yet, however, a univocal opinion on the ideal measurement method has not emerged. Neither have there been extensive comparative studies on the accuracy of the available methods. Recognizing this reality, makes apparent the need to establish guideline-recommended standards for the measurement methods and reference values, without which clinical application cannot be pursued. This paper enumerates all major local PWV measurement methods while pinpointing their salient methodological considerations and emphasizing the necessity of global standardization. Further, a summary of the advancements in measuring modalities and clinical applications is provided. Additionally, a detailed discussion on the minimally explored concept of incremental local PWV is presented along with suggestions of future research questions.

Waveform Analysis for Camera-based Photoplethysmography Imaging

With the advent of sensitive and affordable cameras, classical contact-based photoplethysmography (PPG) could be enhanced to the spatial domain. Cost-efficient cameras are available in everyday items such as smartphones or computer webcams. The PPG signal, blood volume changes in the vascularity, can be measured remotely by using the camera as a 2-D-PPG detector. However, the evaluation of the extracted signals has mostly been limited to the pulse rate and sometimes the systolic amplitude. In this work, we motivate to generate images and video sequences based on features from the PPG waveform commonly not extracted via cameras. This is achieved by calculating the features for timeseries extracted from an evenly spaced grid of virtual PPG sensors. We briefly discuss the adaption of conventional PPG algorithms to camera-based PPG imaging (PPGI). The extracted parameters are associated with vessel properties and thus, mapping these to images could lead to enhanced vascular diagnostics. In this work, we test the feasibility of the mapping approach: we present the preliminary results gathered from the analysis of two videos of lab experiments with healthy subjects.