A CMOS Camera-Based Pulse Oximetry Imaging System

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

Infant heart-rate measurement and oxygen desaturation detection with a digital video camera using imaging photoplethysmography

OBJECTIVE

To assess the feasibility of using an ordinary digital video camera to measure heart rate and detect oxygen desaturations in healthy infants.

STUDY DESIGN

Heart rate and oxygen saturation were measured with a video camera by detecting small color changes in 28 infants' foreheads and compared with standard pulse oximetry measures. Multivariable regression examined the relationship between infant characteristics and heart-rate measurement precision.

RESULTS

The average bias of camera heart-rate measures was -4.2 beats per minute (BPM) and 95% limits of agreement were ±43.8 BPM. Desaturations detected by camera were 75% sensitive (15/20) and had a positive predictive value of 20% (15/74). Lower birth-weight was independently correlated with more precise heart-rate measures (8.05 BPM per kg, [95% CI 0.764-15.3]).

CONCLUSIONS

A digital video camera provides accurate but imprecise measures of infant heart rate and may provide a rough screening tool for oxygen desaturations.

Noncontact Physiological Measurement Using a Camera: A Technical Review and Future Directions

Using a camera as an optical sensor to monitor physiological parameters has garnered considerable research interest in biomedical engineering in recent decades. Researchers have explored the use of a camera for monitoring a variety of physiological waveforms, together with the vital signs carried by these waveforms. Most of the obtained waveforms are related to the human respiratory and cardiovascular systems, and in addition of being indicative of overall health, they can also detect early signs of certain diseases. While using a camera for noncontact physiological signal monitoring offers the advantages of low cost and operational ease, it also has the disadvantages such as vulnerability to motion and lack of burden-free calibration solutions in some use cases. This study presents an overview of the existing camera-based methods that have been reported in recent years. It introduces the physiological principles behind these methods, signal acquisition approaches, various types of acquired signals, data processing algorithms, and application scenarios of these methods. It also discusses the technological gaps between the camera-based methods and traditional medical techniques, which are mostly contact-based. Furthermore, we present the manner in which noncontact physiological signal monitoring use has been extended, particularly over the recent years, to more day-to-day aspects of individuals' lives, so as to go beyond the more conventional use case scenarios. We also report on the development of novel approaches that facilitate easier measurement of less often monitored and recorded physiological signals. These have the potential of ushering a host of new medical and lifestyle applications. We hope this study can provide useful information to the researchers in the noncontact physiological signal measurement community.

Noncontact Sensing of Contagion

The World Health Organization (WHO) has declared COVID-19 a pandemic. We review and reduce the clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020. Vital signs associated with respiratory distress and fever, coughing, and visible infections have been reported. Fever screening by temperature monitoring is currently popular. However, improved noncontact detection is sought. Vital signs including heart rate and respiratory rate are affected by the condition. Cough, fatigue, and visible infections are also reported as common symptoms. There are non-contact methods for measuring vital signs remotely that have been shown to have acceptable accuracy, reliability, and practicality in some settings. Each has its pros and cons and may perform well in some challenges but be inadequate in others. Our review shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date, thermal cameras due to their potential to measure all likely symptoms on a single camera, especially temperature, and video cameras due to their availability, cost, adaptability, and compatibility. Substantial supply chain disruptions during the pandemic and the widespread nature of the problem means that cost-effectiveness and availability are important considerations.

An iPPG-Based Device for Pervasive Monitoring of Multi-Dimensional Cardiovascular Hemodynamics

Hemodynamic activities, as an essential measure of physiological and psychological characteristics, can be used for cardiovascular and cerebrovascular disease detection. Photoplethysmography imaging (iPPG) can be applied for such purposes with non-contact advances, however, most cardiovascular hemodynamics of iPPG systems are developed for laboratory research, which limits the application in pervasive healthcare. In this study, a video-based facial iPPG detecting equipment was devised to provide multi-dimensional spatiotemporal hemodynamic pulsations for applications with high portability and self-monitoring requirements. A series of algorithms have also been developed for physiological indices such as heart rate and breath rate extraction, facial region analysis, and visualization of hemodynamic pulsation distribution. Results showed that the new device can provide a reliable measurement of a rich range of cardiovascular hemodynamics. Combined with the advanced computing techniques, the new non-contact iPPG system provides a promising solution for user-friendly pervasive healthcare.

An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis

Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sensitivities together with the number of bands are the main focus. An optical model of skin is used to generate a specialized training set to improve spectral reconstruction. The quantitative comparison of the cameras is based on reconstruction of measured skin spectra, colorimetric accuracy, and oxygenation level estimation differences. Specific spectral sensitivity shapes influence the results directly and a 9-channel camera performed best regarding the spectral reconstruction metrics. Sensitivities at key wavelengths influence the performance of oxygenation level estimation the strongest. The proposed framework allows to compare spectral filter array cameras and can guide their application-specific development.

Doubling the Signal Quality of Smartphone Camera Pulse Oximetry Using the Display Screen as a Controllable Selective Light Source

Recent smartphones have the potential to bring camera oximetry to everyone using their powerful sensors and the capability to process measurements in real-time, potentially augmenting people's lives through always-available oximetry monitoring everywhere. The challenge of camera oximetry on smartphones is the low contrast between reflections from oxyhemoglobin and deoxyhemoglobin. In this paper, we show that this is the result of using the camera flash for illumination, which illuminates evenly across bands than thus leads to the diminished contrast in reflections. Instead, we propose capturing pulse using the front-facing camera and illuminating with the phone's display, a selective illuminant in the red, green, and blue band. We evaluate the spectral characteristics of the phone display using a spectroradiometer in a controlled experiment, convolve them with the sensitivity curves of the phone's camera, and show that the screen's narrow-band display illumination increases the contrast between the reflections in the desired bands by a factor of two compared to flash illumination. Our preliminary evaluation showed further support for our approach and findings.

Algorithms for Monitoring Heart Rate and Respiratory Rate From the Video of a User's Face

Smartphone cameras can measure heart rate (HR) by detecting pulsatile photoplethysmographic (iPPG) signals from post-processing the video of a subject’s face. The iPPG signal is often derived from variations in the intensity of the green channel as shown by Poh et. al. and Verkruysse et. al.. In this pilot study, we have introduced a novel iPPG method where by measuring variations in color of reflected light, i.e., Hue, and can therefore measure both HR and respiratory rate (RR) from the video of a subject’s face. This paper was performed on 25 healthy individuals (Ages 20–30, 15 males and 10 females, and skin color was Fitzpatrick scale 1–6). For each subject we took two 20 second video of the subject’s face with minimal movement, one with flash ON and one with flash OFF. While recording the videos we simultaneously measuring HR using a Biosync B-50DL Finger Heart Rate Monitor, and RR using self-reporting. This paper shows that our proposed approach of measuring iPPG using Hue (range 0–0.1) gives more accurate readings than the Green channel. HR/Hue (range 0–0.1) (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$r=0.9201$ \end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p$ \end{document}-value = 4.1617, and RMSE = 0.8887) is more accurate compared with HR/Green (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$r=0.4916$ \end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p$ \end{document}-value = 11.60172, and RMSE = 0.9068). RR/Hue (range 0–0.1) (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$r=0.6575$ \end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p$ \end{document}-value = 0.2885, and RMSE = 3.8884) is more accurate compared with RR/Green (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$r=0.3352$ \end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p$ \end{document}-value = 0.5608, and RMSE = 5.6885). We hope that this hardware agnostic approach for detection of vital signals will have a huge potential impact in telemedicine, and can be used to tackle challenges, such as continuous non-contact monitoring of neo-natal and elderly patients. An implementation of the algorithm can be found at https://pulser.thinkbiosolution.com

New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring

Finger-oximeters are ubiquitously used for patient monitoring in hospitals worldwide. Recently, remote measurement of arterial blood oxygenation (SpO₂) with a camera has been demonstrated. Both contact and remote measurements, however, require the subject to remain static for accurate SpO₂ values. This is due to the use of the common ratio-of-ratios measurement principle that measures the relative pulsatility at different wavelengths. Since the amplitudes are small, they are easily corrupted by motion-induced variations. We introduce a new principle that allows accurate remote measurements even during significant subject motion. We demonstrate the main advantage of the principle, i.e. that the optimal signature remains the same even when the SNR of the PPG signal drops significantly due to motion or limited measurement area. The evaluation uses recordings with breath-holding events, which induce hypoxemia in healthy moving subjects. The events lead to clinically relevant SpO₂ levels in the range 80–100%. The new principle is shown to greatly outperform current remote ratio-of-ratios based methods. The mean-absolute SpO₂-error (MAE) is about 2 percentage-points during head movements, where the benchmark method shows a MAE of 24 percentage-points. Consequently, we claim ours to be the first method to reliably measure SpO₂ remotely during significant subject motion.

Imaging photoplethysmography for clinical assessment of cutaneous microcirculation at two different depths

The feasibility of bispectral imaging photoplethysmography (iPPG) system for clinical assessment of cutaneous microcirculation at two different depths is proposed. The iPPG system has been developed and evaluated for in vivo conditions during various tests: (1) topical application of vasodilatory liniment on the skin, (2) skin local heating, (3) arterial occlusion, and (4) regional anesthesia. The device has been validated by the measurements of a laser Doppler imager (LDI) as a reference. The hardware comprises four bispectral light sources (530 and 810 nm) for uniform illumination of skin, video camera, and the control unit for triggering of the system. The PPG signals were calculated and the changes of perfusion index (PI) were obtained during the tests. The results showed convincing correlations for PI obtained by iPPG530 nm and LDI at (1) topical liniment (r = 0.98) and (2) heating (r = 0.98) tests. The topical liniment and local heating tests revealed good selectivity of the system for superficial microcirculation monitoring. It is confirmed that the iPPG system could be used for assessment of cutaneous perfusion at two different depths, morphologically and functionally different vascular networks, and thus utilized in clinics as a cost-effective alternative to the LDI.

Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-Wavelength Imaging System

We present a noncontact method to monitor blood oxygen saturation (SpO2). The method uses a CMOS camera with a trigger control to allow recording of photoplethysmography (PPG) signals alternatively at two particular wavelengths, and determines the SpO2 from the measured ratios of the pulsatile to the nonpulsatile components of the PPG signals at these wavelengths. The signal-to-noise ratio (SNR) of the SpO2 value depends on the choice of the wavelengths. We found that the combination of orange (λ = 611 nm) and near infrared (λ = 880 nm) provides the best SNR for the noncontact video-based detection method. This combination is different from that used in traditional contact-based SpO 2 measurement since the PPG signal strengths and camera quantum efficiencies at these wavelengths are more amenable to SpO2 measurement using a noncontact method. We also conducted a small pilot study to validate the noncontact method over an SpO2 range of 83%-98%. This study results are consistent with those measured using a reference contact SpO2 device ( r = 0.936, ). The presented method is particularly suitable for tracking one's health and wellness at home under free-living conditions, and for those who cannot use traditional contact-based PPG devices.

Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging

Photoplethysmography (PPG) is a noninvasive optical technique for detecting microvascular blood volume changes in tissues. Its ease of use, low cost and convenience make it an attractive area of research in the biomedical and clinical communities. Nevertheless, its single spot monitoring and the need to apply a PPG sensor directly to the skin limit its practicality in situations such as perfusion mapping and healing assessments or when free movement is required. The introduction of fast digital cameras into clinical imaging monitoring and diagnosis systems, the desire to reduce the physical restrictions, and the possible new insights that might come from perfusion imaging and mapping inspired the evolution of the conventional PPG technology to imaging PPG (IPPG). IPPG is a noncontact method that can detect heart-generated pulse waves by means of peripheral blood perfusion measurements. Since its inception, IPPG has attracted significant public interest and provided opportunities to improve personal healthcare. This study presents an overview of the wide range of IPPG systems currently being introduced along with examples of their application in various physiological assessments. We believe that the widespread acceptance of IPPG is happening, and it will dramatically accelerate the promotion of this healthcare model in the near future.

A novel method based on two cameras for accurate estimation of arterial oxygen saturation

BACKGROUND

Photoplethysmographic imaging (PPGi) that is based on camera allows acquiring photoplethysmogram and measuring physiological parameters such as pulse rate, respiration rate and perfusion level. It has also shown potential for estimation of arterial oxygen saturation (SaO2). However, there are some technical limitations such as optical shunting, different camera sensitivity to different light spectra, different AC-to-DC ratios (the peak-to-peak amplitude to baseline ratio) of the PPGi signal for different portions of the sensor surface area, the low sampling rate and the inconsistency of contact force between the fingertip and camera lens.

METHODS

In this paper, we take full account of the above-mentioned design challenges and present an accurate SaO2 estimation method based on two cameras. The hardware system we used consisted of an FPGA development board (XC6SLX150T-3FGG676 from Xilinx), with connected to it two commercial cameras and an SD card. The two cameras were placed back to back, one camera acquired PPGi signal from the right index fingertip under 660 nm light illumination while the other camera acquired PPGi signal from the thumb fingertip using an 800 nm light illumination. The both PPGi signals were captured simultaneously, recorded in a text file on the SD card and processed offline using MATLAB®. The calculation of SaO2 was based on the principle of pulse oximetry. The AC-to-DC ratio was acquired by the ratio of powers of AC and DC components of the PPGi signal in the time-frequency domain using the smoothed pseudo Wigner-Ville distribution. The calibration curve required for SaO2 measurement was obtained by linear regression analysis.

RESULTS

The results of our estimation method from 12 subjects showed a high correlation and accuracy with those of conventional pulse oximetry for the range from 90 to 100%.

CONCLUSIONS

Our method is suitable for mobile applications implemented in smartphones, which could allow SaO2 measurement in a pervasive environment.

Recognition of correct finger placement for photoplethysmographic imaging

In mobile health applications, non-expert users often perform the required medical measurements without supervision. Therefore, it is important that the mobile device guides them through the correct measurement process and automatically detects potential errors that could impact the readings. Camera oximetry provides a non-invasive measurement of heart rate and blood oxygen saturation using the camera of a mobile phone. We describe a novel method to automatically detect the correct finger placement on the camera lens for camera oximetry. Incorrect placement can cause optical shunt and if ignored, lead to low quality oximetry readings. The presented algorithm uses the spectral properties of the pixels to discriminate between correct and incorrect placements. Experimental results demonstrate high mean accuracy (99.06%), sensitivity (98.06%) and specificity (99.30%) with low variability. By sub-sampling pixels, the computational cost of classifying a frame has been reduced by more than three orders of magnitude. The algorithm has been integrated in a newly developed application called OxiCam where it provides real-time user feedback.

Design challenges for camera oximetry on a mobile phone

The use of mobile consumer devices as medical diagnostic tools allows standard medical tests to be performed anywhere. Cameras embedded in consumer devices have previously been used as pulse oximetry sensors. However, technical limitations and implementation challenges have not been described. This manuscript provides a critical analysis of pulse oximeter technology and technical limitations of cameras that can potentially impact implementation of pulse oximetry in mobile phones. Theoretical and practical examples illustrate difficulties and recommendations to overcome these challenges.