Robust respiration detection from remote photoplethysmography

Face-Free Chest Detection Using Convolutional Neural Networks for Non-Contact Respiration Monitoring

Non-contact methods for monitoring respiration face limitations when it comes to selecting the chest region of interest. The semi-automatic method, which requires the user to select the chest region in the first frame, is not suitable for real-time applications. The automatic method, which tracks the face first and then detects the chest region based on the face's position, can be inaccurate if the face is not visible or is rotated. Moreover, using the face region to track the chest region can under-utilize camera pixels since the face is not essential for monitoring respiration. This approach may adversely affect the quality of the respiration signal being measured. To address these issues, we propose a face-free chest detection model based on Convolutional Neural Networks. Our model enhances the measured non-contact respiration signal quality and utilizes more pixels for the chest region alone. In our quantitative study, we demonstrate that our method outperforms traditional methods that require the presence of the face. This approach offers potential benefits for real-time, non-contact respiration monitoring applicationsClinical relevance- This work enhances the performance of non-contact respiration monitoring techniques by precisely detecting the chest region without the need of face in it through a CNN-based model. The use of the CNN-based chest detection model also enhances the real-time monitoring capabilities of non-contact respiration monitoring techniques.

Clinical applications of contactless photoplethysmography for vital signs monitoring in pediatrics: A systematic review and meta-analysis

Background

Contactless photoplethysmography (PPG) potentially affords the ability to obtain vital signs in pediatric populations without disturbing the child. Most validity studies have been conducted in laboratory settings or with healthy adult volunteers. This review aims to evaluate the current literature on contactless vital signs monitoring in pediatric populations and within a clinical setting.

Methods

OVID, Webofscience, Cochrane library, and clinicaltrials.org were systematically searched by two authors for research studies which used contactless PPG to assess vital signs in children and within a clinical setting.

Results

Fifteen studies were included with a total of 170 individuals. Ten studies were included in a meta-analysis for neonatal heart rate (HR), which demonstrated a pooled mean bias of -0.25 (95% limits of agreement (LOA), -1.83 to 1.32). Four studies assessed respiratory rate (RR) in neonates, and meta-analysis demonstrated a pooled mean bias of 0.65 (95% LOA, -3.08 to 4.37). All studies were small, and there were variations in the methods used and risk of bias.

Conclusion

Contactless PPG is a promising tool for vital signs monitoring in children and accurately measures neonatal HR and RR. Further research is needed to assess children of different age groups, the effects of skin type variation, and the addition of other vital signs.

Non-contact Vital Signs Monitoring in Paediatric Anaesthesia - Current Challenges and Future Direction

Non-contact vital sign monitoring is an area of increasing interest in the clinical scenario since it offers advantages over traditional monitoring using leads and wires. These advantages include reduction in transmission of infection and more freedom of movement. Yet there is a paucity of studies available in the clinical setting particularly in paediatric anaesthesia. This scoping review aims to investigate why contactless monitoring, specifically with red-green-blue cameras, is not implemented in mainstream practise. The challenges, drawbacks and limitations of non-contact vital sign monitoring, will be outlined, together with future direction on how it can potentially be implemented in the setting of paediatric anaesthesia, and in the critical care scenario.

Non-contact respiratory rate monitoring using thermal and visible imaging: a pilot study on neonates

Respiratory rate (RR) monitoring is essential in neonatal intensive care units. Despite its importance, RR is still monitored intermittently by manual counting instead of continuous monitoring due to the risk of skin damage with prolonged use of contact electrodes in preterm neonates and false signals due to displacement of electrodes. Thermal imaging has recently gained significance as a non-contact method for RR detection because of its many advantages. However, due to the lack of information in thermal images, the selection and tracking of the region of interest (ROI) in thermal images for neonates are challenging. This paper presents the integration of visible (RGB) and thermal (T) image sequences for the selection and tracking of ROI for breathing rate extraction. The deep-learning based tracking-by-detection approach is employed to detect the ROI in the RGB images, and it is mapped to the thermal images using the RGB-T image registration. The mapped ROI in thermal spectrum sequences gives the respiratory rate. The study was conducted first on healthy adults in different modes, including steady, motion, talking, and variable respiratory order. Subsequently, the method is tested on neonates in a clinical settings. The findings have been validated with a contact-based reference method.The average absolute error between the proposed and belt-based contact method in healthy adults reached 0.1 bpm and for more challenging conditions was approximately 1.5 bpm and 1.8 bpm, respectively. In the case of neonates, the average error is 1.5 bpm, which are promising results. The Bland–Altman analysis showed a good agreement of estimated RR with the reference method RR and this pilot study provided the evidence of using the proposed approach as a contactless method for the respiratory rate detection of neonates in clinical settings.

Contactless SpO2 with an RGB camera: experimental proof of calibrated SpO2

Camera-based blood oxygen saturation (SpO2) monitoring allows reliable measurements without touching the skin and is therefore very attractive when there is a risk of cross-infection, in case of fragile skin, and/or to improve the clinical workflow. Despite promising results, productization of the technology is hampered by the unavailability of adequate hardware, especially a camera, which can capture the optimal wavelengths for SpO2 measurements in the red near-infrared region. A regular color (RGB) camera is attractive because of its availability, but also poses several risks and challenges which affect the accuracy of the measurement. To mitigate the most important risks, we propose to add low-cost commercial off-the-shelf (COTS) components to the setup. We executed two studies with this setup: one at a hypoxia lab with SpO2 values in the range 70 - 100% with the purpose to determine the calibration model, and the other study on volunteers to investigate the accuracy for different spot-check scenarios. The proposed processing pipeline includes face tracking and a robust method to estimate the ratio of relative amplitudes of the photoplethysmographic waveforms. Results show that the error is smaller than 4 percent points for realistic screening scenarios where the subject is seated, either with or without head support and/or ambient light.

Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey

Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.

Multitask Siamese Network for Remote Photoplethysmography and Respiration Estimation

Heart and respiration rates represent important vital signs for the assessment of a person’s health condition. To estimate these vital signs accurately, we propose a multitask Siamese network model (MTS) that combines the advantages of the Siamese network and the multitask learning architecture. The MTS model was trained by the images of the cheek including nose and mouth and forehead areas while sharing the same parameters between the Siamese networks, in order to extract the features about the heart and respiratory information. The proposed model was constructed with a small number of parameters and was able to yield a high vital-sign-prediction accuracy, comparable to that obtained from the single-task learning model; furthermore, the proposed model outperformed the conventional multitask learning model. As a result, we can simultaneously predict the heart and respiratory signals with the MTS model, while the number of parameters was reduced by 16 times with the mean average errors of heart and respiration rates being 2.84 and 4.21. Owing to its light weight, it would be advantageous to implement the vital-sign-monitoring model in an edge device such as a mobile phone or small-sized portable devices.

Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera

We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.

Non-contact physiological monitoring of post-operative patients in the intensive care unit

Prolonged non-contact camera-based monitoring in critically ill patients presents unique challenges, but may facilitate safe recovery. A study was designed to evaluate the feasibility of introducing a non-contact video camera monitoring system into an acute clinical setting. We assessed the accuracy and robustness of the video camera-derived estimates of the vital signs against the electronically-recorded reference values in both day and night environments. We demonstrated non-contact monitoring of heart rate and respiratory rate for extended periods of time in 15 post-operative patients. Across day and night, heart rate was estimated for up to 53.2% (103.0 h) of the total valid camera data with a mean absolute error (MAE) of 2.5 beats/min in comparison to two reference sensors. We obtained respiratory rate estimates for 63.1% (119.8 h) of the total valid camera data with a MAE of 2.4 breaths/min against the reference value computed from the chest impedance pneumogram. Non-contact estimates detected relevant changes in the vital-sign values between routine clinical observations. Pivotal respiratory events in a post-operative patient could be identified from the analysis of video-derived respiratory information. Continuous vital-sign monitoring supported by non-contact video camera estimates could be used to track early signs of physiological deterioration during post-operative care.

Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera

This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.

Contact and Remote Breathing Rate Monitoring Techniques: A Review

Breathing rate monitoring is a must for hospitalized patients with the current coronavirus disease 2019 (COVID-19). We review in this paper recent implementations of breathing monitoring techniques, where both contact and remote approaches are presented. It is known that with non-contact monitoring, the patient is not tied to an instrument, which improves patients' comfort and enhances the accuracy of extracted breathing activity, since the distress generated by a contact device is avoided. Remote breathing monitoring allows screening people infected with COVID-19 by detecting abnormal respiratory patterns. However, non-contact methods show some disadvantages such as the higher set-up complexity compared to contact ones. On the other hand, many reported contact methods are mainly implemented using discrete components. While, numerous integrated solutions have been reported for non-contact techniques, such as continuous wave (CW) Doppler radar and ultrawideband (UWB) pulsed radar. These radar chips are discussed and their measured performances are summarized and compared.

Respiration Monitoring via Forcecardiography Sensors

In the last few decades, a number of wearable systems for respiration monitoring that help to significantly reduce patients’ discomfort and improve the reliability of measurements have been presented. A recent research trend in biosignal acquisition is focusing on the development of monolithic sensors for monitoring multiple vital signs, which could improve the simultaneous recording of different physiological data. This study presents a performance analysis of respiration monitoring performed via forcecardiography (FCG) sensors, as compared to ECG-derived respiration (EDR) and electroresistive respiration band (ERB), which was assumed as the reference. FCG is a novel technique that records the cardiac-induced vibrations of the chest wall via specific force sensors, which provide seismocardiogram-like information, along with a novel component that seems to be related to the ventricular volume variations. Simultaneous acquisitions were obtained from seven healthy subjects at rest, during both quiet breathing and forced respiration at higher and lower rates. The raw FCG sensor signals featured a large, low-frequency, respiratory component (R-FCG), in addition to the common FCG signal. Statistical analyses of R-FCG, EDR and ERB signals showed that FCG sensors ensure a more sensitive and precise detection of respiratory acts than EDR (sensitivity: 100% vs. 95.8%, positive predictive value: 98.9% vs. 92.5%), as well as a superior accuracy and precision in interbreath interval measurement (linear regression slopes and intercepts: 0.99, 0.026 s (R² = 0.98) vs. 0.98, 0.11 s (R² = 0.88), Bland–Altman limits of agreement: ±0.61 s vs. ±1.5 s). This study represents a first proof of concept for the simultaneous recording of respiration signals and forcecardiograms with a single, local, small, unobtrusive, cheap sensor. This would extend the scope of FCG to monitoring multiple vital signs, as well as to the analysis of cardiorespiratory interactions, also paving the way for the continuous, long-term monitoring of patients with heart and pulmonary diseases.

Non-Contact Respiration Measurement Method Based on RGB Camera Using 1D Convolutional Neural Networks

Conventional respiration measurement requires a separate device and/or can cause discomfort, so it is difficult to perform routinely, even for patients with respiratory diseases. The development of contactless respiration measurement technology would reduce discomfort and help detect and prevent fatal diseases. Therefore, we propose a respiration measurement method using a learning-based region-of-interest detector and a clustering-based respiration pixel estimation technique. The proposed method consists of a model for classifying whether a pixel conveys respiration information based on its variance and a method for classifying pixels with clear breathing components using the symmetry of the respiration signals. The proposed method was evaluated with the data of 14 men and women acquired in an actual environment, and it was confirmed that the average error was within approximately 0.1 bpm. In addition, a Bland-Altman analysis confirmed that the measurement result had no error bias, and regression analysis confirmed that the correlation of the results with the reference is high. The proposed method, designed to be inexpensive, fast, and robust to noise, is potentially suitable for practical use in clinical scenarios.

Camera-Based Vital Signs Monitoring During Sleep - A Proof of Concept Study

Polysomnography (PSG) is the current gold standard for the diagnosis of sleep disorders. However, this multi-parametric sleep monitoring tool also has some drawbacks, e.g. it limits the patient's mobility during the night and it requires the patient to come to a specialized sleep clinic or hospital to attach the sensors. Unobtrusive techniques for the detection of sleep disorders such as sleep apnea are therefore gaining increasing interest. Remote photoplethysmography using video is a technique which enables contactless detection of hemodynamic information. Promising results in near-infrared have been reported for the monitoring of sleep-relevant physiological parameters pulse rate, respiration and blood oxygen saturation. In this study we validate a contactless monitoring system on eight patients with a high likelihood of relevant obstructive sleep apnea, which are enrolled for a sleep study at a specialized sleep center. The dataset includes 46.5 hours of video recordings, full polysomnography and metadata. The camera can detect pulse and respiratory rate within 2 beats/breaths per minute accuracy 92% and 91% of the time, respectively. Estimated blood oxygen values are within 4 percentage points of the finger-oximeter 89% of the time. These results demonstrate the potential of a camera as a convenient diagnostic tool for sleep apnea, and sleep disorders in general.

Spatiotemporal monitoring of changes in oxy/deoxy-hemoglobin concentration and blood pulsation on human skin using smartphone-enabled remote multispectral photoplethysmography

We propose a smartphone-enabled remote multispectral photoplethysmography (SP-rmPPG) system and method to realize spatiotemporal monitoring of perfusion changes and pulsations of the oxyhemoglobin (HbO2) and deoxyhemoglobin (Hb) information of the effective blood volume within light interrogated skin tissue beds. The system is implemented on an unmodified smartphone utilizing its built-in camera and flashlight to acquire videos of the skin reflectance. The SP-rmPPG method converts the RGB video into multispectral cubes, upon which to decouple the dynamic changes in HbO2 and Hb information using a modified Beer-Lambert law and the selective wavelength bands of 500 nm and 650 nm. Blood pulsation amplitudes are then obtained by applying a window-based lock-in amplification on the derived spatiotemporal changes in HbO2 or Hb signals. To demonstrate the feasibility of proposed method, we conduct two experiments on the skin tissue beds that are conditioned by occlusive maneuver of supplying arteries: one using the popular blood cuff pressure maneuver on the upper arm, and another artificially inducing a transient ischemic condition on the facial skin tissue beds by finger pressing on the supplying external carotid artery. The cuff experiment shows that the measured dynamic information of HbO2 and Hb in the downstream agrees well with the parallel measurements of oxygenation saturation given by the standard pulse oximeter. We also observe the expected imbalance of spatiotemporal changes in the HbO2 and Hb between the right and left cheeks when the transient ischemic condition is induced in the one side of facial skin tissue beds. The results from the two experiments sufficiently demonstrate the feasibility of the proposed method to monitor the spatiotemporal changes in the skin hemodynamics, including blood oxygenation and pulsation amplitudes. Considering the ever-growing accessibility and affordability of the smartphone to the general public, the proposed strategy promises the early screening of vascular diseases and improving general public health particularly in rural areas with low resource settings.

Non-contact breathing rate monitoring in newborns: A review

The neonatal period - the first 4 weeks of life - is the most critical time for a child's survival. Breathing rate is a vital indicator of the health condition and requires continuous monitoring in case of sickness or preterm birth. Breathing movements can be counted by contact and non-contact methods. In the case of newborn infants, the non-contact breathing rate monitoring need is high, as a contact-based approach may interfere while providing care and is subject to interference by non-breathing movements. This review article delivers a factual summary, and describes the methods and processing involved in non-contact based breathing rate monitoring. The article also provides the advantages, limitations, and clinical applications of these methods. Additionally, signal processing, feasibility, and future direction of different non-contact neonatal breathing rate monitoring are discussed.

Noncontact Physiological Measurement Using a Camera: A Technical Review and Future Directions

Using a camera as an optical sensor to monitor physiological parameters has garnered considerable research interest in biomedical engineering in recent decades. Researchers have explored the use of a camera for monitoring a variety of physiological waveforms, together with the vital signs carried by these waveforms. Most of the obtained waveforms are related to the human respiratory and cardiovascular systems, and in addition of being indicative of overall health, they can also detect early signs of certain diseases. While using a camera for noncontact physiological signal monitoring offers the advantages of low cost and operational ease, it also has the disadvantages such as vulnerability to motion and lack of burden-free calibration solutions in some use cases. This study presents an overview of the existing camera-based methods that have been reported in recent years. It introduces the physiological principles behind these methods, signal acquisition approaches, various types of acquired signals, data processing algorithms, and application scenarios of these methods. It also discusses the technological gaps between the camera-based methods and traditional medical techniques, which are mostly contact-based. Furthermore, we present the manner in which noncontact physiological signal monitoring use has been extended, particularly over the recent years, to more day-to-day aspects of individuals' lives, so as to go beyond the more conventional use case scenarios. We also report on the development of novel approaches that facilitate easier measurement of less often monitored and recorded physiological signals. These have the potential of ushering a host of new medical and lifestyle applications. We hope this study can provide useful information to the researchers in the noncontact physiological signal measurement community.

Smart Recognition COVID-19 System to Predict Suspicious Persons Based on Face Features

The coronavirus (COVID-19) is identified at first in Wuhan in December 2019. The apparition of the COVID-19 virus is widely spread to concern all countries worldwide. The World Health Organization (WHO) on March 11 declare COVID-19 a pandemic. This Virus causes a serious infection of the respiratory system. Its high transmission constitutes great problems and challenges. The WHO proposes many actions to limit the spread of the virus such as quarantine and decrease or halt flights between states. The actions taken by states in airports are to detect suspicious persons with COVID-19. We aimed to provide a Computer-Aided Diagnosis (CAD) framework to predict suspicious COVID-19 person. This prediction identifies suspicious persons who suffer from shortness breath which is the main symptom of this disease. Extract shortness breath anomaly through the estimated heart rate from face based-video is the main contribution of the present paper. We developed a Smart Recognition COVID-19 (SRC) system to estimate the breath score. In conclusion, our study achieves an accurate breath score. The error is about 1 breath per minute. The proposed solution is of great importance because it helps managers in the airport to predict suspicious COVID-19 passengers.

Advancing PPG Signal Quality and Know-How Through Knowledge Translation—From Experts to Student and Researcher

Objective: Despite the vast number of photoplethysmography (PPG) research publications and growing demands for such sensing in Digital and Wearable Health platforms, there appears little published on signal quality expectations for morphological pulse analysis. Aim: to determine a consensus regarding the minimum number of undistorted i.e., diagnostic quality pulses required, as well as a threshold proportion of noisy beats for recording rejection.

Approach: Questionnaire distributed to international fellow researchers in skin contact PPG measurements on signal quality expectations and associated factors concerning recording length, expected artifact-free pulses (“diagnostic quality”) in a trace, proportion of trace having artifact to justify excluding/repeating measurements, minimum beats required, and number of respiratory cycles.

Main Results: 18 (of 26) PPG researchers responded. Modal range estimates considered a 2-min recording time as target for morphological analysis. Respondents expected a recording to have 86–95% of diagnostic quality pulses, at least 11–20 sequential pulses of diagnostic quality and advocated a 26–50% noise threshold for recording rejection. There were broader responses found for the required number of undistorted beats (although a modal range of 51–60 beats for both finger and toe sites was indicated).

Significance: For morphological PPG pulse wave analysis recording acceptability was indicated if <50% of beats have artifact and preferably that a minimum of 50 non-distorted PPG pulses are present (with at least 11–20 sequential) to be of diagnostic quality. Estimates from this knowledge transfer exercise should help inform students and researchers as a guide in standards development for PPG study design.

Multispectral Video Fusion for Non-Contact Monitoring of Respiratory Rate and Apnea

Continuous monitoring of respiratory activity is desirable in many clinical applications to detect respiratory events. Non-contact monitoring of respiration can be achieved with near- and far-infrared spectrum cameras. However, current technologies are not sufficiently robust to be used in clinical applications. For example, they fail to estimate an accurate respiratory rate (RR) during apnea. We present a novel algorithm based on multispectral data fusion that aims at estimating RR also during apnea. The algorithm independently addresses the RR estimation and apnea detection tasks. Respiratory information is extracted from multiple sources and fed into an RR estimator and an apnea detector whose results are fused into a final respiratory activity estimation. We evaluated the system retrospectively using data from 30 healthy adults who performed diverse controlled breathing tasks while lying supine in a dark room and reproduced central and obstructive apneic events. Combining multiple respiratory information from multispectral cameras improved the root mean square error (RMSE) accuracy of the RR estimation from up to 4.64 monospectral data down to 1.60 breaths/min. The median F1 scores for classifying obstructive (0.75 to 0.86) and central apnea (0.75 to 0.93) also improved. Furthermore, the independent consideration of apnea detection led to a more robust system (RMSE of 4.44 vs. 7.96 breaths/min). Our findings may represent a step towards the use of cameras for vital sign monitoring in medical applications.

Advancements in Methods and Camera-Based Sensors for the Quantification of Respiration

Assessment of respiratory function allows early detection of potential disorders in the respiratory system and provides useful information for medical management. There is a wide range of applications for breathing assessment, from measurement systems in a clinical environment to applications involving athletes. Many studies on pulmonary function testing systems and breath monitoring have been conducted over the past few decades, and their results have the potential to broadly impact clinical practice. However, most of these works require physical contact with the patient to produce accurate and reliable measures of the respiratory function. There is still a significant shortcoming of non-contact measuring systems in their ability to fit into the clinical environment. The purpose of this paper is to provide a review of the current advances and systems in respiratory function assessment, particularly camera-based systems. A classification of the applicable research works is presented according to their techniques and recorded/quantified respiration parameters. In addition, the current solutions are discussed with regards to their direct applicability in different settings, such as clinical or home settings, highlighting their specific strengths and limitations in the different environments.

Doppler ultrasound cardiac gating of intracranial flow at 7T

BACKGROUND

Ultra-high field magnetic resonance imaging (MR) may be used to improve intracranial blood flow measurements. However, standard cardiac synchronization methods tend to fail at ultra-high field MR. Therefore, this study aims to investigate an alternative synchronization technique using Doppler ultrasound.

METHODS

Healthy subjects (n = 9) were examined with 7T MR. Flow was measured in the M1-branch of the middle cerebral artery (MCA) and in the cerebral aqueduct (CA) using through-plane phase contrast (2D flow). Flow in the circle of Willis was measured with three-dimensional, three-directional phase contrast (4D flow). Scans were gated with Doppler ultrasound (DUS) and electrocardiogram (ECG), and pulse oximetry data (POX) was collected simultaneously. False negative and false positive trigger events were counted for ECG, DUS and POX, and quantitative flow measures were compared.

RESULTS

There were fewer false positive triggers for DUS compared to ECG (5.3 ± 11 vs. 25 ± 31, p = 0.031), while no other measured parameters differed significantly. Net blood flow in M1 was similar between DUS and ECG for 2D flow (1.5 ± 0.39 vs. 1.6 ± 0.41, bias ± 1.96SD: - 0.021 ± 0.36) and 4D flow (1.8 ± 0.48 vs. 9 ± 0.59, bias ± 1.96SD: - 0.086 ± 0.57 ml). Net CSF flow per heart beat in the CA was also similar for DUS and ECG (3.6 ± 2.1 vs. 3.0 ± 5.8, bias ± 1.96SD: 0.61 ± 13.6 μl).

CONCLUSION

Gating with DUS produced fewer false trigger events than using ECG, with similar quantitative flow values. DUS gating is a promising technique for cardiac synchronization at 7T.

Enhanced Contactless Vital Sign Estimation from Real-Time Multimodal 3D Image Data

The contactless estimation of vital signs using conventional color cameras and ambient light can be affected by motion artifacts and changes in ambient light. On both these problems, a multimodal 3D imaging system with an irritation-free controlled illumination was developed in this work. In this system, real-time 3D imaging was combined with multispectral and thermal imaging. Based on 3D image data, an efficient method was developed for the compensation of head motions, and novel approaches based on the use of 3D regions of interest were proposed for the estimation of various vital signs from multispectral and thermal video data. The developed imaging system and algorithms were demonstrated with test subjects, delivering a proof-of-concept.

Exploiting Weak Head Movements for Camera-based Respiration Detection

In recent years, considerable progress has been made in the non-contact based detection of the respiration rate from video sequences. Common techniques either directly assess the movement of the chest due to breathing or are based on analyzing subtle color changes that occur as a result of hemodynamic properties of the skin tissue by means of remote photoplethysmography (rPPG). However, extracting hemodynamic parameters from rPPG is often difficult especially if the skin is not visible to the camera. In contrast, extracting respiratory signals from chest movements turned out to be a robust method. However, the detectability of chest regions cannot be guaranteed in any application scenario, for instance if the camera setting is optimized to provide close-up images of the head. In such a case an alternative method for respiration detection is required.It is reasonable to assume that the mechanical coupling between chest and head induces minor movements of the head which, like in rPPG, can be detected from subtle color changes as well. Although the strength of these movements is expected to be much smaller in scale, sensing these intensity variations could provide a reasonably suited respiration signal for subsequent respiratory rate analysis.In order to investigate this coupling we conducted an experimental study with 12 subjects and applied motion-and rPPG-based methods to estimate the respiratory frequency from both head regions and chest. Our results show that it is possible to derive signals correlated to chest movement from facial regions. The method is a feasible alternative to rPPG-based respiratory rate estimations when rPPG-signals cannot be derived reliably and chest movement detection cannot be applied as well.

Non-invasive continuous respiratory monitoring using temperature-based sensors

Respiratory rate (RR) is a key vital sign that has been traditionally employed in the clinical assessment of patients and in the prevention of respiratory compromise. Despite its relevance, current practice for monitoring RR in non-intubated patients strongly relies on visual counting, which delivers an intermittent and error-prone assessment of the respiratory status. Here, we present a novel non-invasive respiratory monitor that continuously measures the RR in human subjects. The respiratory activity of the user is inferred by sensing the thermal transfer between the breathing airflow and a temperature sensor placed between the nose and the mouth. The performance of the respiratory monitor is assessed through respiratory experiments performed on healthy subjects. Under spontaneous breathing, the mean RR difference between our respiratory monitor and visual counting was 0.4 breaths per minute (BPM), with a 95% confidence interval equal to [- 0.5, 1.3] BPM. The robustness of the respiratory sensor to the position is assessed by studying the signal-to-noise ratio in different locations on the upper lip, displaying a markedly better performance than traditional thermal sensors used for respiratory airflow measurements.

An open-source remote heart rate imaging method with practical apparatus and algorithms

Recent developments in computer science and digital image processing have enabled the extraction of an individual’s heart pulsations from pixel changes in recorded video images of human skin surfaces. This method is termed remote photoplethysmography (rPPG) and can be achieved with consumer-level cameras (e.g., a webcam or mobile camera). The goal of the present publication is two-fold. First, we aim to organize future rPPG software developments in a tractable and nontechnical manner, such that the public gains access to a basic open-source rPPG code, comes to understand its utility, and can follow its most recent progressions. The second goal is to investigate rPPG’s accuracy in detecting heart rates from the skin surfaces of several body parts after physical exercise and under ambient lighting conditions with a consumer-level camera. We report that rPPG is highly accurate when the camera is aimed at facial skin tissue, but that the heart rate recordings from wrist regions are less reliable, and recordings from the calves are unreliable. Facial rPPG remained accurate despite the high heart rates after exercise. The proposed research procedures and the experimental findings provide guidelines for future studies on rPPG.

A Contactless Respiratory Rate Estimation Method Using a Hermite Magnification Technique and Convolutional Neural Networks

The monitoring of respiratory rate is a relevant factor in medical applications and day-to-day activities. Contact sensors have been used mostly as a direct solution and they have shown their effectiveness, but with some disadvantages for example in vulnerable skins such as burns patients. For this reason, contactless monitoring systems are gaining increasing attention for respiratory detection. In this paper, we present a new non-contact strategy to estimate respiratory rate based on Eulerian motion video magnification technique using Hermite transform and a system based on a Convolutional Neural Network (CNN). The system tracks chest movements of the subject using two strategies: using a manually selected ROI and without the selection of a ROI in the image frame. The system is based on the classifications of the frames as an inhalation or exhalation using CNN. Our proposal has been tested on 10 healthy subjects in different positions. To compare performance of methods to detect respiratory rate the mean average error and a Bland and Altman analysis is used to investigate the agreement of the methods. The mean average error for the automatic strategy is 3.28 ± 3.33 % with and agreement with respect of the reference of ≈98%.

Non-contact physiological monitoring of preterm infants in the Neonatal Intensive Care Unit

The implementation of video-based non-contact technologies to monitor the vital signs of preterm infants in the hospital presents several challenges, such as the detection of the presence or the absence of a patient in the video frame, robustness to changes in lighting conditions, automated identification of suitable time periods and regions of interest from which vital signs can be estimated. We carried out a clinical study to evaluate the accuracy and the proportion of time that heart rate and respiratory rate can be estimated from preterm infants using only a video camera in a clinical environment, without interfering with regular patient care. A total of 426.6 h of video and reference vital signs were recorded for 90 sessions from 30 preterm infants in the Neonatal Intensive Care Unit (NICU) of the John Radcliffe Hospital in Oxford. Each preterm infant was recorded under regular ambient light during daytime for up to four consecutive days. We developed multi-task deep learning algorithms to automatically segment skin areas and to estimate vital signs only when the infant was present in the field of view of the video camera and no clinical interventions were undertaken. We propose signal quality assessment algorithms for both heart rate and respiratory rate to discriminate between clinically acceptable and noisy signals. The mean absolute error between the reference and camera-derived heart rates was 2.3 beats/min for over 76% of the time for which the reference and camera data were valid. The mean absolute error between the reference and camera-derived respiratory rate was 3.5 breaths/min for over 82% of the time. Accurate estimates of heart rate and respiratory rate could be derived for at least 90% of the time, if gaps of up to 30 seconds with no estimates were allowed.

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Remote pulse rate measurement from facial video has gained particular attention over the last few years. Research exhibits significant advancements and demonstrates that common video cameras correspond to reliable devices that can be employed to measure a large set of biomedical parameters without any contact with the subject. A new framework for measuring and mapping pulse rate from video is presented in this pilot study. The method, which relies on convolutional 3D networks, is fully automatic and does not require any special image preprocessing. In addition, the network ensures concurrent mapping by producing a prediction for each local group of pixels. A particular training procedure that employs only synthetic data is proposed. Preliminary results demonstrate that this convolutional 3D network can effectively extract pulse rate from video without the need for any processing of frames. The trained model was compared with other state-of-the-art methods on public data. Results exhibit significant agreement between estimated and ground-truth measurements: the root mean square error computed from pulse rate values assessed with the convolutional 3D network is equal to 8.64 bpm, which is superior to 10 bpm for the other state-of-the-art methods. The robustness of the method to natural motion and increases in performance correspond to the two main avenues that will be considered in future works.

Data-Driven Calibration Estimation for Robust Remote Pulse-Oximetry

Pulse-oximetry has become a core monitoring modality in most fields of medicine. Typical dual-wavelength pulse-oximeters estimate blood oxygen saturation (SpO2) levels from a relationship between the amplitudes of red and infrared photoplethysmographic (PPG) waveforms. When captured with a camera, the PPG waveforms are much weaker and consequently the measurement is more sensitive to distortions and noises. Therefore, an indirect method has recently been proposed where, instead of extracting the relative amplitudes from the individual waveforms, the waveforms are linearly combined to construct a collection of pulse signals with different pulse signatures, each corresponding to a specific oxygen saturation level. This method has been shown to outperform the conventional ratio-of-ratios based methods, especially when adding a third wavelength. Adding wavelengths, however, complicates the calibration. Inaccuracies in the calibration model threaten the performance of the method. Opto-physiological models have been shown earlier to provide useful calibration parameter estimates. In this paper, we show that the accuracy can be improved using a data-driven approach. We performed 5-fold cross validation on recordings with variations in oxygen saturation and optimized for pulse quality. All evaluated wavelength combinations, also without visible red, meet the required ISO standard accuracy with the calibration from the proposed method. This scalable approach is not only helpful to fine-tune the calibration model, but even allows computation of the calibration model parameters from scratch without prior knowledge of the data acquisition details, i.e., the properties of camera and illumination.

A Systematic Review of Tools to Measure Respiratory Rate in Order to Identify Childhood Pneumonia

Pneumonia is the leading infectious cause of death in children worldwide, with most deaths occurring in developing countries. Measuring respiratory rate is critical to the World Health Organization's guidelines for diagnosing childhood pneumonia in low-resource settings, yet it is difficult to accurately measure. We conducted a systematic review to landscape existing respiratory rate measurement technologies. We searched PubMed, Embase, and Compendex for studies published through September 2017 assessing the accuracy of respiratory rate measurement technologies in children. We identified 16 studies: 2 describing manual devices and 14 describing automated devices. Although both studies describing manual devices took place in low-resource settings, all studies describing automated devices were conducted in well-resourced settings. Direct comparison between studies was complicated by small sample size, absence of a consistent reference standard, and variations in comparison methodology. There is an urgent need for affordable and appropriate innovations that can reliably measure a child's respiratory rate in low-resource settings. Accelerating development or scale-up of these technologies could have the potential to advance childhood pneumonia diagnosis worldwide.

A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

OBJECTIVES

Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting "outliers".

METHODS

We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed.

RESULTS

Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning.

CONCLUSION

Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.

Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review

OBJECTIVE

Over the last 15 years, developments in camera technology have coincided with increased availability and affordability. This has led to an increasing interest in using these technologies in healthcare settings. Image-based monitoring methods potentially allow multiple vital signs to be measured concurrently using a non-contact sensor. We have undertaken a systematic review of the current availability and performance of these monitoring methods.

APPROACH

A multiple database search was conducted using MEDLINE, Embase, CINAHL, Cochrane Library, OpenGrey, IEEE Xplore Library and ACM Digital Library to July 2018. We included studies comparing image-based heart rate, respiratory rate, oxygen saturation and blood pressure monitoring methods against one or more validated reference device(s). Each included study was assessed using the modified GRRAS criteria for reporting bias.

MAIN RESULTS

Of 30 279 identified studies, 161 were included in the final analysis. Twenty studies (20/161, 12%) were carried out on patients in clinical settings, while the remainder were conducted in academic settings using healthy volunteer populations. The 18-40 age group was best represented across the identified studies. One hundred and twenty studies (120/161, 75%) estimated heart rate, followed by 62 studies (62/161, 39%) estimating respiratory rate. Fewer studies focused on oxygen saturation (11/161, 7%) or blood pressure (6/161, 4%) estimation. Fifty-one heart rate studies (51/120, 43%) and 24 respiratory rate studies (24/62, 39%) used Bland-Altman analysis to report their results. Of the heart rate studies, 28 studies (28/51, 55%) showed agreement within industry standards of [Formula: see text]5 beats per minute. Only two studies achieved this within clinical settings. Of the respiratory rate studies, 13 studies (13/24, 54%) showed agreement within industry standards of [Formula: see text]3 breaths per minute, but only one study achieved this in a clinical setting. Statistical analysis was heterogeneous across studies with frequent inappropriate use of correlation. The majority of studies (99/161, 61%) monitored subjects for under 5 min. Three studies (3/161, 2%) monitored subjects for over 60 min, all of which were conducted in hospital settings.

SIGNIFICANCE

Heart rate and respiratory rate monitoring using video images is currently possible and performs within clinically acceptable limits under experimental conditions. Camera-derived estimates were less accurate in the proportion of studies conducted in clinical settings. We would encourage thorough reporting of the population studied, details of clinically relevant aspects of methodology, and the use of appropriate statistical methods in future studies. Systematic review registration: PROSPERO CRD42016029167 Protocol: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0615-3.

Unobtrusive Respiratory Flow Monitoring Using a Thermopile Array: A Feasibility Study

Low-resolution thermal cameras have already been used in the detection of respiratory flow. However, microbolometer technology has a high production cost compared to thermopile arrays. In this work, the feasibility of using a thermopile array to detect respiratory flow has been investigated in multiple settings. To prove the concept, we tested the detector on six healthy subjects. Our method automatically selects the region-of-interest by discriminating between sensor elements that output noise and flow-induced signals. The thermopile array yielded an average root mean squared error of 1.59 b r e a t h s p e r m i n u t e . Parameters such as distance, breathing rate, orientation, and oral or nasal breathing resulted in being fundamental in the detection of respiratory flow. The paper provides the proof-of-concept that low-cost thermopile-arrays can be used to monitor respiratory flow in a lab setting and without the need for facial landmark detection. Further development could provide a more attractive alternative for the earlier bolometer-based proposals.

Cardiovascular assessment by imaging photoplethysmography - a review

Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.

A Comparison of Reflective Photoplethysmography for Detection of Heart Rate, Blood Oxygen Saturation, and Respiration Rate at Various Anatomical Locations

Monitoring of vital signs is critical for patient triage and management. Principal assessments of patient conditions include respiratory rate heart/pulse rate and blood oxygen saturation. However, these assessments are usually carried out with multiple sensors placed in different body locations. The aim of this paper is to identify a single location on the human anatomy whereby a single 1 cm × 1 cm non-invasive sensor could simultaneously measure heart rate (HR), blood oxygen saturation (SpO₂), and respiration rate (RR), at rest and while walking. To evaluate the best anatomical location, we analytically compared eight anatomical locations for photoplethysmography (PPG) sensors simultaneously acquired by a single microprocessor at rest and while walking, with a comparison to a commercial pulse oximeter and respiration rate ground truth. Our results show that the forehead produced the most accurate results for HR and SpO₂ both at rest and walking, however, it had poor RR results. The finger recorded similar results for HR and SpO₂, however, it had more accurate RR results. Overall, we found the finger to be the best location for measurement of all three parameters at rest; however, no site was identified as capable of measuring all parameters while walking.

Estimating carotid pulse and breathing rate from near-infrared video of the neck

OBJECTIVE

Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can be easily analyzed and/or require expensive and custom hardware to perform the measurements.

APPROACH

This work introduces a low-cost method to measure subtle motions associated with the carotid pulse and breathing movement from the neck using near-infrared (NIR) video imaging. A skin reflection model of the neck was established to provide a theoretical foundation for the method. In particular, the method relies on template matching for neck detection, principal component analysis for feature extraction, and hidden Markov models for data smoothing.

MAIN RESULTS

We compared the estimated HR and BR measures with ones provided by an FDA-cleared device in a 12-participant laboratory study: the estimates achieved a mean absolute error of 0.36 beats per minute and 0.24 breaths per minute under both bright and dark lighting.

SIGNIFICANCE

This work advances the possibilities of non-contact physiological measurement in real-life conditions in which environmental illumination is limited and in which the face of the person is not readily available or needs to be protected. Due to the increasing availability of NIR imaging devices, the described methods are readily scalable.

Unobtrusive Vital Sign Monitoring in Automotive Environments-A Review

This review provides an overview of unobtrusive monitoring techniques that could be used to monitor some of the human vital signs (i.e., heart activity, breathing activity, temperature and potentially oxygen saturation) in a car seat. It will be shown that many techniques actually measure mechanical displacement, either on the body surface and/or inside the body. However, there are also techniques like capacitive electrocardiogram or bioimpedance that reflect electrical activity or passive electrical properties or thermal properties (infrared thermography). In addition, photopleythysmographic methods depend on optical properties (like scattering and absorption) of biological tissues and-mainly-blood. As all unobtrusive sensing modalities are always fragile and at risk of being contaminated by disturbances (like motion, rapidly changing environmental conditions, triboelectricity), the scope of the paper includes a survey on redundant sensor arrangements. Finally, this review also provides an overview of automotive demonstrators for vital sign monitoring.

Assessing blood vessel perfusion and vital signs through retinal imaging photoplethysmography

One solution to the global challenge of increasing ocular disease is a cost-effective technique for rapid screening and assessment. Current ophthalmic imaging techniques, e.g. scanning and ocular blood flow systems, are expensive, complex to operate and utilize invasive contrast agents during assessment. The work presented here demonstrates a simple retinal imaging photoplethysmography (iPPG) system with the potential to provide screening, diagnosis, monitoring and assessment that is non-invasive, painless and radiationless. Time series of individual retinal blood vessel images, captured with an eye fundus camera, are processed using standard filtering, amplitude demodulation and principle component analysis (PCA) methods to determine the values of the heart rate (HR) and respiration rate (RR), which are in compliance with simultaneously obtained measurements using commercial pulse oximetry. It also seems possible that some information on the dynamic changes in oxygen saturation levels (SpO2) in a retinal blood vessel may also be obtained. As a consequence, the retinal iPPG modality system demonstrates a potential avenue for rapid ophthalmic screening, and even early diagnosis, against ocular disease without the need for fluorescent or contrast agents.

Breathing Rate Estimation From the Electrocardiogram and Photoplethysmogram: A Review

Breathing rate (BR) is a key physiological parameter used in a range of clinical settings. Despite its diagnostic and prognostic value, it is still widely measured by counting breaths manually. A plethora of algorithms have been proposed to estimate BR from the electrocardiogram (ECG) and pulse oximetry (photoplethysmogram, PPG) signals. These BR algorithms provide opportunity for automated, electronic, and unobtrusive measurement of BR in both healthcare and fitness monitoring. This paper presents a review of the literature on BR estimation from the ECG and PPG. First, the structure of BR algorithms and the mathematical techniques used at each stage are described. Second, the experimental methodologies that have been used to assess the performance of BR algorithms are reviewed, and a methodological framework for the assessment of BR algorithms is presented. Third, we outline the most pressing directions for future research, including the steps required to use BR algorithms in wearable sensors, remote video monitoring, and clinical practice.

Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging

The ability to monitor the respiratory rate, one of the vital signs, is extremely important for the medical treatment, healthcare and fitness sectors. In many situations, mobile methods, which allow users to undertake everyday activities, are required. However, current monitoring systems can be obtrusive, requiring users to wear respiration belts or nasal probes. Alternatively, contactless digital image sensor based remote-photoplethysmography (PPG) can be used. However, remote PPG requires an ambient source of light, and does not work properly in dark places or under varying lighting conditions. Recent advances in thermographic systems have shrunk their size, weight and cost, to the point where it is possible to create smart-phone based respiration rate monitoring devices that are not affected by lighting conditions. However, mobile thermal imaging is challenged in scenes with high thermal dynamic ranges (e.g. due to the different environmental temperature distributions indoors and outdoors). This challenge is further amplified by general problems such as motion artifacts and low spatial resolution, leading to unreliable breathing signals. In this paper, we propose a novel and robust approach for respiration tracking which compensates for the negative effects of variations in the ambient temperature and motion artifacts and can accurately extract breathing rates in highly dynamic thermal scenes. The approach is based on tracking the nostril of the user and using local temperature variations to infer inhalation and exhalation cycles. It has three main contributions. The first is a novel Optimal Quantization technique which adaptively constructs a color mapping of absolute temperature to improve segmentation, classification and tracking. The second is the Thermal Gradient Flow method that computes thermal gradient magnitude maps to enhance the accuracy of the nostril region tracking. Finally, we introduce the Thermal Voxel method to increase the reliability of the captured respiration signals compared to the traditional averaging method. We demonstrate the extreme robustness of our system to track the nostril-region and measure the respiratory rate by evaluating it during controlled respiration exercises in high thermal dynamic scenes (e.g. strong correlation (r = 0.9987) with the ground truth from the respiration-belt sensor). We also demonstrate how our algorithm outperformed standard algorithms in settings with different amounts of environmental thermal changes and human motion. We open the tracked ROI sequences of the datasets collected for these studies (i.e. under both controlled and unconstrained real-world settings) to the community to foster work in this area.