Non-Contact Respiratory Monitoring Using an RGB Camera for Real-World Applications

Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild

Heart rate is an essential vital sign to evaluate human health. Remote heart monitoring using cheaply available devices has become a necessity in the twenty-first century to prevent any unfortunate situation caused by the hectic pace of life. In this paper, we propose a new method based on the transformer architecture with a multi-skip connection biLSTM decoder to estimate heart rate remotely from videos. Our method is based on the skin color variation caused by the change in blood volume in its surface. The presented heart rate estimation framework consists of three main steps: (1) the segmentation of the facial region of interest (ROI) based on the landmarks obtained by 3DDFA; (2) the extraction of the spatial and global features; and (3) the estimation of the heart rate value from the obtained features based on the proposed method. This paper investigates which feature extractor performs better by captioning the change in skin color related to the heart rate as well as the optimal number of frames needed to achieve better accuracy. Experiments were conducted using two publicly available datasets (LGI-PPGI and Vision for Vitals) and our own in-the-wild dataset (12 videos collected by four drivers). The experiments showed that our approach achieved better results than the previously published methods, making it the new state of the art on these datasets.

Contactless Heart and Respiration Rates Estimation and Classification of Driver Physiological States Using CW Radar and Temporal Neural Networks

The measurement and analysis of vital signs are a subject of significant research interest, particularly for monitoring the driver's physiological state, which is of crucial importance for road safety. Various approaches have been proposed using contact techniques to measure vital signs. However, all of these methods are invasive and cumbersome for the driver. This paper proposes using a non-contact sensor based on continuous wave (CW) radar at 24 GHz to measure vital signs. We associate these measurements with distinct temporal neural networks to analyze the signals to detect and extract heart and respiration rates as well as classify the physiological state of the driver. This approach offers robust performance in estimating the exact values of heart and respiration rates and in classifying the driver's physiological state. It is non-invasive and requires no physical contact with the driver, making it particularly practical and safe. The results presented in this paper, derived from the use of a 1D Convolutional Neural Network (1D-CNN), a Temporal Convolutional Network (TCN), a Recurrent Neural Network particularly the Bidirectional Long Short-Term Memory (Bi-LSTM), and a Convolutional Recurrent Neural Network (CRNN). Among these, the CRNN emerged as the most effective Deep Learning approach for vital signal analysis.

Breathing-Associated Facial Region Segmentation for Thermal Camera-Based Indirect Breathing Monitoring

Breathing can be measured in a non-contact method using a thermal camera. The objective of this study investigates non-contact breathing measurements using thermal cameras, which have previously been limited to measuring the nostril only from the front where it is clearly visible. The previous method is challenging to use for other angles and frontal views, where the nostril is not well-represented. In this paper, we defined a new region called the breathing-associated-facial-region (BAFR) that reflects the physiological characteristics of breathing, and extract breathing signals from views of 45 and 90 degrees, including the frontal view where the nostril is not clearly visible. Experiments were conducted on fifteen healthy subjects in different views, including frontal with and without nostril, 45-degree, and 90-degree views. A thermal camera (A655sc model, FLIR systems) was used for non-contact measurement, and biopac (MP150, Biopac-systems-Inc) was used as a chest breathing reference. The results showed that the proposed algorithm could extract stable breathing signals at various angles and views, achieving an average breathing cycle accuracy of 90.9% when applied compared to 65.6% without proposed algorithm. The average correlation value increases from 0.587 to 0.885. The proposed algorithm can be monitored in a variety of environments and extract the BAFR at diverse angles and views.

Face-Free Chest Detection Using Convolutional Neural Networks for Non-Contact Respiration Monitoring

Non-contact methods for monitoring respiration face limitations when it comes to selecting the chest region of interest. The semi-automatic method, which requires the user to select the chest region in the first frame, is not suitable for real-time applications. The automatic method, which tracks the face first and then detects the chest region based on the face's position, can be inaccurate if the face is not visible or is rotated. Moreover, using the face region to track the chest region can under-utilize camera pixels since the face is not essential for monitoring respiration. This approach may adversely affect the quality of the respiration signal being measured. To address these issues, we propose a face-free chest detection model based on Convolutional Neural Networks. Our model enhances the measured non-contact respiration signal quality and utilizes more pixels for the chest region alone. In our quantitative study, we demonstrate that our method outperforms traditional methods that require the presence of the face. This approach offers potential benefits for real-time, non-contact respiration monitoring applicationsClinical relevance- This work enhances the performance of non-contact respiration monitoring techniques by precisely detecting the chest region without the need of face in it through a CNN-based model. The use of the CNN-based chest detection model also enhances the real-time monitoring capabilities of non-contact respiration monitoring techniques.

Respiratory Rate Estimation during Walking and Running Using Breathing Sounds Recorded with a Microphone

Emerging evidence suggests that respiratory frequency (f_R) is a valid marker of physical effort. This has stimulated interest in developing devices that allow athletes and exercise practitioners to monitor this vital sign. The numerous technical challenges posed by breathing monitoring in sporting scenarios (e.g., motion artifacts) require careful consideration of the variety of sensors potentially suitable for this purpose. Despite being less prone to motion artifacts than other sensors (e.g., strain sensors), microphone sensors have received limited attention so far. This paper proposes the use of a microphone embedded in a facemask for estimating f_R from breath sounds during walking and running. f_R was estimated in the time domain as the time elapsed between consecutive exhalation events retrieved from breathing sounds every 30 s. Data were collected from ten healthy subjects (both males and females) at rest and during walking (at 3 km/h and 6 km/h) and running (at 9 km/h and 12 km/h) activities. The reference respiratory signal was recorded with an orifice flowmeter. The mean absolute error (MAE), the mean of differences (MOD), and the limits of agreements (LOAs) were computed separately for each condition. Relatively good agreement was found between the proposed system and the reference system, with MAE and MOD values increasing with the increase in exercise intensity and ambient noise up to a maximum of 3.8 bpm (breaths per minute) and -2.0 bpm, respectively, during running at 12 km/h. When considering all the conditions together, we found an MAE of 1.7 bpm and an MOD ± LOAs of -0.24 ± 5.07 bpm. These findings suggest that microphone sensors can be considered among the suitable options for estimating f_R during exercise.

Design and Testing of a Smart Facemask for Respiratory Monitoring during Cycling Exercise

Given the importance of respiratory frequency (f_R) as a valid marker of physical effort, there is a growing interest in developing wearable devices measuring f_R in applied exercise settings. Biosensors measuring chest wall movements are attracting attention as they can be integrated into textiles, but their susceptibility to motion artefacts may limit their use in some sporting activities. Hence, there is a need to exploit sensors with signals minimally affected by motion artefacts. We present the design and testing of a smart facemask embedding a temperature biosensor for f_R monitoring during cycling exercise. After laboratory bench tests, the proposed solution was tested on cyclists during a ramp incremental frequency test (RIFT) and high-intensity interval training (HIIT), both indoors and outdoors. A reference flowmeter was used to validate the f_R extracted from the temperature respiratory signal. The smart facemask showed good performance, both at a breath-by-breath level (MAPE = 2.56% and 1.64% during RIFT and HIIT, respectively) and on 30 s average f_R values (MAPE = 0.37% and 0.23% during RIFT and HIIT, respectively). Both accuracy and precision (MOD ± LOAs) were generally superior to those of other devices validated during exercise. These findings have important implications for exercise testing and management in different populations.

Non-contact Vital Signs Monitoring in Paediatric Anaesthesia - Current Challenges and Future Direction

Non-contact vital sign monitoring is an area of increasing interest in the clinical scenario since it offers advantages over traditional monitoring using leads and wires. These advantages include reduction in transmission of infection and more freedom of movement. Yet there is a paucity of studies available in the clinical setting particularly in paediatric anaesthesia. This scoping review aims to investigate why contactless monitoring, specifically with red-green-blue cameras, is not implemented in mainstream practise. The challenges, drawbacks and limitations of non-contact vital sign monitoring, will be outlined, together with future direction on how it can potentially be implemented in the setting of paediatric anaesthesia, and in the critical care scenario.

Breathing Pattern Monitoring by Using Remote Sensors

The ability to continuously and unobtrusively monitor and classify breathing patterns can be very valuable for automated health assessments because respiration is tightly coupled to many physiological processes. Pathophysiological changes in these processes often manifest in altered breathing patterns and can thus be immediately detected. In order to develop a breathing pattern monitoring system, a study was conducted in which volunteer subjects were asked to breathe according to a predefined breathing protocol containing multiple breathing patterns while being recorded with color and thermal cameras. The recordings were used to develop and compare several respiratory signal extraction algorithms. An algorithm for the robust extraction of multiple respiratory features was developed and evaluated, capable of differentiating a wide range of respiratory patterns. These features were used to train a one vs. one multiclass support vector machine, which can distinguish between breathing patterns with an accuracy of 95.79 %. The recorded dataset was published to enable further improvement of contactless breathing pattern classification, especially for complex breathing patterns.

Electrodeless Heart and Respiratory Rate Estimation during Sleep Using a Single Fabric Band and Event-Based Edge Processing

Heart rate (HR) and respiratory rate (RR) are two vital parameters of the body medically used for diagnosing short/long-term illness. Out-of-the-body, non-skin-contact HR/RR measurement remains a challenge due to imprecise readings. "Invisible" wearables integrated into day-to-day garments have the potential to produce precise readings with a comfortable user experience. Sleep studies and patient monitoring benefit from "Invisibles" due to longer wearability without significant discomfort. This paper suggests a novel method to reduce the footprint of sleep monitoring devices. We use a single silver-coated nylon fabric band integrated into a substrate of a standard cotton/nylon garment as a resistive elastomer sensor to measure air and blood volume change across the chest. We introduce a novel event-based architecture to process data at the edge device and describe two algorithms to calculate real-time HR/RR on ARM Cortex-M3 and Cortex-M4F microcontrollers. RR estimations show a sensitivity of 99.03% and a precision of 99.03% for identifying individual respiratory peaks. The two algorithms used for HR calculation show a mean absolute error of 0.81 ± 0.97 and 0.86±0.61 beats/min compared with a gold standard ECG-based HR. The event-based algorithm converts the respiratory/pulse waveform into instantaneous events, therefore reducing the data size by 40-140 times and requiring 33% less power to process and transfer data. Furthermore, we show that events hold enough information to reconstruct the original waveform, retaining pulse and respiratory activity. We suggest fabric sensors and event-based algorithms would drastically reduce the device footprint and increase the performance for HR/RR estimations during sleep studies, providing a better user experience.

Multi-ROI Spectral Approach for the Continuous Remote Cardio-Respiratory Monitoring from Mobile Device Built-In Cameras

Heart rate (HR) and respiratory rate (f_R) can be estimated by processing videos framing the upper body and face regions without any physical contact with the subject. This paper proposed a technique for continuously monitoring HR and f_R via a multi-ROI approach based on the spectral analysis of RGB video frames recorded with a mobile device (i.e., a smartphone's camera). The respiratory signal was estimated by the motion of the chest, whereas the cardiac signal was retrieved from the pulsatile activity at the level of right and left cheeks and forehead. Videos were recorded from 18 healthy volunteers in four sessions with different user-camera distances (i.e., 0.5 m and 1.0 m) and illumination conditions (i.e., natural and artificial light). For HR estimation, three approaches were investigated based on single or multi-ROI approaches. A commercially available multiparametric device was used to record reference respiratory signals and electrocardiogram (ECG). The results demonstrated that the multi-ROI approach outperforms the single-ROI approach providing temporal trends of both the vital parameters comparable to those provided by the reference, with a mean absolute error (MAE) consistently below 1 breaths·min^-1 for f_R in all the scenarios, and a MAE between 0.7 bpm and 6 bpm for HR estimation, whose values increase at higher distances.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.