Motion- Based Respiratory Rate Estimation with Motion Artifact Removal Using Video of Face and Upper Body

Video-Based Multiphysiological Disentanglement and Remote Robust Estimation for Respiration

Remote noncontact respiratory rate estimation by facial visual information has great research significance, providing valuable priors for health monitoring, clinical diagnosis, and anti-fraud. However, existing studies suffer from disturbances in epidermal specular reflections induced by head movements and facial expressions. Furthermore, diffuse reflections of light in the skin-colored subcutaneous tissue caused by multiple time-varying physiological signals independent of breathing are entangled with the intention of the respiratory process, leading to confusion in current research. To address these issues, this article proposes a novel network for natural light video-based remote respiration estimation. Specifically, our model consists of a two-stage architecture that progressively implements vital measurements. The first stage adopts an encoder-decoder structure to recharacterize the facial motion frame differences of the input video based on the gradient binary state of the respiratory signal during inspiration and expiration. Then, the obtained generative mapping, which is disentangled from various time-varying interferences and is only linearly related to the respiratory state, is combined with the facial appearance in the second stage. To further improve the robustness of our algorithm, we design a targeted long-term temporal attention module and embed it between the two stages to enhance the network's ability to model the breathing cycle that occupies ultra many frames and to mine hidden timing change clues. We train and validate the proposed network on a series of publicly available respiration estimation datasets, and the experimental results demonstrate its competitiveness against the state-of-the-art breathing and physiological prediction frameworks.

Enhance Heart Rate Measurement from Remote PPG with Head Motion Awareness from Image

Measurement of cardiac pulse rate through image-based remote photoplethysmography (rPPG) is drawing attention to applications of continuous health monitoring. Meanwhile, extracting clean rPPG signals and reliable heart rate (HR) remotely is challenging especially in real-life scenarios where users can move freely. In this paper, we leverage head motion information in the video to increase tolerance of vital estimation against the motion. A motion artifact classification model relying on rPPG and real-time head motion signals is developed to identify motion artifacts and reject outliers. We handcrafted 106 features and selected 20 features from both time and frequency domains. The model and methodology are validated comprehensively in a dataset of 30 subjects with 25 motion tasks in three motion intensity levels: low-motion, medium-motion, and high-motion. The motion-aware pipeline achieves a mean absolute error of 4.03 bpm for high-motion intensity tasks, improved by 31% by removing artifacts with specificity over 75%. In addition, the pipeline is tested with various light intensities to show that the motion detection is robust in darker conditions.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Estimating SpO2 with Deep Oxygen Desaturations from Facial Video Under Various Lighting Conditions: A Feasibility Study

This paper presents a feasibility study to collect data, process signals, and validate accuracy of peripheral oxygen saturation (SpO2) estimation from facial video in various lighting conditions. We collected facial videos using RGB camera, without auto-tuning, from subjects when they were breathing through a mouth tube with their nose clipped. The videos were record under four lighting conditions: warm color temperature and normal brightness, neutral color temperature and normal brightness, cool color temperature and normal brightness, neutral color temperature and dim brightness. The air inhaled by the subjects was manually controlled to gradually induce hypoxemia and lower subjects' SpO2 to as low as 81%. We first extracted the remote photoplethysmogram (rPPG) signals from the videos. We applied the principle of pulse oximetry and extracted the ratio of ratios (RoR) for two color combinations: Red/Blue and Red/Green. Next, we assessed SpO2 estimation accuracy against the ground truth, a Transfer Standard Pulse Oximeter. We have achieved an RMSE of 1.93% and a PCC of 0.97 under the warm color temperature and normal brightness lighting condition using leave-one-subject-out cross validation between two subjects. The results have demonstrated the feasibility to estimate SpO2 remotely and accurately using consumer level RGB camera with suitable camera configuration and lighting condition.Clinical Relevance- This work demonstrates that SpO2 can be estimated accurately using an RGB camera without auto-tuning and under warm color temperature, enabling continuous SpO2 monitoring applications that require noncontact sensing.

Contactless Monitoring of Respiratory Rate and Breathing Absence from Head Movements Using an RGB Camera

Contactless vital sign monitoring is more demanding for long-term, continuous, and unobtrusive measurements. Camera-based respiratory monitoring is receiving growing interest with advanced video technologies and computational power. The volume variations of the lungs for airflow changes create a periodic movement of the torso, but identifying the torso is more challenging than face detection in a video. In this paper, we present a unique approach to monitoring respiratory rate (RR) and breathing absence by leveraging head movements alone from an RGB video because respiratory motion also influences the head. Besides our novel RR estimation, an independent algorithm for breathing absence detection using signal feature extraction and machine learning techniques identifies an apnea event and improves overall RR estimation accuracy. The proposed approach was evaluated using videos from 30 healthy subjects who performed various breathing tasks. The breathing absence detector had 0.87 F1 score, 0.9 sensitivity, and 0.85 specificity. The accuracy of spontaneous breathing rate estimation increased from 2.46 to 1.91 bpm MAE and 3.54 to 2.7 bpm RMSE when combining the breathing absence result with the estimated RR.Clinical relevance- Our contactless respiratory monitoring can utilize a consumer RGB camera to offer a significant benefit in continuous monitoring of neonatal monitoring, sleep monitoring, telemedicine or telehealth, home fitness with mild physical movement, and emotion detection in the clinic and remote locations.

Contactless Technologies, Sensors, and Systems for Cardiac and Respiratory Measurement during Sleep: A Systematic Review

Sleep is essential to physical and mental health. However, the traditional approach to sleep analysis-polysomnography (PSG)-is intrusive and expensive. Therefore, there is great interest in the development of non-contact, non-invasive, and non-intrusive sleep monitoring systems and technologies that can reliably and accurately measure cardiorespiratory parameters with minimal impact on the patient. This has led to the development of other relevant approaches, which are characterised, for example, by the fact that they allow greater freedom of movement and do not require direct contact with the body, i.e., they are non-contact. This systematic review discusses the relevant methods and technologies for non-contact monitoring of cardiorespiratory activity during sleep. Taking into account the current state of the art in non-intrusive technologies, we can identify the methods of non-intrusive monitoring of cardiac and respiratory activity, the technologies and types of sensors used, and the possible physiological parameters available for analysis. To do this, we conducted a literature review and summarised current research on the use of non-contact technologies for non-intrusive monitoring of cardiac and respiratory activity. The inclusion and exclusion criteria for the selection of publications were established prior to the start of the search. Publications were assessed using one main question and several specific questions. We obtained 3774 unique articles from four literature databases (Web of Science, IEEE Xplore, PubMed, and Scopus) and checked them for relevance, resulting in 54 articles that were analysed in a structured way using terminology. The result was 15 different types of sensors and devices (e.g., radar, temperature sensors, motion sensors, cameras) that can be installed in hospital wards and departments or in the environment. The ability to detect heart rate, respiratory rate, and sleep disorders such as apnoea was among the characteristics examined to investigate the overall effectiveness of the systems and technologies considered for cardiorespiratory monitoring. In addition, the advantages and disadvantages of the considered systems and technologies were identified by answering the identified research questions. The results obtained allow us to determine the current trends and the vector of development of medical technologies in sleep medicine for future researchers and research.