Remote Blood Oxygen Estimation From Videos Using Neural Networks

Exploring the Dynamics of Canine-Assisted Interactions: A Wearable Approach to Understanding Interspecies Well-Being

Canine-assisted interactions (CAIs) have been explored to offer therapeutic benefits to human participants in various contexts, from addressing cancer-related fatigue to treating post-traumatic stress disorder. Despite their widespread adoption, there are still unresolved questions regarding the outcomes for both humans and animals involved in these interactions. Previous attempts to address these questions have suffered from core methodological weaknesses, especially due to absence of tools for an efficient objective evaluation and lack of focus on the canine perspective. In this article, we present a first-of-its-kind system and study to collect simultaneous and continuous physiological data from both of the CAI interactants. Motivated by our extensive field reviews and stakeholder feedback, this comprehensive wearable system is composed of custom-designed and commercially available sensor devices. We performed a repeated-measures pilot study, to combine data collected via this system with a novel dyadic behavioral coding method and short- and long-term surveys. We evaluated these multimodal data streams independently, and we further correlated the psychological, physiological, and behavioral metrics to better elucidate the outcomes and dynamics of CAIs. Confirming previous field results, human electrodermal activity is the measure most strongly distinguished between the dyads' non-interaction and interaction periods. Valence, arousal, and the positive affect of the human participant significantly increased during interaction with the canine participant. Also, we observed in our pilot study that (a) the canine heart rate was more dynamic than the human's during interactions, (b) the surveys proved to be the best indicator of the subjects' affective state, and (c) the behavior coding approaches best tracked the bond quality between the interacting dyads. Notably, we found that most of the interaction sessions were characterized by extended neutral periods with some positive and negative peaks, where the bonded pairs might display decreased behavioral synchrony. We also present three new representations of the internal and overall dynamics of CAIs for adoption by the broader field. Lastly, this paper discusses ongoing options for further dyadic analysis, interspecies emotion prediction, integration of contextually relevant environmental data, and standardization of human-animal interaction equipment and analytical approaches. Altogether, this work takes a significant step forward on a promising path to our better understanding of how CAIs improve well-being and how interspecies psychophysiological states can be appropriately measured.

Evaluating AI Methods for Pulse Oximetry: Performance, Clinical Accuracy, and Comprehensive Bias Analysis

Blood oxygen saturation (SpO2) is vital for patient monitoring, particularly in clinical settings. Traditional SpO2 estimation methods have limitations, which can be addressed by analyzing photoplethysmography (PPG) signals with artificial intelligence (AI) techniques. This systematic review, following PRISMA guidelines, analyzed 183 unique references from WOS, PubMed, and Scopus, with 26 studies meeting the inclusion criteria. The review examined AI models, key features, oximeters used, datasets, tested saturation intervals, and performance metrics while also assessing bias through the QUADAS-2 criteria. Linear regression models and deep neural networks (DNNs) emerged as the leading AI methodologies, utilizing features such as statistical metrics, signal-to-noise ratios, and intricate waveform morphology to enhance accuracy. Gaussian Process models, in particular, exhibited superior performance, achieving Mean Absolute Error (MAE) values as low as 0.57% and Root Mean Square Error (RMSE) as low as 0.69%. The bias analysis highlighted the need for better patient selection, reliable reference standards, and comprehensive SpO2 intervals to improve model generalizability. A persistent challenge is the reliance on non-invasive methods over the more accurate arterial blood gas analysis and the limited datasets representing diverse physiological conditions. Future research must focus on improving reference standards, test protocols, and addressing ethical considerations in clinical trials. Integrating AI with traditional physiological models can further enhance SpO2 estimation accuracy and robustness, offering significant advancements in patient care.

Innovative approaches in imaging photoplethysmography for remote blood oxygen monitoring

Peripheral Capillary Oxygen Saturation (SpO2) has received increasing attention during the COVID-19 pandemic. Clinical investigations have demonstrated that individuals afflicted with COVID-19 exhibit notably reduced levels of SpO2 before the deterioration of their health status. To cost-effectively enable individuals to monitor their SpO2, this paper proposes a novel neural network model named "ITSCAN" based on Temporal Shift Module. Benefiting from the widespread use of smartphones, this model can assess an individual's SpO2 in real time, utilizing standard facial video footage, with a temporal granularity of seconds. The model is interweaved by two distinct branches: the motion branch, responsible for extracting spatiotemporal data features and the appearance branch, focusing on the correlation between feature channels and the location information of feature map using coordinate attention mechanisms. Accordingly, the SpO2 estimator generates the corresponding SpO2 value. This paper summarizes for the first time 5 loss functions commonly used in the SpO2 estimation model. Subsequently, a novel loss function has been contributed through the examination of various combinations and careful selection of hyperparameters. Comprehensive ablation experiments analyze the independent impact of each module on the overall model performance. Finally, the experimental results based on the public dataset (VIPL-HR) show that our model has obvious advantages in MAE (1.10%) and RMSE (1.19%) compared with related work, which implies more accuracy of the proposed method to contribute to public health.

Leveraging 3D convolutional neural network and 3D visible-near-infrared multimodal imaging for enhanced contactless oximetry

Significance

Monitoring oxygen saturation (SpO 2 ) is important in healthcare, especially for diagnosing and managing pulmonary diseases. Non-contact approaches broaden the potential applications ofSpO 2 measurement by better hygiene, comfort, and capability for long-term monitoring. However, existing studies often encounter challenges such as lower signal-to-noise ratios and stringent environmental conditions.

Aim

We aim to develop and validate a contactlessSpO 2 measurement approach using 3D convolutional neural networks (3D CNN) and 3D visible-near-infrared (VIS-NIR) multimodal imaging, to offer a convenient, accurate, and robust alternative forSpO 2 monitoring.

Approach

We propose an approach that utilizes a 3D VIS-NIR multimodal camera system to capture facial videos, in whichSpO 2 is estimated through 3D CNN by simultaneously extracting spatial and temporal features. Our approach includes registration of multimodal images, tracking of the 3D region of interest, spatial and temporal preprocessing, and 3D CNN-based feature extraction andSpO 2 regression.

Results

In a breath-holding experiment involving 23 healthy participants, we obtained multimodal video data with referenceSpO 2 values ranging from 80% to 99% measured by pulse oximeter on the fingertip. The approach achieved a mean absolute error (MAE) of 2.31% and a Pearson correlation coefficient of 0.64 in the experiment, demonstrating good agreement with traditional pulse oximetry. The discrepancy of estimatedSpO 2 values was within 3% of the referenceSpO 2 for ∼ 80 % of all 1-s time points. Besides, in clinical trials involving patients with sleep apnea syndrome, our approach demonstrated robust performance, with an MAE of less than 2% inSpO 2 estimations compared to gold-standard polysomnography.

Conclusions

The proposed approach offers a promising alternative for non-contact oxygen saturation measurement with good sensitivity to desaturation, showing potential for applications in clinical settings.

CL-SPO2Net: Contrastive Learning Spatiotemporal Attention Network for Non-Contact Video-Based SpO2 Estimation

Video-based peripheral oxygen saturation (SpO2) estimation, utilizing solely RGB cameras, offers a non-contact approach to measuring blood oxygen levels. Previous studies set a stable and unchanging environment as the premise for non-contact blood oxygen estimation. Additionally, they utilized a small amount of labeled data for system training and learning. However, it is challenging to train optimal model parameters with a small dataset. The accuracy of blood oxygen detection is easily affected by ambient light and subject movement. To address these issues, this paper proposes a contrastive learning spatiotemporal attention network (CL-SPO2Net), an innovative semi-supervised network for video-based SpO2 estimation. Spatiotemporal similarities in remote photoplethysmography (rPPG) signals were found in video segments containing facial or hand regions. Subsequently, integrating deep neural networks with machine learning expertise enabled the estimation of SpO2. The method had good feasibility in the case of small-scale labeled datasets, with the mean absolute error between the camera and the reference pulse oximeter of 0.85% in the stable environment, 1.13% with lighting fluctuations, and 1.20% in the facial rotation situation.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Patent and Bibliometric Analysis of the Scientific Landscape of the Use of Pulse Oximeters and Their Prospects in the Field of Digital Medicine

This study conducted a comprehensive patent and bibliometric analysis to elucidate the evolving scientific landscape surrounding the development and application of pulse oximeters, including in the field of digital medicine. Utilizing data from the Lens database for the period of 2000-2023, we identified the United States, China, the Republic of Korea, Japan, Canada, Australia, Taiwan, and the United Kingdom as the predominant countries in patent issuance for pulse oximeter technology. Our bibliometric analysis revealed a consistent temporal trend in both the volume of publications and citations, underscoring the growing importance of pulse oximeters in digitally-enabled medical practice. Using the VOSviewer software(version 1.6.18), we discerned six primary research clusters: (1) measurement accuracy; (2) integration with the Internet of Things; (3) applicability across diverse pathologies; (4) telemedicine and mobile applications; (5) artificial intelligence and deep learning; and (6) utilization in anesthesiology, resuscitation, and intensive care departments. The findings of this study indicate the prospects for leveraging digital technologies in the use of pulse oximetry in various fields of medicine, with implications for advancing the understanding, diagnosis, prevention, and treatment of cardio-respiratory pathologies. The conducted patent and bibliometric analysis allowed the identification of technical solutions to reduce the risks associated with pulse oximetry: improving precision and validity, technically improved clinical diagnostic use, and the use of machine learning.

Impact of Different Skin Penetration Depths of Red and Green Wavelengths on Camera-based SpO2 Measurement

Remote camera-based estimation of blood oxygen saturation (SpO2) using visible lights has been studied recently, typically for red (660 nm) and green (550 nm) wavelengths. This paper investigates the impact of different skin penetration depths of red and green wavelengths on the SpO2 estimation based on mathematical modeling and experiments, where the SpO2-calibritability between two illumination setups, narrow-band red/green and narrow-band red/infrared, are statistically compared using the "ratio-of-ratios" method. The results show that the performance of the setup using red/green is less consistent among 17 volunteers than the setup using red/infrared, and larger SpO2 disparity between different skin regions (by SpO2 imaging) have been found for individuals in the red/green wavelengths setup. The use of visible light (red and green) may impose a risk of SpO2 calibration due to the different skin penetration depths of these two wavelengths.