Towards Continuous Camera-Based Respiration Monitoring in Infants

Thermal Cameras for Continuous and Contactless Respiration Monitoring

Continuous respiration monitoring is an important tool in assessing the patient's health and diagnosing pulmonary, cardiovascular, and sleep-related breathing disorders. Various techniques and devices, both contact and contactless, can be used to monitor respiration. Each of these techniques can provide different types of information with varying accuracy. Thermal cameras have become a focal point in research due to their contactless nature, affordability, and the type of data they provide, i.e., information on respiration motion and respiration flow. Several studies have demonstrated the feasibility of this technology and developed robust algorithms to extract important information from thermal camera videos. This paper describes the current state-of-the-art in respiration monitoring using thermal cameras, dividing the system into acquiring data, defining and tracking the region of interest, and extracting the breathing signal and respiration rate. The approaches taken to address the various challenges, the limitations of these methods, and possible applications are discussed.

Emerging innovations in neonatal monitoring: a comprehensive review of progress and potential for non-contact technologies

Continuous monitoring of high-risk neonates is essential for the timely management of medical conditions. However, the current reliance on wearable or contact sensor technologies for vital sign monitoring often leads to complications including discomfort, skin damage, and infections which can impede medical management, nursing care, and parental bonding. Moreover, the dependence on multiple devices is problematic since they are not interconnected or time-synchronized, use a variety of different wires and probes/sensors, and are designed based on adult specifications. Therefore, there is an urgent unmet need to enable development of wireless, non- or minimal-contact, and non-adhesive technologies capable of integrating multiple signals into a single platform, specifically designed for neonates. This paper summarizes the limitations of existing wearable devices for neonates, discusses advancements in non-contact sensor technologies, and proposes directions for future research and development.

The Derivation of Epigastric Motion to Assess Neonatal Breathing and Sleep: An Exploratory Study

INTRODUCTION

New non-medical monitors are offered for respiration monitoring of neonates. Epigastric motion during sleep was investigated by means of a wearable tracker in parallel to clinical monitoring.

COHORT

23 hospitalised neonates ready for discharge.

METHODS

A 3-axes-accelerometer and -gyroscope was placed in a standard epigastric position. Between two routine care rounds signals were recorded in parallel to monitoring of impedance pneumography (IP), ECG and pulse oximetry. Motion signals vs. time charts were evaluated using 10-min episodes and semiquantitatively assigned to breathing signal quality, regular breathing, periodic breathing and confounding artefacts. The results were compared with the impedance pneumographic data.

RESULTS

26 recordings (mean duration: 210 min/infant) were conducted without bradycardia or apnea alarm. The gestational age at birth ranged 28.9 to 41.1 and at recording from 35.6 to 42.3 postmenstrual weeks. Motion patterns of quiet sleep with regular breathing, periodic breathing and active sleep with confounding body movements were found. The longitudinal and transversal gyroscope axes resulted in best signal quality. Periodic breathing was found in up to 80% of episodes and decreased inversely with gestational age showing significantly more periodic breathing in preterm infants. Respiration signals of the gyroscope vs. IP showed a low bias and highly variating frequencies.

CONCLUSIONS

Standardized motion trackers may detect typical neonatal breathing and body-motion-patterns, that could help to classify neonatal sleep. Respiratory rates can only be determined during quiet sleep.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Continuous sensing and quantification of body motion in infants: A systematic review

Abnormal body motion in infants may be associated with neurodevelopmental delay or critical illness. In contrast to continuous patient monitoring of the basic vitals, the body motion of infants is only determined by discrete periodic clinical observations of caregivers, leaving the infants unattended for observation for a longer time. One step to fill this gap is to introduce and compare different sensing technologies that are suitable for continuous infant body motion quantification. Therefore, we conducted this systematic review for infant body motion quantification based on the PRISMA method (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). In this systematic review, we introduce and compare several sensing technologies with motion quantification in different clinical applications. We discuss the pros and cons of each sensing technology for motion quantification. Additionally, we highlight the clinical value and prospects of infant motion monitoring. Finally, we provide suggestions with specific needs in clinical practice, which can be referred by clinical users for their implementation. Our findings suggest that motion quantification can improve the performance of vital sign monitoring, and can provide clinical value to the diagnosis of complications in infants.

Respiratory Rate Extraction from Neonatal Near-Infrared Spectroscopy Signals

Background: Near-infrared spectroscopy (NIRS) relative concentration signals contain 'noise' from physiological processes such as respiration and heart rate. Simultaneous assessment of NIRS and respiratory rate (RR) using a single sensor would facilitate a perfectly time-synced assessment of (cerebral) physiology. Our aim was to extract respiratory rate from cerebral NIRS intensity signals in neonates admitted to a neonatal intensive care unit (NICU). Methods: A novel algorithm, NRR (NIRS RR), is developed for extracting RR from NIRS signals recorded from critically ill neonates. In total, 19 measurements were recorded from ten neonates admitted to the NICU with a gestational age and birth weight of 38 ± 5 weeks and 3092 ± 990 g, respectively. We synchronously recorded NIRS and reference RR signals sampled at 100 Hz and 0.5 Hz, respectively. The performance of the NRR algorithm is assessed in terms of the agreement and linear correlation between the reference and extracted RRs, and it is compared statistically with that of two existing methods. Results: The NRR algorithm showed a mean error of 1.1 breaths per minute (BPM), a root mean square error of 3.8 BPM, and Bland-Altman limits of agreement of 6.7 BPM averaged over all measurements. In addition, a linear correlation of 84.5% (p < 0.01) was achieved between the reference and extracted RRs. The statistical analyses confirmed the significant (p < 0.05) outperformance of the NRR algorithm with respect to the existing methods. Conclusions: We showed the possibility of extracting RR from neonatal NIRS in an intensive care environment, which showed high correspondence with the reference RR recorded. Adding the NRR algorithm to a NIRS system provides the opportunity to record synchronously different physiological sources of information about cerebral perfusion and respiration by a single monitoring system. This allows for a concurrent integrated analysis of the impact of breathing (including apnea) on cerebral hemodynamics.

Non-contact respiratory rate monitoring using thermal and visible imaging: a pilot study on neonates

Respiratory rate (RR) monitoring is essential in neonatal intensive care units. Despite its importance, RR is still monitored intermittently by manual counting instead of continuous monitoring due to the risk of skin damage with prolonged use of contact electrodes in preterm neonates and false signals due to displacement of electrodes. Thermal imaging has recently gained significance as a non-contact method for RR detection because of its many advantages. However, due to the lack of information in thermal images, the selection and tracking of the region of interest (ROI) in thermal images for neonates are challenging. This paper presents the integration of visible (RGB) and thermal (T) image sequences for the selection and tracking of ROI for breathing rate extraction. The deep-learning based tracking-by-detection approach is employed to detect the ROI in the RGB images, and it is mapped to the thermal images using the RGB-T image registration. The mapped ROI in thermal spectrum sequences gives the respiratory rate. The study was conducted first on healthy adults in different modes, including steady, motion, talking, and variable respiratory order. Subsequently, the method is tested on neonates in a clinical settings. The findings have been validated with a contact-based reference method.The average absolute error between the proposed and belt-based contact method in healthy adults reached 0.1 bpm and for more challenging conditions was approximately 1.5 bpm and 1.8 bpm, respectively. In the case of neonates, the average error is 1.5 bpm, which are promising results. The Bland–Altman analysis showed a good agreement of estimated RR with the reference method RR and this pilot study provided the evidence of using the proposed approach as a contactless method for the respiratory rate detection of neonates in clinical settings.

A Comparison of Video-based Methods for Neonatal Body Motion Detection

Preterm infants in a neonatal intensive care unit (NICU) are continuously monitored for their vital signs, such as heart rate and oxygen saturation. Body motion patterns are documented intermittently by clinical observations. Changing motion patterns in preterm infants are associated with maturation and clinical events such as late-onset sepsis and seizures. However, continuous motion monitoring in the NICU setting is not yet performed. Video-based motion monitoring is a promising method due to its non-contact nature and therefore unobtrusiveness. This study aims to determine the feasibility of simple video-based methods for infant body motion detection. We investigated and compared four methods to detect the motion in videos of infants, using two datasets acquired with different types of cameras. The thermal dataset contains 32 hours of annotated videos from 13 infants in open beds. The RGB dataset contains 9 hours of annotated videos from 5 infants in incubators. The compared methods include background substruction (BS), sparse optical flow (SOF), dense optical flow (DOF), and oriented FAST and rotated BRIEF (ORB). The detection performance and computation time were evaluated by the area under receiver operating curves (AUC) and run time. We conducted experiments to detect motion and gross motion respectively. In the thermal dataset, the best performance of both experiments is achieved by BS with mean (standard deviation) AUCs of 0.86 (0.03) and 0.93 (0.03). In the RGB dataset, SOF outperforms the other methods in both experiments with AUCs of 0.82 (0.10) and 0.91 (0.05). All methods are efficient to be integrated into a camera system when using low-resolution thermal cameras.

Camera fusion for real-time temperature monitoring of neonates using deep learning

Abstract

The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module.

Graphical abstract

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

Contactless radar-based breathing monitoring of premature infants in the neonatal intensive care unit

Vital sign monitoring systems are essential in the care of hospitalized neonates. Due to the immaturity of their organs and immune system, premature infants require continuous monitoring of their vital parameters and sensors need to be directly attached to their fragile skin. Besides mobility restrictions and stress, these sensors often cause skin irritation and may lead to pressure necrosis. In this work, we show that a contactless radar-based approach is viable for breathing monitoring in the Neonatal intensive care unit (NICU). For the first time, different scenarios common to the NICU daily routine are investigated, and the challenges of monitoring in a real clinical setup are addressed through different contributions in the signal processing framework. Rather than just discarding measurements under strong interference, we present a novel random body movement mitigation technique based on the time-frequency decomposition of the recovered signal. In addition, we propose a simple and accurate frequency estimator which explores the harmonic structure of the breathing signal. As a result, the proposed radar-based solution is able to provide reliable breathing frequency estimation, which is close to the reference cabled device values most of the time. Our findings shed light on the strengths and limitations of this technology and lay the foundation for future studies toward a completely contactless solution for vital signs monitoring.

Video-Based Automatic Baby Motion Analysis for Early Neurological Disorder Diagnosis: State of the Art and Future Directions

Neurodevelopmental disorders (NDD) are impairments of the growth and development of the brain and/or central nervous system. In the light of clinical findings on early diagnosis of NDD and prompted by recent advances in hardware and software technologies, several researchers tried to introduce automatic systems to analyse the baby's movement, even in cribs. Traditional technologies for automatic baby motion analysis leverage contact sensors. Alternatively, remotely acquired video data (e.g., RGB or depth) can be used, with or without active/passive markers positioned on the body. Markerless approaches are easier to set up and maintain (without any human intervention) and they work well on non-collaborative users, making them the most suitable technologies for clinical applications involving children. On the other hand, they require complex computational strategies for extracting knowledge from data, and then, they strongly depend on advances in computer vision and machine learning, which are among the most expanding areas of research. As a consequence, also markerless video-based analysis of movements in children for NDD has been rapidly expanding but, to the best of our knowledge, there is not yet a survey paper providing a broad overview of how recent scientific developments impacted it. This paper tries to fill this gap and it lists specifically designed data acquisition tools and publicly available datasets as well. Besides, it gives a glimpse of the most promising techniques in computer vision, machine learning and pattern recognition which could be profitably exploited for children motion analysis in videos.

Polymeric piezoresistive airflow sensor to monitor respiratory patterns

Monitoring human respiratory patterns is of great importance as it gives essential information for various medical conditions, e.g. sleep apnoea syndrome and chronic obstructive pulmonary disease and asthma, etc. Herein, we have developed a polymeric airflow sensor based on nanocomposites of vertically grown graphene nanosheets (VGNs) with polydimethylsiloxane (PDMS) and explored their applications in monitoring human respiration. The sensing performance of the VGNs/PDMS nanocomposite was characterized by exposing to a range of airflow rates (20-130 l min-1), and a linear performance with high sensitivity and low response time (mostly below 1 s) was observed. To evaluate the experimental results, finite-element simulation models were developed in the COMSOL Multiphysics package. The piezoresistive properties of VGNs/PDMS thin film and fluid-solid interaction were thoroughly studied. Laser Doppler vibrometry measures of sensor tip displacement closely approximated simulated deflection results and validated the dynamic response of the sensor. By comparing the proposed sensor and some other airflow sensors in the literature, it is concluded that the VGNs/PDMS airflow sensor has excellent features in terms of sensor height, detection range and sensitivity. The potential application of the VGNs/PDMS airflow sensor in detecting the respiration pattern of human exercises like walking, jogging and running has been demonstrated.

Body Motion Detection in Neonates Based on Motion Artifacts in Physiological Signals from a Clinical Patient Monitor

Motion patterns in newborns contain important information. Motion patterns change upon maturation and changes in the nature of motion may precede critical clinical events such as the onset of sepsis, seizures and apneas. However, in clinical practice, motion monitoring is still limited to observations by caregivers. In this study, we investigated a practical yet reliable method for motion detection using routinely used physiological signals in the patient monitor. Our method calculated motion measures with a continuous wavelet transform (CWT) and a signal instability index (SII) to detect gross-motor motion in 15 newborns using 40 hours of physiological data with annotated videos. We compared the performance of these measures on three signal modalities (electrocardiogram ECG, chest impedance, and photo plethysmography). In addition, we investigated whether their combinations increased performance. The best performance was achieved with the ECG signal with a median (interquartile range, IQR) area under receiver operating curve (AUC) of 0.92(0.87-0.95), but differences were small as both measures had a robust performance on all signal modalities. We then applied the algorithm on combined measures and modalities. The full combination outperformed all single-modal methods with a median (IQR) AUC of 0.95(0.91-0.96) when discriminating gross-motor motion from still. Our study demonstrates the feasibility of gross-motor motion detection method based on only clinically-available vital signs and that best results can be obtained by combining measures and vital signs.

Automatic Separation of Respiratory Flow from Motion in Thermal Videos for Infant Apnea Detection

Both Respiratory Flow (RF) and Respiratory Motion (RM) are visible in thermal recordings of infants. Monitoring these two signals usually requires landmark detection for the selection of a region of interest. Other approaches combine respiratory signals coming from both RF and RM, obtaining a Mixed Respiratory (MR) signal. The detection and classification of apneas, particularly common in preterm infants with low birth weight, would benefit from monitoring both RF and RM, or MR, signals. Therefore, we propose in this work an automatic RF pixel detector not based on facial/body landmarks. The method is based on the property of RF pixels in thermal videos, which are in areas with a smooth circular gradient. We defined 5 features combined with the use of a bank of Gabor filters that together allow selection of the RF pixels. The algorithm was tested on thermal recordings of 9 infants amounting to a total of 132 min acquired in a neonatal ward. On average the percentage of correctly identified RF pixels was 84%. Obstructive Apneas (OAs) were simulated as a proof of concept to prove the advantage in monitoring the RF signal compared to the MR signal. The sensitivity in the simulated OA detection improved for the RF signal reaching 73% against the 23% of the MR signal. Overall, the method yielded promising results, although the positioning and number of cameras used could be further optimized for optimal RF visibility.

Non-Contact Automatic Vital Signs Monitoring of Infants in a Neonatal Intensive Care Unit Based on Neural Networks

Infants with fragile skin are patients who would benefit from non-contact vital sign monitoring due to the avoidance of potentially harmful adhesive electrodes and cables. Non-contact vital signs monitoring has been studied in clinical settings in recent decades. However, studies on infants in the Neonatal Intensive Care Unit (NICU) are still limited. Therefore, we conducted a single-center study to remotely monitor the heart rate (HR) and respiratory rate (RR) of seven infants in NICU using a digital camera. The region of interest (ROI) was automatically selected using a convolutional neural network and signal decomposition was used to minimize the noise artefacts. The experimental results have been validated with the reference data obtained from an ECG monitor. They showed a strong correlation using the Pearson correlation coefficients (PCC) of 0.9864 and 0.9453 for HR and RR, respectively, and a lower error rate with RMSE 2.23 beats/min and 2.69 breaths/min between measured data and reference data. A Bland-Altman analysis of the data also presented a close correlation between measured data and reference data for both HR and RR. Therefore, this technique may be applicable in clinical environments as an economical, non-contact, and easily deployable monitoring system, and it also represents a potential application in home health monitoring.