Remote monitoring of breathing dynamics using infrared thermography

Video-Based Multiphysiological Disentanglement and Remote Robust Estimation for Respiration

Remote noncontact respiratory rate estimation by facial visual information has great research significance, providing valuable priors for health monitoring, clinical diagnosis, and anti-fraud. However, existing studies suffer from disturbances in epidermal specular reflections induced by head movements and facial expressions. Furthermore, diffuse reflections of light in the skin-colored subcutaneous tissue caused by multiple time-varying physiological signals independent of breathing are entangled with the intention of the respiratory process, leading to confusion in current research. To address these issues, this article proposes a novel network for natural light video-based remote respiration estimation. Specifically, our model consists of a two-stage architecture that progressively implements vital measurements. The first stage adopts an encoder-decoder structure to recharacterize the facial motion frame differences of the input video based on the gradient binary state of the respiratory signal during inspiration and expiration. Then, the obtained generative mapping, which is disentangled from various time-varying interferences and is only linearly related to the respiratory state, is combined with the facial appearance in the second stage. To further improve the robustness of our algorithm, we design a targeted long-term temporal attention module and embed it between the two stages to enhance the network's ability to model the breathing cycle that occupies ultra many frames and to mine hidden timing change clues. We train and validate the proposed network on a series of publicly available respiration estimation datasets, and the experimental results demonstrate its competitiveness against the state-of-the-art breathing and physiological prediction frameworks.

Thermal Cameras for Continuous and Contactless Respiration Monitoring

Continuous respiration monitoring is an important tool in assessing the patient's health and diagnosing pulmonary, cardiovascular, and sleep-related breathing disorders. Various techniques and devices, both contact and contactless, can be used to monitor respiration. Each of these techniques can provide different types of information with varying accuracy. Thermal cameras have become a focal point in research due to their contactless nature, affordability, and the type of data they provide, i.e., information on respiration motion and respiration flow. Several studies have demonstrated the feasibility of this technology and developed robust algorithms to extract important information from thermal camera videos. This paper describes the current state-of-the-art in respiration monitoring using thermal cameras, dividing the system into acquiring data, defining and tracking the region of interest, and extracting the breathing signal and respiration rate. The approaches taken to address the various challenges, the limitations of these methods, and possible applications are discussed.

Predicting Sleep Quality through Biofeedback: A Machine Learning Approach Using Heart Rate Variability and Skin Temperature

Sleep quality (SQ) is a crucial aspect of overall health. Poor sleep quality may cause cognitive impairment, mood disturbances, and an increased risk of chronic diseases. Therefore, assessing sleep quality helps identify individuals at risk and develop effective interventions. SQ has been demonstrated to affect heart rate variability (HRV) and skin temperature even during wakefulness. In this perspective, using wearables and contactless technologies to continuously monitor HR and skin temperature is highly suited for assessing objective SQ. However, studies modeling the relationship linking HRV and skin temperature metrics evaluated during wakefulness to predict SQ are lacking. This study aims to develop machine learning models based on HRV and skin temperature that estimate SQ as assessed by the Pittsburgh Sleep Quality Index (PSQI). HRV was measured with a wearable sensor, and facial skin temperature was measured by infrared thermal imaging. Classification models based on unimodal and multimodal HRV and skin temperature were developed. A Support Vector Machine applied to multimodal HRV and skin temperature delivered the best classification accuracy, 83.4%. This study can pave the way for the employment of wearable and contactless technologies to monitor SQ for ergonomic applications. The proposed method significantly advances the field by achieving a higher classification accuracy than existing state-of-the-art methods. Our multimodal approach leverages the synergistic effects of HRV and skin temperature metrics, thus providing a more comprehensive assessment of SQ. Quantitative performance indicators, such as the 83.4% classification accuracy, underscore the robustness and potential of our method in accurately predicting sleep quality using non-intrusive measurements taken during wakefulness.

Respiratory muscles's thermographic analysis in asthmatic youth with and without bronchospasm induced by eucapnic voluntary hyperpnea

OBJECTIVE

To compare the thermographic pattern of regions of interest (ROI) of respiratory muscles in young asthmatics with and without bronchospasm induced by eucapnic voluntary hyperpnea (EVH).

MATERIALS AND METHODS

Cross-sectional study carried out with 55 young (55% male and 45% females) aged 12.5 ± 3.3 years, divided in nine nonasthmatics, 22 asthmatics without exercise-induced bronchospasm compatible response (EIB-cr) and 24 asthmatics with EIB-cr. The diagnosis of EIB was given to subjects with a fall in forced expiratory volume in the first second (FEV1) ≥ 10% compared to baseline. Thermographic recordings of respiratory muscles were delimited in ROI of the sternocleidomastoid (SCM), pectoral, and rectus abdominis intention area. Thermal captures and FEV1 were taken before and 5, 10, 15 and 30 min after EVH.

RESULTS

Twenty-four (52.1%) of asthmatics had EIB-cr. There was a decrease in temperature at 10 min after EVH test in the SCM, pectoral and rectus abdominis ROIs in all groups (both with p < 0.05). There was a decrease in temperature (% basal) in asthmatic with EIB-cr compared to nonasthmatics in the rectus abdominis area (p < 0.05).

CONCLUSION

There was a decrease in temperature in the ROIs of different muscle groups, especially in asthmatics. The greater drop in FEV1 observed in individuals with EIB-cr was initially associated with a decrease in skin temperature, with a difference between the nonasthmatics in the abdominal muscle area. It is likely that this decrease in temperature occurred due to a temporary displacement of blood flow to the most used muscle groups, with a decrease in the region of the skin evaluated in the thermography.

Integrating RGB-thermal image sensors for non-contact automatic respiration rate monitoring

Respiration rate (RR) holds significance as a human health indicator. Presently, the conventional RR monitoring system requires direct physical contact, which may cause discomfort and pain. Therefore, this paper proposes a non-contact RR monitoring system integrating RGB and thermal imaging through RGB-thermal image alignment. The proposed method employs an advanced image processing algorithm for automatic region of interest (ROI) selection. The experimental results demonstrated a close correlation and a lower error rate between measured thermal, measured RGB, and reference data. In summary, the proposed non-contact system emerges as a promising alternative to conventional contact-based approaches without the associated discomfort and pain.

Fusion of Multimodal Imaging and 3D Digitization Using Photogrammetry

Multimodal sensors capture and integrate diverse characteristics of a scene to maximize information gain. In optics, this may involve capturing intensity in specific spectra or polarization states to determine factors such as material properties or an individual's health conditions. Combining multimodal camera data with shape data from 3D sensors is a challenging issue. Multimodal cameras, e.g., hyperspectral cameras, or cameras outside the visible light spectrum, e.g., thermal cameras, lack strongly in terms of resolution and image quality compared with state-of-the-art photo cameras. In this article, a new method is demonstrated to superimpose multimodal image data onto a 3D model created by multi-view photogrammetry. While a high-resolution photo camera captures a set of images from varying view angles to reconstruct a detailed 3D model of the scene, low-resolution multimodal camera(s) simultaneously record the scene. All cameras are pre-calibrated and rigidly mounted on a rig, i.e., their imaging properties and relative positions are known. The method was realized in a laboratory setup consisting of a professional photo camera, a thermal camera, and a 12-channel multispectral camera. In our experiments, an accuracy better than one pixel was achieved for the data fusion using multimodal superimposition. Finally, application examples of multimodal 3D digitization are demonstrated, and further steps to system realization are discussed.

Breath Measurement Method for Synchronized Reproduction of Biological Tones in an Augmented Reality Auscultation Training System

An educational augmented reality auscultation system (EARS) is proposed to enhance the reality of auscultation training using a simulated patient. The conventional EARS cannot accurately reproduce breath sounds according to the breathing of a simulated patient because the system instructs the breathing rhythm. In this study, we propose breath measurement methods that can be integrated into the chest piece of a stethoscope. We investigate methods using the thoracic variations and frequency characteristics of breath sounds. An accelerometer, a magnetic sensor, a gyro sensor, a pressure sensor, and a microphone were selected as the sensors. For measurement with the magnetic sensor, we proposed a method by detecting the breathing waveform in terms of changes in the magnetic field accompanying the surface deformation of the stethoscope based on thoracic variations using a magnet. During breath sound measurement, the frequency spectra of the breath sounds acquired by the built-in microphone were calculated. The breathing waveforms were obtained from the difference in characteristics between the breath sounds during exhalation and inhalation. The result showed the average value of the correlation coefficient with the reference value reached 0.45, indicating the effectiveness of this method as a breath measurement method. And the evaluations suggest more accurate breathing waveforms can be obtained by selecting the measurement method according to breathing method and measurement point.

Anti-motion imaging photoplethysmography via self-adaptive multi-ROI tracking and selection

Objective.The imaging photoplethysmography (IPPG) technique allows people to measure heart rate (HR) from face videos. However, motion artifacts caused by rigid head movements and nonrigid facial muscular movements are one of the key challenges.Approach.This paper proposes a self-adaptive region of interest (ROI) pre-tracking and signal selection method to resist motion artifacts. Based on robust facial landmark detection, we split the whole facial skin (including the forehead, cheeks, and chin) symmetrically into small circular regions. And two symmetric sub-regions constitute a complete ROI. These ROIs are tracked and the motion state is simultaneously assessed to automatically determine the visibility of these ROIs. The obscured or invisible sub-regions will be discarded while the corresponding symmetric sub-regions will be retained as available ROIs to ensure the continuity of the IPPG signal. In addition, based on the frequency spectrum features of IPPG signals extracted from different ROIs, a self-adaptive selection module is constructed to select the optimum IPPG signal for HR calculation. All these operations are updated per frame dynamically for the real-time monitor.Results.Experimental results on the four public databases show that the IPPG signal derived by our proposed method exhibits higher quality for more accurate HR estimation. Compared with the previous method, metrics of the evaluated HR value on our approach demonstrates superior or comparable performance on PURE, VIPL-HR, UBFC-RPPG and MAHNOB-HCI datasets. For instance, the RMSEs on PURE, VIPL-HR, and UBFC-RPPG datasets decrease from 4.29, 7.62, and 3.80 to 4.15, 3.87, and 3.35, respectively.Significance.Our proposed method can help enhance the robustness of IPPG in real applications, especially given motion disturbances.

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require multiple steps that consist of image and signal processing. This might be difficult to deploy on edge devices that often do not have specialized digital signal processors (DSP). Therefore, the goal of this study is to develop a single neural network realizing the entire process of RR estimation in a single forward pass. The proposed solution builds on recent advances in video recognition, capturing both spatial and temporal information in a multi-path network. Both paths process the data at different sampling rates to capture rapid and slow changes that are associated with differences in the temperature of the nostril area during the breathing episodes. The preliminary results show that the introduced end-to-end solution achieves better performance compared to state-of-the-art methods, without requiring additional pre/post-processing steps and signal-processing techniques. In addition, the presented results demonstrate its robustness on low-resolution thermal video sequences that are often used at the embedded edge due to the size and power constraints of such systems. Taking that into account, the proposed approach has the potential for efficient and convenient respiratory rate estimation across various markets in solutions deployed locally, close to end users.

Time-domain Features of Angular-velocity Signals for Camera-based Respiratory RoI detection: A Clinical Study in NICU

Camera-based measurement of respiratory rate (RR) is emerging for preterm infants monitoring in Neonatal Intensive Care Units (NICU). Accurate detection of respiratory region of interest (Resp-RoI), e.g. thorax and abdomen of infants, is essential for achieving a fully-automatic solution and for high-quality RR estimation. However, the application of fast Fourier transform (FFT) for detecting Resp-RoI in premature infants may not be appropriate due to their irregular breathing patterns. This study proposes a new method for detecting Resp-RoIs in premature infants that uses time-domain features of angular-velocity of respiration. By fusing respiratory motion on orthogonal directions, the proposed method is more robust to variations of infant posture in the incubator.. In addition, using inter-beat interval (IBI) features in the time domain helps to distinguish between Resp-RoI and background. The proposed method was validated on 20 preterm infants in NICU. It obtains a clear improvement on Resp-RoI detection (RoI correspondence = 0.74) and RR estimation (MAE = 3.62 bpm) against the benchmarked approaches (maxFFT: RoI correspondence = 0.45, MAE = 5.61 bpm).

Non-invasive measurements of respiration and heart rate across wildlife species using Eulerian Video Magnification of infrared thermal imagery

BACKGROUND

An animal's metabolic rate, or energetic expenditure, both impacts and is impacted by interactions with its environment. However, techniques for obtaining measurements of metabolic rate are invasive, logistically difficult, and costly. Red-green-blue (RGB) imaging tools have been used in humans and select domestic mammals to accurately measure heart and respiration rate, as proxies of metabolic rate. The purpose of this study was to investigate if infrared thermography (IRT) coupled with Eulerian video magnification (EVM) would extend the applicability of imaging tools towards measuring vital rates in exotic wildlife species with different physical attributes.

RESULTS

We collected IRT and RGB video of 52 total species (39 mammalian, 7 avian, 6 reptilian) from 36 taxonomic families at zoological institutions and used EVM to amplify subtle changes in temperature associated with blood flow for respiration and heart rate measurements. IRT-derived respiration and heart rates were compared to 'true' measurements determined simultaneously by expansion of the ribcage/nostrils and stethoscope readings, respectively. Sufficient temporal signals were extracted for measures of respiration rate in 36 species (85% success in mammals; 50% success in birds; 100% success in reptiles) and heart rate in 24 species (67% success in mammals; 33% success in birds; 0% success in reptiles) using IRT-EVM. Infrared-derived measurements were obtained with high accuracy (respiration rate, mean absolute error: 1.9 breaths per minute, average percent error: 4.4%; heart rate, mean absolute error: 2.6 beats per minute, average percent error: 1.3%). Thick integument and animal movement most significantly hindered successful validation.

CONCLUSION

The combination of IRT with EVM analysis provides a non-invasive method to assess individual animal health in zoos, with great potential to monitor wildlife metabolic indices in situ.

Non-contact respiratory rate monitoring using thermal and visible imaging: a pilot study on neonates

Respiratory rate (RR) monitoring is essential in neonatal intensive care units. Despite its importance, RR is still monitored intermittently by manual counting instead of continuous monitoring due to the risk of skin damage with prolonged use of contact electrodes in preterm neonates and false signals due to displacement of electrodes. Thermal imaging has recently gained significance as a non-contact method for RR detection because of its many advantages. However, due to the lack of information in thermal images, the selection and tracking of the region of interest (ROI) in thermal images for neonates are challenging. This paper presents the integration of visible (RGB) and thermal (T) image sequences for the selection and tracking of ROI for breathing rate extraction. The deep-learning based tracking-by-detection approach is employed to detect the ROI in the RGB images, and it is mapped to the thermal images using the RGB-T image registration. The mapped ROI in thermal spectrum sequences gives the respiratory rate. The study was conducted first on healthy adults in different modes, including steady, motion, talking, and variable respiratory order. Subsequently, the method is tested on neonates in a clinical settings. The findings have been validated with a contact-based reference method.The average absolute error between the proposed and belt-based contact method in healthy adults reached 0.1 bpm and for more challenging conditions was approximately 1.5 bpm and 1.8 bpm, respectively. In the case of neonates, the average error is 1.5 bpm, which are promising results. The Bland–Altman analysis showed a good agreement of estimated RR with the reference method RR and this pilot study provided the evidence of using the proposed approach as a contactless method for the respiratory rate detection of neonates in clinical settings.

Breathing Pattern Monitoring by Using Remote Sensors

The ability to continuously and unobtrusively monitor and classify breathing patterns can be very valuable for automated health assessments because respiration is tightly coupled to many physiological processes. Pathophysiological changes in these processes often manifest in altered breathing patterns and can thus be immediately detected. In order to develop a breathing pattern monitoring system, a study was conducted in which volunteer subjects were asked to breathe according to a predefined breathing protocol containing multiple breathing patterns while being recorded with color and thermal cameras. The recordings were used to develop and compare several respiratory signal extraction algorithms. An algorithm for the robust extraction of multiple respiratory features was developed and evaluated, capable of differentiating a wide range of respiratory patterns. These features were used to train a one vs. one multiclass support vector machine, which can distinguish between breathing patterns with an accuracy of 95.79 %. The recorded dataset was published to enable further improvement of contactless breathing pattern classification, especially for complex breathing patterns.

Goulart A, Desuó IC, Gomes G. Application of thermography to estimate respiratory rate in the emergency room: The journal Temperature toolbox

Among the vital signs collected during hospital triage, respiratory rate is an important parameter associated with physiological, pathophysiological, and emotional changes. In recent years, the importance of its verification in emergency centers due to the severe acute respiratory syndrome 2 (SARS2) pandemic has become very clear, although it is still one of the least evaluated and collected vital signs. In this context, infrared imaging has been shown to be a reliable estimator of respiratory rate, with the advantage of not requiring physical contact with patients. The objective of this study was to evaluate the potential of analyzing a sequence of thermal images as an estimator of respiratory rate in the clinical routine of an emergency room. We used an infrared thermal camera (T540, Flir Systems) to obtain the respiratory rate data of 136 patients, based on nostrils' temperature fluctuation, during the peak of the COVID-19 pandemic in Brazil and compared it with the chest incursion count method, commonly employed in the emergency screening procedures. We found a good agreement between both methods, with Bland-Altman limits of agreement ranging from -4 to 4 min-1, no proportional bias (R2 = 0.021, p = 0.095), and a strong correlation between them (r = 0.95, p < 0.001). Our results suggest that infrared thermography has potential to be a good estimator of respiratory rate in the routine of an emergency room.

Accurate Contactless Sleep Apnea Detection Framework with Signal Processing and Machine Learning Methods

The detection of sleep apnea is critical for assessing sleep quality. It is also a proven biometric in diagnosing cardiovascular and other diseases. Recent studies have shown that radar-based non-contact vital sign monitoring system can effectively detect sleep apnea. However, the detection accuracy in the current study still needs to be improved. In this paper, we propose a sleep apnea detection framework based on FMCW radar. First, the radar system is employed to record the sleep data throughout the night with polysomnography (PSG) comparison. Then, in order to extract more accurate respiratory signal from the raw radar data, the signal processing methods are investigated to solve the observed discontinuity phenomenon. Finally, machine learning methods are adopted. The apneic and not-apneic events are classified accurately by selecting effective features of respiratory signal. As shown in the experimental results, the proposed system could achieve a good classification performance with an accuracy of 95.53%, a sensitivity of 72.60%, a specificity of 97.32%, a Kappa of 0.68, and an F-score of 0.84.

COVID-19 classification using thermal images

SIGNIFICANCE

There is a scarcity of published research on the potential role of thermal imaging in the remote detection of respiratory issues due to coronavirus disease-19 (COVID-19). This is a comprehensive study that explores the potential of this imaging technology resulting from its convenient aspects that make it highly accessible: it is contactless, noninvasive, and devoid of harmful radiation effects, and it does not require a complicated installation process.

AIM

We aim to investigate the role of thermal imaging, specifically thermal video, for the identification of SARS-CoV-2-infected people using infrared technology and to explore the role of breathing patterns in different parts of the thorax for the identification of possible COVID-19 infection.

APPROACH

We used signal moment, signal texture, and shape moment features extracted from five different body regions of interest (whole upper body, chest, face, back, and side) of images obtained from thermal video clips in which optical flow and super-resolution were used. These features were classified into positive and negative COVID-19 using machine learning strategies.

RESULTS

COVID-19 detection for male models [receiver operating characteristic (ROC) area under the ROC curve (AUC) = 0.605 95% confidence intervals (CI) 0.58 to 0.64] is more reliable than for female models (ROC AUC = 0.577 95% CI 0.55 to 0.61). Overall, thermal imaging is not very sensitive nor specific in detecting COVID-19; the metrics were below 60% except for the chest view from males.

CONCLUSIONS

We conclude that, although it may be possible to remotely identify some individuals affected by COVID-19, at this time, the diagnostic performance of current methods for body thermal imaging is not good enough to be used as a mass screening tool.

Infrared-based visualization of exhalation flows while wearing protective face masks

Since the onset of the COVID-19 pandemic, a large number of flow visualization procedures have been proposed to assess the effect of personal protective equipment on respiratory flows. This study suggests infrared thermography as a beneficial visualization technique because it is completely noninvasive and safe and, thus, can be used on live individuals rather than mannequins or lung simulators. Here, we examine the effect of wearing either of three popular face coverings (a surgical mask, a cloth mask, or an N95 respirator with an exhalation valve) on thermal signatures of exhaled airflows near a human face while coughing, talking, or breathing. The flow visualization using a mid-wave infrared camera captures the dynamics of thermal inhomogeneities induced by increased concentrations of carbon dioxide in the exhaled air. Thermal images demonstrate that both surgical and cloth face masks allow air leakage through the edges and the fabric itself, but they decrease the initial forward velocity of a cough jet by a factor of four. The N95 respirator, on the other hand, reduces the infrared emission of carbon dioxide near the person's face almost completely. This confirms that the N95-type mask may indeed lead to excessive inhalation of carbon dioxide as suggested by some recent studies.

Non-Contact Spirometry Using a Mobile Thermal Camera and AI Regression

Non-contact physiological measurements have been under investigation for many years, and among these measurements is non-contact spirometry, which could provide acute and chronic pulmonary disease monitoring and diagnosis. This work presents a feasibility study for non-contact spirometry measurements using a mobile thermal imaging system. Thermal images were acquired from 19 subjects for measuring the respiration rate and the volume of inhaled and exhaled air. A mobile application was built to measure the respiration rate and export the respiration signal to a personal computer. The mobile application acquired thermal video images at a rate of nine frames/second and the OpenCV library was used for localization of the area of interest (nose and mouth). Artificial intelligence regressors were used to predict the inhalation and exhalation air volume. Several regressors were tested and four of them showed excellent performance: random forest, adaptive boosting, gradient boosting, and decision trees. The latter showed the best regression results, with an R-square value of 0.9998 and a mean square error of 0.0023. The results of this study showed that non-contact spirometry based on a thermal imaging system is feasible and provides all the basic measurements that the conventional spirometers support.

Remote Monitoring of Patient Respiration with Mask Attachment—A Pragmatic Solution for Medical Facilities

Remote monitoring of vital signs in infectious patients minimizes the risks of viral transmissions to healthcare professionals. Donning face masks could reduce the risk of viral transmissions and is currently practiced in medical facilities. An acoustic-sensing device was attached to face masks to assist medical facilities in remotely monitoring patients’ respiration rate and wheeze occurrence. Usability and functionality studies of the modified face mask were evaluated on 16 healthy participants. Participants were blindfolded throughout the data collection process. Respiratory rates of the participants were evaluated for one minute. The wheeze detection algorithm was assessed by playing 176 wheezes and 176 normal breaths through a foam mannequin. No discomfort was reported from the participants who used the modified mask. The mean error of respiratory rate was found to be 2.0 ± 1.3 breath per minute. The overall accuracy of the wheeze detection algorithm was 91.9%. The microphone sensor that was first designed to be chest-worn has been proven versatile to be adopted as a mask attachment. The current findings support and suggest the use of the proposed mask attachment in medical facilities. This application can be especially helpful in managing a sudden influx of patients in the face of a pandemic.

Towards Accurate, Cost-Effective, Ultra-Low-Power and Non-Invasive Respiration Monitoring: A Reusable Wireless Wearable Sensor for an Off-the-Shelf KN95 Mask

Respiratory rate is a critical vital sign that indicates health condition, sleep quality, and exercise intensity. This paper presents a non-invasive, ultra-low-power, and cost-effective wireless wearable sensor, which is installed on an off-the-shelf KN95 mask to facilitate respiration monitoring. The sensing principle is based on the periodic airflow temperature variations caused by exhaled hot air and inhaled cool air in respiratory cycles. By measuring the periodic temperature variations at the exhalation valve of mask, the respiratory parameters can be accurately and reliably detected, regardless of body movements and breathing pathways through nose or mouth. Specifically, we propose a voltage divider with controllable resistors and corresponding selection criteria to improve the sensitivity of temperature measurement, a peak detection algorithm with spline interpolation to increase sampling period without reducing the detection accuracy, and effective low-power optimization measures to prolong the battery life. The experimental results have demonstrated the effectiveness of the proposed sensor, showing a small mean absolute error (MAE) of 0.449 bpm and a very low power consumption of 131.4 μW. As a high accuracy, low cost, low power, and reusable miniature wearing device for convenient respiration monitoring in daily life, the proposed sensor holds promise in real-world feasibility.

Automatic Separation of Respiratory Flow from Motion in Thermal Videos for Infant Apnea Detection

Both Respiratory Flow (RF) and Respiratory Motion (RM) are visible in thermal recordings of infants. Monitoring these two signals usually requires landmark detection for the selection of a region of interest. Other approaches combine respiratory signals coming from both RF and RM, obtaining a Mixed Respiratory (MR) signal. The detection and classification of apneas, particularly common in preterm infants with low birth weight, would benefit from monitoring both RF and RM, or MR, signals. Therefore, we propose in this work an automatic RF pixel detector not based on facial/body landmarks. The method is based on the property of RF pixels in thermal videos, which are in areas with a smooth circular gradient. We defined 5 features combined with the use of a bank of Gabor filters that together allow selection of the RF pixels. The algorithm was tested on thermal recordings of 9 infants amounting to a total of 132 min acquired in a neonatal ward. On average the percentage of correctly identified RF pixels was 84%. Obstructive Apneas (OAs) were simulated as a proof of concept to prove the advantage in monitoring the RF signal compared to the MR signal. The sensitivity in the simulated OA detection improved for the RF signal reaching 73% against the 23% of the MR signal. Overall, the method yielded promising results, although the positioning and number of cameras used could be further optimized for optimal RF visibility.

Home-Use and Real-Time Sleep-Staging System Based on Eye Masks and Mobile Devices with a Deep Learning Model

Purpose

Sleep is an important human activity. Comfortable sensing and accurate analysis in sleep monitoring is beneficial to many healthcare and medical applications. From 2020, owing to the COVID‑19 pandemic that spreads between people when they come into close physical contact with one another, the willingness to go to hospital for receiving care has reduced; care-at-home is the trend in modern healthcare. Therefore, a home-use and real-time sleep-staging system is developed in this paper.

Methods

We developed and implemented a real-time sleep staging system that integrates a wearable eye mask for high-quality electroencephalogram/electrooculogram measurement and a mobile device with MobileNETV2 deep learning model for sleep-stage identification. In the experiments, 25 all-night recordings were acquired, 17 of which were used for training, and the remaining eight were used for testing.

Results

The averaged scoring agreements for the wake, light sleep, deep sleep, and rapid eye movement stages were 85.20%, 87.17%, 82.87%, and 89.30%, respectively, for our system compared with the manual scoring of PSG recordings. In addition, the mean absolute errors of four objective sleep measurements, including sleep efficiency, total sleep time, sleep onset time, and wake after sleep onset time were 1.68%, 7.56 min, 5.50 min, and 3.94 min, respectively. No significant differences were observed between the proposed system and manual PSG scoring in terms of the percentage of each stage and the objective sleep measurements.

Conclusion

These experimental results demonstrate that our system provides high scoring agreements in sleep staging and unbiased sleep measurements owing to the use of EEG and EOG signals and powerful mobile computing based on deep learning networks. These results also suggest that our system is applicable for home-use real-time sleep monitoring.

Contact and Remote Breathing Rate Monitoring Techniques: A Review

Breathing rate monitoring is a must for hospitalized patients with the current coronavirus disease 2019 (COVID-19). We review in this paper recent implementations of breathing monitoring techniques, where both contact and remote approaches are presented. It is known that with non-contact monitoring, the patient is not tied to an instrument, which improves patients' comfort and enhances the accuracy of extracted breathing activity, since the distress generated by a contact device is avoided. Remote breathing monitoring allows screening people infected with COVID-19 by detecting abnormal respiratory patterns. However, non-contact methods show some disadvantages such as the higher set-up complexity compared to contact ones. On the other hand, many reported contact methods are mainly implemented using discrete components. While, numerous integrated solutions have been reported for non-contact techniques, such as continuous wave (CW) Doppler radar and ultrawideband (UWB) pulsed radar. These radar chips are discussed and their measured performances are summarized and compared.

Estimation of Respiratory Rate from Thermography Using Respiratory Likelihood Index

Respiration is a key vital sign used to monitor human health status. Monitoring respiratory rate (RR) under non-contact is particularly important for providing appropriate pre-hospital care in emergencies. We propose an RR estimation system using thermal imaging cameras, which are increasingly being used in the medical field, such as recently during the COVID-19 pandemic. By measuring temperature changes during exhalation and inhalation, we aim to track the respiration of the subject in a supine or seated position in real-time without any physical contact. The proposed method automatically selects the respiration-related regions from the detected facial regions and estimates the respiration rate. Most existing methods rely on signals from nostrils and require close-up or high-resolution images, while our method only requires the facial region to be captured. Facial region is detected using YOLO v3, an object detection model based on deep learning. The detected facial region is divided into subregions. By calculating the respiratory likelihood of each segmented region using the newly proposed index, called the Respiratory Quality Index, the respiratory region is automatically selected and the RR is estimated. An evaluation of the proposed RR estimation method was conducted on seven subjects in their early twenties, with four 15 s measurements being taken. The results showed a mean absolute error of 0.66 bpm. The proposed method can be useful as an RR estimation method.

Towards Continuous Camera-Based Respiration Monitoring in Infants

Aiming at continuous unobtrusive respiration monitoring, motion robustness is paramount. However, some types of motion can completely hide the respiration information and the detection of these events is required to avoid incorrect rate estimations. Therefore, this work proposes a motion detector optimized to specifically detect severe motion of infants combined with a respiration rate detection strategy based on automatic pixels selection, which proved to be robust to motion of the infants involving head and limbs. A dataset including both thermal and RGB (Red Green Blue) videos was used amounting to a total of 43 h acquired on 17 infants. The method was successfully applied to both RGB and thermal videos and compared to the chest impedance signal. The Mean Absolute Error (MAE) in segments where some motion is present was 1.16 and 1.97 breaths/min higher than the MAE in the ideal moments where the infants were still for testing and validation set, respectively. Overall, the average MAE on the testing and validation set are 3.31 breaths/min and 5.36 breaths/min, using 64.00% and 69.65% of the included video segments (segments containing events such as interventions were excluded based on a manual annotation), respectively. Moreover, we highlight challenges that need to be overcome for continuous camera-based respiration monitoring. The method can be applied to different camera modalities, does not require skin visibility, and is robust to some motion of the infants.

A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients

Infrared thermography for camera-based skin temperature measurement is increasingly used in medical practice, e.g., to detect fevers and infections, such as recently in the COVID-19 pandemic. This contactless method is a promising technology to continuously monitor the vital signs of patients in clinical environments. In this study, we investigated both skin temperature trend measurement and the extraction of respiration-related chest movements to determine the respiratory rate using low-cost hardware in combination with advanced algorithms. In addition, the frequency of medical examinations or visits to the patients was extracted. We implemented a deep learning-based algorithm for real-time vital sign extraction from thermography images. A clinical trial was conducted to record data from patients on an intensive care unit. The YOLOv4-Tiny object detector was applied to extract image regions containing vital signs (head and chest). The infrared frames were manually labeled for evaluation. Validation was performed on a hold-out test dataset of 6 patients and revealed good detector performance (0.75 intersection over union, 0.94 mean average precision). An optical flow algorithm was used to extract the respiratory rate from the chest region. The results show a mean absolute error of 2.69 bpm. We observed a computational performance of 47 fps on an NVIDIA Jetson Xavier NX module for YOLOv4-Tiny, which proves real-time capability on an embedded GPU system. In conclusion, the proposed method can perform real-time vital sign extraction on a low-cost system-on-module and may thus be a useful method for future contactless vital sign measurements.

Noncontact Sensing of Contagion

The World Health Organization (WHO) has declared COVID-19 a pandemic. We review and reduce the clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020. Vital signs associated with respiratory distress and fever, coughing, and visible infections have been reported. Fever screening by temperature monitoring is currently popular. However, improved noncontact detection is sought. Vital signs including heart rate and respiratory rate are affected by the condition. Cough, fatigue, and visible infections are also reported as common symptoms. There are non-contact methods for measuring vital signs remotely that have been shown to have acceptable accuracy, reliability, and practicality in some settings. Each has its pros and cons and may perform well in some challenges but be inadequate in others. Our review shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date, thermal cameras due to their potential to measure all likely symptoms on a single camera, especially temperature, and video cameras due to their availability, cost, adaptability, and compatibility. Substantial supply chain disruptions during the pandemic and the widespread nature of the problem means that cost-effectiveness and availability are important considerations.

Multispectral Video Fusion for Non-Contact Monitoring of Respiratory Rate and Apnea

Continuous monitoring of respiratory activity is desirable in many clinical applications to detect respiratory events. Non-contact monitoring of respiration can be achieved with near- and far-infrared spectrum cameras. However, current technologies are not sufficiently robust to be used in clinical applications. For example, they fail to estimate an accurate respiratory rate (RR) during apnea. We present a novel algorithm based on multispectral data fusion that aims at estimating RR also during apnea. The algorithm independently addresses the RR estimation and apnea detection tasks. Respiratory information is extracted from multiple sources and fed into an RR estimator and an apnea detector whose results are fused into a final respiratory activity estimation. We evaluated the system retrospectively using data from 30 healthy adults who performed diverse controlled breathing tasks while lying supine in a dark room and reproduced central and obstructive apneic events. Combining multiple respiratory information from multispectral cameras improved the root mean square error (RMSE) accuracy of the RR estimation from up to 4.64 monospectral data down to 1.60 breaths/min. The median F1 scores for classifying obstructive (0.75 to 0.86) and central apnea (0.75 to 0.93) also improved. Furthermore, the independent consideration of apnea detection led to a more robust system (RMSE of 4.44 vs. 7.96 breaths/min). Our findings may represent a step towards the use of cameras for vital sign monitoring in medical applications.

Detection of Respiratory Infections Using RGB-Infrared Sensors on Portable Device

Coronavirus Disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronaviruses 2 (SARS-CoV-2) has become a serious global pandemic in the past few months and caused huge loss to human society worldwide. For such a large-scale pandemic, early detection and isolation of potential virus carriers is essential to curb the spread of the pandemic. Recent studies have shown that one important feature of COVID-19 is the abnormal respiratory status caused by viral infections. During the pandemic, many people tend to wear masks to reduce the risk of getting sick. Therefore, in this paper, we propose a portable non-contact method to screen the health conditions of people wearing masks through analysis of the respiratory characteristics from RGB-infrared sensors. We first accomplish a respiratory data capture technique for people wearing masks by using face recognition. Then, a bidirectional GRU neural network with an attention mechanism is applied to the respiratory data to obtain the health screening result. The results of validation experiments show that our model can identify the health status of respiratory with 83.69% accuracy, 90.23% sensitivity and 76.31% specificity on the real-world dataset. This work demonstrates that the proposed RGB-infrared sensors on portable device can be used as a pre-scan method for respiratory infections, which provides a theoretical basis to encourage controlled clinical trials and thus helps fight the current COVID-19 pandemic. The demo videos of the proposed system are available at: https://doi.org/10.6084/m9.figshare.12028032.

Enhanced Contactless Vital Sign Estimation from Real-Time Multimodal 3D Image Data

The contactless estimation of vital signs using conventional color cameras and ambient light can be affected by motion artifacts and changes in ambient light. On both these problems, a multimodal 3D imaging system with an irritation-free controlled illumination was developed in this work. In this system, real-time 3D imaging was combined with multispectral and thermal imaging. Based on 3D image data, an efficient method was developed for the compensation of head motions, and novel approaches based on the use of 3D regions of interest were proposed for the estimation of various vital signs from multispectral and thermal video data. The developed imaging system and algorithms were demonstrated with test subjects, delivering a proof-of-concept.

Non-Contact Respiration Monitoring and Body Movements Detection for Sleep Using Thermal Imaging

Monitoring of respiration and body movements during sleep is a part of screening sleep disorders related to health status. Nowadays, thermal-based methods are presented to monitor the sleeping person without any sensors attached to the body to protect privacy. A non-contact respiration monitoring based on thermal videos requires visible facial landmarks like nostril and mouth. The limitation of these techniques is the failure of face detection while sleeping with a fixed camera position. This study presents the non-contact respiration monitoring approach that does not require facial landmark visibility under the natural sleep environment, which implies an uncontrolled sleep posture, darkness, and subjects covered with a blanket. The automatic region of interest (ROI) extraction by temperature detection and breathing motion detection is based on image processing integrated to obtain the respiration signals. A signal processing technique was used to estimate respiration and body movements information from a sequence of thermal video. The proposed approach has been tested on 16 volunteers, for which video recordings were carried out by themselves. The participants were also asked to wear the Go Direct respiratory belt for capturing reference data. The result revealed that our proposed measuring respiratory rate obtains root mean square error (RMSE) of 1.82±0.75 bpm. The advantage of this approach lies in its simplicity and accessibility to serve users who require monitoring the respiration during sleep without direct contact by themselves.

Multi-camera infrared thermography for infant respiration monitoring

Respiration is monitored in neonatal wards using chest impedance (CI), which is obtrusive and can cause skin damage to the infants. Therefore, unobtrusive solutions based on infrared thermography are being investigated. This work proposes an algorithm to merge multiple thermal camera views and automatically detect the pixels containing respiration motion or flow using three features. The method was tested on 152 minutes of recordings acquired on seven infants. We performed a comparison with the CI respiration rate yielding a mean absolute error equal to 2.07 breaths/min. Merging the three features resulted in reducing the dependency on the window size typical of spectrum-based features.

Driver Stress State Evaluation by Means of Thermal Imaging: A Supervised Machine Learning Approach Based on ECG Signal

Traffic accidents determine a large number of injuries, sometimes fatal, every year. Among other factors affecting a driver’s performance, an important role is played by stress which can decrease decision-making capabilities and situational awareness. In this perspective, it would be beneficial to develop a non-invasive driver stress monitoring system able to recognize the driver’s altered state. In this study, a contactless procedure for drivers’ stress state assessment by means of thermal infrared imaging was investigated. Thermal imaging was acquired during an experiment on a driving simulator, and thermal features of stress were investigated with comparison to a gold-standard metric (i.e., the stress index, SI) extracted from contact electrocardiography (ECG). A data-driven multivariate machine learning approach based on a non-linear support vector regression (SVR) was employed to estimate the SI through thermal features extracted from facial regions of interest (i.e., nose tip, nostrils, glabella). The predicted SI showed a good correlation with the real SI (r = 0.61, p = ~0). A two-level classification of the stress state (STRESS, SI ≥ 150, versus NO STRESS, SI < 150) was then performed based on the predicted SI. The ROC analysis showed a good classification performance with an AUC of 0.80, a sensitivity of 77%, and a specificity of 78%.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

Non-contact diagnosis of obstructive sleep apnea using impulse-radio ultra-wideband radar

While full-night polysomnography is the gold standard for the diagnosis of obstructive sleep apnea, its limitations include a high cost and first-night effects. This study developed an algorithm for the detection of respiratory events based on impulse-radio ultra-wideband radar and verified its feasibility for the diagnosis of obstructive sleep apnea. A total of 94 subjects were enrolled in this study (23 controls and 24, 14, and 33 with mild, moderate, and severe obstructive sleep apnea, respectively). Abnormal breathing detected by impulse-radio ultra-wideband radar was defined as a drop in the peak radar signal by ≥30% from that in the pre-event baseline. We compared the abnormal breathing index obtained from impulse-radio ultra-wideband radar and apnea-hypopnea index (AHI) measured from polysomnography. There was an excellent agreement between the Abnormal Breathing Index and AHI (intraclass correlation coefficient = 0.927). The overall agreements of the impulse-radio ultra-wideband radar were 0.93 for Model 1 (AHI ≥ 5), 0.91 for Model 2 (AHI ≥ 15), and 1 for Model 3 (AHI ≥ 30). Impulse-radio ultra-wideband radar accurately detected respiratory events (apneas and hypopneas) during sleep without subject contact. Therefore, impulse-radio ultra-wideband radar may be used as a screening tool for obstructive sleep apnea.

Human respiration monitoring using infrared thermography and artificial intelligence

The respiration rate (RR) is the most vital parameter used for the determination of human health. The most widely adopted techniques, used to monitor the RR are contact in nature and face many drawbacks. This paper reports the use of Infrared Thermography, in reliably monitoring the RR in a contact-less and non-invasive way. A thermal camera is used to monitor the variation in nasal temperature during respiration continuously. Further, the nostrils (region of interest) are tracked during head motion and object occlusion, by implementing a computer vision algorithm that makes use of 'Histogram of oriented gradients' and 'Support vector machine' (SVM). The signal to noise ratio (SNR) of the acquired breathing signals is very low; hence they are subjected to appropriate filtering methods. The filters are compared depending on the performance metrics such as SNR and Mean square error. The breaths per minute are obtained without any manual intervention by implementing the 'Breath detection algorithm' (BDA). This algorithm is implemented on 150 breathing signals and its performance is determined by computing the parameters such as Precision, Sensitivity, Spurious cycle rate, and Missed cycle rate values, obtained as 98.6%, 97.2%, 1.4%, and 2.8% respectively. The parameters obtained from the BDA are fed to the k-Nearest Neighbour (k-NN) and SVM classifiers, that determine whether the human volunteers have abnormal or normal respiration, or have Bradypnea (slow breathing), or Tachypnea (fast breathing). The Validation accuracies obtained are 96.25% and 99.5% with Training accuracies 97.75% and 99.4% for SVM and k-NN classifiers respectively. The Testing accuracies of the completely built SVM and k-NN classifiers are 96% and 99%, respectively. The various performance metrics like Sensitivity, Specificity, Precision, G-mean and F-measure are calculated as well, for every class, for both the classifiers. Finally, the Standard deviation values of the SVM and k-NN classifiers are computed and are obtained as 0.022 and 0.007, respectively. It is observed that the k-NN classifier shows a better performance compared to the SVM classifier. The pattern between the data points fed to the classifiers is viewed by making use of the t-Stochastic Neighbor Embedding algorithm. It is noticed from these plots that the separation between the data points belonging to different classes, improves and shows minimal overlap by increasing the perplexity value and number of iterations.

Estimation of respiratory rate using infrared video in an inpatient population: an observational study

Respiratory rate (RR) is one of the most sensitive markers of a deteriorating patient. Despite this, there is significant inter-observer discrepancy when measured by clinical staff, and modalities used in clinical practice such as ECG bioimpedance are prone to error. This study utilized infrared thermography (IRT) to measure RR in a critically ill population in the Intensive Care Unit. This study was carried out in a Single Hospital Centre. Respiratory rate in 27 extubated ICU patients was counted by two observers and compared to ECG Bioimpedance and IRT-derived RR at distances of 0.4-0.6 m and > 1 m respectively. IRT-derived RR using two separate computer vision algorithms outperformed ECG derived RR at distances of 0.4-0.6 m. Using an Autocorrelation estimator, mean bias was - 0.667 breaths/min. Using a Fast Fourier Transform estimator, mean bias was - 1.000 breaths/min. At distances greater than 1 m no statistically significant signal could be obtained. Over all frequencies, there was a significant relationship between the RR estimated using IRT and via manual counting, with Pearson correlation coefficients between 0.796 and 0.943 (p < 0.001). Correlation between counting and ECG-derived RR demonstrated significance only at > 19 bpm (r = 0.562, p = 0.029). Overall agreement between IRT-derived RR at distances of 0.4-0.6 m and gold standard counting was satisfactory, and outperformed ECG derived bioimpedance. Contactless IRT derived RR may be feasible as a routine monitoring modality in wards and subacute inpatient settings.

Multispectral camera fusion increases robustness of ROI detection for biosignal estimation with nearables in real-world scenarios

Thermal cameras enable non-contact estimation of the respiratory rate (RR). Accurate estimation of RR is highly dependent on the reliable detection of the region of interest (ROI), especially when using cameras with low pixel resolution. We present a novel approach for the automatic detection of the human nose ROI, based on facial landmark detection from an RGB camera that is fused with the thermal image after tracking. We evaluated the detection rate and spatial accuracy of the novel algorithm on recordings obtained from 16 subjects under challenging detection scenarios. Results show a high detection rate (median: 100%, 5^th-95^th percentile: 92%- 100%) and very good spatial accuracy with an average root mean square error of 2 pixels in the detected ROI center when compared to manual labeling. Therefore, the implementation of a multispectral camera fusion algorithm is a valid strategy to improve the reliability of non-contact RR estimation with nearable devices featuring thermal cameras.

Stable Contactless Sensing of Vital Signs Using RGB-Thermal Image Fusion System with Facial Tracking for Infection Screening

Infrared thermography (IRT) has been used to screen febrile passengers in international airports for over a decade. However, fever-based infection screening using IRT suffered from low sensitivity because measurements can be affected by ambient temperature, humidity, etc. In our previous study, we proposed an RGB-thermal image fusion system to measure vital signs i.e., the RGB camera detects tiny changes in color from facial skin, associated with blood flow, to estimate heart rate, and IRT senses temperature changes around the nasal area, caused by respiration, to measure respiratory rate). The inclusion of heart and respiratory rates lead to increased screening accuracy. In the present study, to promote the widespread use of our system in real-world settings, a face detection and tracking method was developed and implemented into the system, thereby enabling the accurate and stable measurement of vital signs. We assessed heart and respiratory rate estimation via an RGB-thermal image fusion system using Bland-Altman plots and statistical analysis.

Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review

Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.

Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review

OBJECTIVE

Over the last 15 years, developments in camera technology have coincided with increased availability and affordability. This has led to an increasing interest in using these technologies in healthcare settings. Image-based monitoring methods potentially allow multiple vital signs to be measured concurrently using a non-contact sensor. We have undertaken a systematic review of the current availability and performance of these monitoring methods.

APPROACH

A multiple database search was conducted using MEDLINE, Embase, CINAHL, Cochrane Library, OpenGrey, IEEE Xplore Library and ACM Digital Library to July 2018. We included studies comparing image-based heart rate, respiratory rate, oxygen saturation and blood pressure monitoring methods against one or more validated reference device(s). Each included study was assessed using the modified GRRAS criteria for reporting bias.

MAIN RESULTS

Of 30 279 identified studies, 161 were included in the final analysis. Twenty studies (20/161, 12%) were carried out on patients in clinical settings, while the remainder were conducted in academic settings using healthy volunteer populations. The 18-40 age group was best represented across the identified studies. One hundred and twenty studies (120/161, 75%) estimated heart rate, followed by 62 studies (62/161, 39%) estimating respiratory rate. Fewer studies focused on oxygen saturation (11/161, 7%) or blood pressure (6/161, 4%) estimation. Fifty-one heart rate studies (51/120, 43%) and 24 respiratory rate studies (24/62, 39%) used Bland-Altman analysis to report their results. Of the heart rate studies, 28 studies (28/51, 55%) showed agreement within industry standards of [Formula: see text]5 beats per minute. Only two studies achieved this within clinical settings. Of the respiratory rate studies, 13 studies (13/24, 54%) showed agreement within industry standards of [Formula: see text]3 breaths per minute, but only one study achieved this in a clinical setting. Statistical analysis was heterogeneous across studies with frequent inappropriate use of correlation. The majority of studies (99/161, 61%) monitored subjects for under 5 min. Three studies (3/161, 2%) monitored subjects for over 60 min, all of which were conducted in hospital settings.

SIGNIFICANCE

Heart rate and respiratory rate monitoring using video images is currently possible and performs within clinically acceptable limits under experimental conditions. Camera-derived estimates were less accurate in the proportion of studies conducted in clinical settings. We would encourage thorough reporting of the population studied, details of clinically relevant aspects of methodology, and the use of appropriate statistical methods in future studies. Systematic review registration: PROSPERO CRD42016029167 Protocol: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0615-3.

Perspective review of optical imaging in welfare assessment in animal-based research

To refine animal research, vital signs, activity, stress, and pain must be monitored. In chronic studies, some measures can be assessed using telemetry sensors. Although this methodology provides high-precision data, an initial surgery for device implantation is necessary, potentially leading to stress, wound infections, and restriction of motion. Recently, camera systems have been adapted for animal research. We give an overview of parameters that can be assessed using imaging in the visible, near-infrared, and thermal spectrum of light. It focuses on heart activity, respiration, oxygen saturation, and motion, as well as on wound analysis. For each parameter, we offer recommendations on the minimum technical requirements of appropriate systems, regions of interest, and light conditions, among others. In general, these systems demonstrate great performance. For heart and respiratory rate, the error was <4  beats  /  min and 5 breaths/min. Furthermore, the systems are capable of tracking animals during different behavioral tasks. Finally, studies indicate that inhomogeneous temperature distribution around wounds might be an indicator of (pending) infections. In sum, camera-based techniques have several applications in animal research. As vital parameters are currently only assessed in sedated animals, the next step should be the integration of these modalities in home-cage monitoring.

Unobtrusive Respiratory Flow Monitoring Using a Thermopile Array: A Feasibility Study

Low-resolution thermal cameras have already been used in the detection of respiratory flow. However, microbolometer technology has a high production cost compared to thermopile arrays. In this work, the feasibility of using a thermopile array to detect respiratory flow has been investigated in multiple settings. To prove the concept, we tested the detector on six healthy subjects. Our method automatically selects the region-of-interest by discriminating between sensor elements that output noise and flow-induced signals. The thermopile array yielded an average root mean squared error of 1.59 b r e a t h s p e r m i n u t e . Parameters such as distance, breathing rate, orientation, and oral or nasal breathing resulted in being fundamental in the detection of respiratory flow. The paper provides the proof-of-concept that low-cost thermopile-arrays can be used to monitor respiratory flow in a lab setting and without the need for facial landmark detection. Further development could provide a more attractive alternative for the earlier bolometer-based proposals.

Low-cost high integration IR polymer microlens array

In this Letter, a low-cost refractive convex microlens array device based on infrared a polymer is fabricated by a nanoimprinting technique. The device integrates more than 4000 microlenslets within a footprint of 10  mm×10  mm. The surface quality, spectral transmittance, imaging resolution, and surface damage threshold of the device have been fully characterized. The IR imaging and parallel laser inscription experiments confirm the remarkable optical performance of the fabricated device. Owing to the merits of high optical quality, low fluence lose, and simple fabrication, this device is promising in cutting-edge IR applications, such as IR imaging, laser fabrication, and so on.