Unobtrusive Vital Sign Monitoring in Automotive Environments-A Review

Respiratory Monitoring with Textile Inductive Electrodes in Driving Applications: Effect of Electrode's Positioning and Form Factor on Signal Quality

This paper provides insights into where and how to integrate textile inductive electrodes into a car to record optimal-quality respiratory signals. Electrodes of various shapes and sizes were integrated into the seat belt and the seat back of a driving simulator car seat. The electrodes covered various parts of the body: upper back, middle back, lower back, chest, and waist. Three subjects completed driving circuits with their breathing signals being recorded. In general, signal quality while driving versus sitting still was similar, compared to a previous study of ours with no body movements. In terms of positioning, electrodes on seat belt provided better signal quality compared to seat back. Signal quality was directly proportional to electrode's height on the back, with upper back outperforming both middle and lower back. Electrodes on the waist provided either similar or superior signal quality compared to electrodes on the chest. In terms of form factor, rectangular shape outperformed circular shape on seat back. Signal quality is proportional to the size of circular electrodes on seat back, and inversely proportional to size of rectangular electrode on seat belt.

Unobtrusive Bed Monitor State of the Art

On average, people spend more than a quarter of their day in bed. If physiological information could be collected automatically while we sleep, it would be effective not only for health management but also for disease prevention. Unobtrusive bed monitoring devices have been developed over the past 30 years or so to detect physiological information without awareness, and this method attracted attention again in the 2020s, with the proliferation of deep learning, AI, and IoT. This section describes the current state of the art.

Drunk Driver Detection Using Multiple Non-Invasive Biosignals

This study aims to decrease the number of drunk drivers, a significant social problem. Traditional methods to measure alcohol intake include blood alcohol concentration (BAC) and breath alcohol concentration (BrAC) tests. While BAC testing requires blood samples and is impractical, BrAC testing is commonly used in drunk driving enforcement. In this study, the multiple biological signals of electrocardiogram (ECG), photoplethysmogram (PPG), and electrodermal activity (EDA) were collected non-invasively and with minimal driver restraint in a driving simulator. Data were collected from 10 participants for approximately 10 min at BrAC levels of 0.00%, 0.03%, and 0.08%, which align with the latest Korean drunk driving standards. The collected data underwent frequency filtering and were segmented into 30 s intervals with a 10 s overlap to extract heart rate variability (HRV) and pulse arrival time (PAT). Using more than 10 machine learning algorithms, the classification accuracy reached 88%. The results indicate that it is possible to classify a driver's level of intoxication using only non-invasive biological signals within a short period of about 30 s, potentially aiding in the prevention of drunk driving.

A Quantitative Method to Guide the Integration of Textile Inductive Electrodes in Automotive Applications for Respiratory Monitoring

Induction-based breathing sensors in automobiles enable unobtrusive respiratory rate monitoring as an indicator of a driver's alertness and health. This paper introduces a quantitative method based on signal quality to guide the integration of textile inductive electrodes in automotive applications. A case study with a simplified setup illustrated the ability of the method to successfully provide basic design rules about where and how to integrate the electrodes on seat belts and seat backs to gather good quality respiratory signals in an automobile. The best signals came from the subject's waist, then from the chest, then from the upper back, and finally from the lower back. Furthermore, folding the electrodes before their integration on a seat back improves the signal quality for both the upper and lower back. This analysis provided guidelines with three design rules to increase the chance of acquiring good quality signals: (1) use a multi-electrode acquisition approach, (2) place the electrodes in locations that maximize breathing-induced body displacement, and (3) use a mechanical amplifying method such as folding the electrodes in locations with little potential for breathing-induced displacement.

On-Road Evaluation of Unobtrusive In-Car Respiration Monitoring

This paper introduces and evaluates an innovative sensor for unobtrusive in-car respiration monitoring, mounted on the backrest of the driver's seat. The sensor seamlessly integrates into the vehicle, measuring breathing rates continuously without requiring active participation from the driver. The paper proves the feasibility of unobtrusive in-car measurements over long periods of time. Operation of the sensor was investigated over 12 participants sitting in the driver seat. A total of 107 min of driving in diverse conditions with overall coverage rate of 84.45% underscores the sensor potential to reliably capture physiological changes in breathing rate for fatigue and stress detection.

Detection of driver drowsiness level using a hybrid learning model based on ECG signals

OBJECTIVES

Fatigue has a considerable impact on the driver's vehicle and even the driver's own operating ability.

METHODS

An intelligent algorithm is proposed for the problem that it is difficult to classify the degree of drowsiness generated by the driver during the driving process. By studying the driver's electrocardiogram (ECG) during driving, two models were established to jointly classify the ECG signals as awake, stress, and fatigue or drowsiness states for drowsiness levels. Firstly, the deep learning method was used to establish the model_1 to predict the drowsiness of the original ECG, and model_2 was developed using the combination of principal component analysis (PCA) and weighted K-nearest neighbor (WKNN) algorithm to classify the heart rate variability characteristics. Then, the drowsiness prediction results of the two models were weighted according to certain rules, and the hybrid learning model combining dilated convolution and bidirectional long short-term memory network with PCA and WKNN algorithm was established, and the mixed model was denoted as DiCNN-BiLSTM and PCA-WKNN (DBPW). Finally, the validity of the DBPW model was verified by simulation of the public database.

RESULTS

The experimental results show that the average accuracy, sensitivity and F1 score of the test model in the dataset containing multiple drivers are 98.79, 98.81, and 98.79 % respectively, and the recognition accuracy for drowsiness or drowsiness state is 99.33 %.

CONCLUSIONS

Using the proposed algorithm, it is possible to identify driver anomalies and provide new ideas for the development of intelligent vehicles.

cbPPGGAN: A Generic Enhancement Framework for Unpaired Pulse Waveforms in Camera-Based Photoplethysmography

Camera-based photoplethysmography (cbP PG) is a non-contact technique that measures cardiac-related blood volume alterations in skin surface vessels through the analysis of facial videos. While traditional approaches can estimate heart rate (HR) under different illuminations, their accuracy can be affected by motion artifacts, leading to poor waveform fidelity and hindering further analysis of heart rate variability (HRV); deep learning-based approaches reconstruct high-quality pulse waveform, yet their performance significantly degrades under illumination variations. In this work, we aim to leverage the strength of these two methods and propose a framework that possesses favorable generalization capabilities while maintaining waveform fidelity. For this purpose, we propose the cbPPGGAN, an enhancement framework for cbPPG that enables the flexible incorporation of both unpaired and paired data sources in the training process. Based on the waveforms extracted by traditional approaches, the cbPPGGAN reconstructs high-quality waveforms that enable accurate HR estimation and HRV analysis. In addition, to address the lack of paired training data problems in real-world applications, we propose a cycle consistency loss that guarantees the time-frequency consistency before/after mapping. The method enhances the waveform quality of traditional POS approaches in different illumination tests (BH-rPPG) and cross-datasets (UBFC-rPPG) with mean absolute error (MAE) values of 1.34 bpm and 1.65 bpm, and average beat-to-beat (AVBB) values of 27.46 ms and 45.28 ms, respectively. Experimental results demonstrate that the cbPPGGAN enhances cbPPG signal quality and outperforms the state-of-the-art approaches in HR estimation and HRV analysis. The proposed framework opens a new pathway toward accurate HR estimation in an unconstrained environment.

Integrating physiological monitoring systems in military aviation: a brief narrative review of its importance, opportunities, and risks

Military pilots risk their lives during training and operations. Advancements in aerospace engineering, flight profiles, and mission demands may require the pilot to test the safe limits of their physiology. Monitoring pilot physiology (e.g. heart rate, oximetry, and respiration) inflight is in consideration by several nations to inform pilots of reduced performance capacity and guide future developments in aircraft and life-support system design. Numerous challenges, however, prevent the immediate operationalisation of physiological monitoring sensors, particularly their unreliability in the aerospace environment and incompatibility with pilot clothing and protective equipment. Human performance and behaviour are also highly variable and measuring these in controlled laboratory settings do not mirror the real-world conditions pilots must endure. Misleading or erroneous predictive models are unacceptable as these could compromise mission success and lose operator trust. This narrative review provides an overview of considerations for integrating physiological monitoring systems within the military aviation environment.Practitioner summary: Advancements in military technology can conflictingly enhance and compromise pilot safety and performance. We summarise some of the opportunities, limitations, and risks of integrating physiological monitoring systems within military aviation. Our intent is to catalyse further research and technological development.Abbreviations: AGS: anti-gravity suit; AGSM: anti-gravity straining manoeuvre; A-LOC: almost loss of consciousness; CBF: cerebral blood flow; ECG: electrocardiogram; EEG: electroencephalogram; fNIRS: functional near-infrared spectroscopy; G-forces: gravitational forces; G-LOC: gravity-induced loss of consciousness; HR: heart rate; HRV: heart rate variability; LSS: life-support system; NATO: North Atlantic Treaty Organisation; PE: Physiological Episode; PCO2: partial pressure of carbon dioxide; PO2: partial pressure of oxygen; OBOGS: on board oxygen generating systems; SpO2: peripheral blood haemoglobin-oxygen saturation; STANAG: North Atlantic Treaty Organisation Standardisation Agreement; UPE: Unexplained Physiological Episode; WBV: whole body vibration.

In-vehicle Sensing and Data Analysis for Older Drivers with Mild Cognitive Impairment

Driving is a complex daily activity indicating age and disease-related cognitive declines. Therefore, deficits in driving performance compared with ones without mild cognitive impairment (MCI) can reflect changes in cognitive functioning. There is increasing evidence that unobtrusive monitoring of older adults' driving performance in a daily-life setting may allow us to detect subtle early changes in cognition. The objectives of this paper include designing low-cost in-vehicle sensing hardware capable of obtaining high-precision positioning and telematics data, identifying important indicators for early changes in cognition, and detecting early-warning signs of cognitive impairment in a truly normal, day-to-day driving condition with machine learning approaches. Our statistical analysis comparing drivers with MCI to those without reveals that those with MCI exhibit smoother and safer driving patterns. This suggests that drivers with MCI are cognizant of their condition and tend to avoid erratic driving behaviors. Furthermore, our Random Forest models identified the number of night trips, number of trips, and education as the most influential factors in our data evaluation.

Robust in-vehicle heartbeat detection using multimodal signal fusion

A medical check-up during driving enables the early detection of diseases. Heartbeat irregularities indicate possible cardiovascular diseases, which can be determined with continuous health monitoring. Therefore, we develop a redundant sensor system based on electrocardiography (ECG) and photoplethysmography (PPG) sensors attached to the steering wheel, a red, green, and blue (RGB) camera behind the steering wheel. For the video, we integrate the face recognition engine SeetaFace to detect landmarks of face segments continuously. Based on the green channel, we derive colour changes and, subsequently, the heartbeat. We record the ECG, PPG, video, and reference ECG with body electrodes of 19 volunteers during different driving scenarios, each lasting 15 min: city, highway, and countryside. We combine early, signal-based late, and sensor-based late fusion with a hybrid convolutional neural network (CNN) and integrated majority voting to deliver the final heartbeats that we compare to the reference ECG. Based on the measured and the reference heartbeat positions, the usable time was 51.75%, 58.62%, and 55.96% for the driving scenarios city, highway, and countryside, respectively, with the hybrid algorithm and combination of ECG and PPG. In conclusion, the findings suggest that approximately half the driving time can be utilised for in-vehicle heartbeat monitoring.

Robust in-vehicle respiratory rate detection using multimodal signal fusion

Continuous health monitoring in private spaces such as the car is not yet fully exploited to detect diseases in an early stage. Therefore, we develop a redundant health monitoring sensor system and signal fusion approaches to determine the respiratory rate during driving. To recognise the breathing movements, we use a piezoelectric sensor, two accelerometers attached to the seat and the seat belt, and a camera behind the windscreen. We record data from 15 subjects during three driving scenarios (15 min each) city, highway, and countryside. An additional chest belt provides the ground truth. We compare the four convolutional neural network (CNN)-based fusion approaches: early, sensor-based late, signal-based late, and hybrid fusion. We evaluate the performance of fusing for all four signals to determine the portion of driving time and the signal combination. The hybrid algorithm fusing all four signals is most effective in detecting respiratory rates in the city ([Formula: see text]), highway ([Formula: see text]), and countryside ([Formula: see text]). In summary, 60% of the total driving time can be used to measure the respiratory rate. The number of signals used in the multi-signal fusion improves reliability and enables continuous health monitoring in a driving vehicle.

Sensor Fusion of Cardiorespiratory Signals Using an Adaptive Kalman Filter. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38082963 DOI: 10.1109/embc40787.2023.10340942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

For unobtrusive monitoring of vital signs, redundant sensors are beneficial to fuse several sensor measurements which can improve the estimation of, e.g. heart rate and respiratory rate. In this paper, an adaptive unscented Kalman filter is used to estimate respiratory rate and heart rate on a new simplified model for cardiorespiratory coupling. Additionally, the Kalman filter is tuned to incorporate the non-white system noise of the model. The Kalman filter is tested on synthesised data with variations regarding SNR, model mismatch and amount of sensors. For respiratory rate, a median squared error of as low as 0.02BPM2 and, for heart rate, a median squared error of as low as 0.2BPM2 for ideal assumptions is achieved.

A Portable Multi-Modal Cushion for Continuous Monitoring of a Driver's Vital Signs

With higher levels of automation in vehicles, the need for robust driver monitoring systems increases, since it must be ensured that the driver can intervene at any moment. Drowsiness, stress and alcohol are still the main sources of driver distraction. However, physiological problems such as heart attacks and strokes also exhibit a significant risk for driver safety, especially with respect to the ageing population. In this paper, a portable cushion with four sensor units with multiple measurement modalities is presented. Capacitive electrocardiography, reflective photophlethysmography, magnetic induction measurement and seismocardiography are performed with the embedded sensors. The device can monitor the heart and respiratory rates of a vehicle driver. The promising results of the first proof-of-concept study with twenty participants in a driving simulator not only demonstrate the accuracy of the heart (above 70% of medical-grade heart rate estimations according to IEC 60601-2-27) and respiratory rate measurements (around 30% with errors below 2 BPM), but also that the cushion might be useful to monitor morphological changes in the capacitive electrocardiogram in some cases. The measurements can potentially be used to detect drowsiness and stress and thus the fitness of the driver, since heart rate variability and breathing rate variability can be captured. They are also useful for the early prediction of cardiovascular diseases, one of the main reasons for premature death. The data are publicly available in the UnoVis dataset.

Investigating Cardiorespiratory Interaction Using Ballistocardiography and Seismocardiography-A Narrative Review

Ballistocardiography (BCG) and seismocardiography (SCG) are non-invasive techniques used to record the micromovements induced by cardiovascular activity at the body's center of mass and on the chest, respectively. Since their inception, their potential for evaluating cardiovascular health has been studied. However, both BCG and SCG are impacted by respiration, leading to a periodic modulation of these signals. As a result, data processing algorithms have been developed to exclude the respiratory signals, or recording protocols have been designed to limit the respiratory bias. Reviewing the present status of the literature reveals an increasing interest in applying these techniques to extract respiratory information, as well as cardiac information. The possibility of simultaneous monitoring of respiratory and cardiovascular signals via BCG or SCG enables the monitoring of vital signs during activities that require considerable mental concentration, in extreme environments, or during sleep, where data acquisition must occur without introducing recording bias due to irritating monitoring equipment. This work aims to provide a theoretical and practical overview of cardiopulmonary interaction based on BCG and SCG signals. It covers the recent improvements in extracting respiratory signals, computing markers of the cardiorespiratory interaction with practical applications, and investigating sleep breathing disorders, as well as a comparison of different sensors used for these applications. According to the results of this review, recent studies have mainly concentrated on a few domains, especially sleep studies and heart rate variability computation. Even in those instances, the study population is not always large or diversified. Furthermore, BCG and SCG are prone to movement artifacts and are relatively subject dependent. However, the growing tendency toward artificial intelligence may help achieve a more accurate and efficient diagnosis. These encouraging results bring hope that, in the near future, such compact, lightweight BCG and SCG devices will offer a good proxy for the gold standard methods for assessing cardiorespiratory function, with the added benefit of being able to perform measurements in real-world situations, outside of the clinic, and thus decrease costs and time.

Detecting driver fatigue using heart rate variability: A systematic review

Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.

Smart Steering Sleeve (S3): A Non-Intrusive and Integrative Sensing Platform for Driver Physiological Monitoring

Driving is a ubiquitous activity that requires both motor skills and cognitive focus. These aspects become more problematic for some seniors, who have underlining medical conditions and tend to lose some of these capabilities. Therefore, driving can be used as a controlled environment for the frequent, non-intrusive monitoring of bio-physical and cognitive status within drivers. Such information can then be utilized for enhanced assistive vehicle controls and/or driver health monitoring. In this paper, we present a novel multi-modal smart steering sleeve (S3) system with an integrated sensing platform that can non-intrusively and continuously measure a driver's physiological signals, including electrodermal activity (EDA), electromyography (EMG), and hand pressure. The sensor suite was developed by combining low-cost interdigitated electrodes with a piezoresistive force sensor on a single, flexible polymer substrate. Comprehensive characterizations on the sensing modalities were performed with promising results demonstrated. The sweat-sensing unit (SSU) for EDA monitoring works under a 100 Hz alternative current (AC) source. The EMG signal acquired by the EMG-sensing unit (EMGSU) was amplified to within 5 V. The force-sensing unit (FSU) for hand pressure detection has a range of 25 N. This flexible sensor was mounted on an off-the-shelf steering wheel sleeve, making it an add-on system that can be installed on any existing vehicles for convenient and wide-coverage driver monitoring. A cloud-based communication scheme was developed for the ease of data collection and analysis. Sensing platform development, performance, and limitations, as well as other potential applications, are discussed in detail in this paper.

Developing a Multimodal HMI Design Framework for Automotive Wellness in Autonomous Vehicles

With the development of autonomous technology, the research into multimodal human-machine interaction (HMI) for autonomous vehicles (AVs) has attracted extensive attention, especially in automotive wellness. To support the design of HMIs for automotive wellness in AVs, this paper proposes a multimodal design framework. First, three elements of the framework were envisioned based on the typical composition of an interactive system. Second, a five-step process for utilizing the proposed framework was suggested. Third, the framework was applied in a design education course for exemplification. Finally, the AttrakDiff questionnaire was used to evaluate these interactive prototypes with 20 participants who had an affinity for HMI design. The questionnaire responses showed that the overall impression was positive and this framework can help design students to effectively identify research gaps and expand design concepts in a systematic way. The proposed framework offers a design approach for the development of multimodal HMIs for autonomous wellness in AVs.

Accuracy enhancement in reflective pulse oximetry by considering wavelength-dependent pathlengths

Objective. Noninvasive measurement of oxygen saturation (SpO₂) using pulse oximetry based on transmissive photoplethysmography (tPPG) is clinically accepted and widely employed. However, reflective photoplethysmography (rPPG) - present in smartwatches - has not become equally accepted, partially because the pathlengths of the red and infrared PPGs are patient-dependent. Thus, even the most popular "Ratio of Modulation" (R) method requires patient-dependent calibration to reduce the errors in the measurement of SpO2 using rPPGs.Approach. In this paper, a correction factor or "pathlength ratio" β is introduced in an existing calibration-free algorithm that compensates the patient-dependent pathlength variations, and improved accuracy is obtained in the measurement of SpO2 using rPPGs. The proposed β is derived through the analytical model of a rPPG signal. Using the new expression and data obtained from a human hypoxia study wherein arterial oxygen saturation values acquired through Blood Gas Analysis were employed as a reference, β is determined.Main results. The results of the analysis show that a specific combination of the β and the measurements on the pulsating part of the natural logarithm of the red and infrared PPG signals yields a reduced root-mean-square error (RMSE). It is shown that the average RMSE in measuring SpO2 values reduces to 1 %.Significance. The human hypoxia study data used for this work, obtained in a previous study, coversSpO₂values in the range from 70 % to 100 %, and thus shows that the pathlength ratio β proposed here works well in the range of clinical interest. This work demonstrates that the calibration-free method applicable for transmission type PPGs can be extended to determineSpO₂using reflective PPGs with the incorporation of the correction factor β. Our algorithm significantly reduces the number of parameters needed for the estimation, while keeping the RMSE below the clinically accepted 2 %.

Correlation Mapping of Perfusion Patterns in Cutaneous Tissue

Perfusion patterns of cutaneous tissue represent a valuable source of information about the state of the patient’s cardiovascular system and autonomic nervous system (ANS). This concept aims to observe the perfusion changes in the foot sole in two healthy individuals and two subjects affected by diabetes mellitus (DM). We use photoplethysmography imaging (PPGI) to monitor cutaneous perfusion changes. This method, in contrast to conventional contact photoplethysmography (PPG), allows the monitoring of skin perfusion with spatial distribution. We use a machine vision camera and an illumination system using the green light. To induce the perfusion changes, we perform an experiment in the form of a deep breathing test (DBT). The experiment consists of three stages, with the middle stage being the DBT. To evaluate spatial perfusion changes, we use a normalized measure of the correlation of PPGI signals with a reference PPG signal obtained from the foot’s little toe. This method also increases the signal-to-noise ratio (SNR). Subjects with DM shows different patterns of tissue perfusion changes compared to healthy subjects. The DM subjects show increased perfusion after DBT compared to the pre-DBT state, whereas in healthy subjects, the tissue perfusion does not reach the level of the pre-DBT phase. This work can be considered as proof of concept in developing a non-contact and non-intrusive monitoring system that allows a different view of microcirculatory damage in patients with diabetes mellitus, focusing on its spatial distribution.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

What is new in respiratory monitoring?

This paper provides a review of a selection of papers published in the Journal of Clinical Monitoring and Computing in 2020 and 2021 highlighting what is new within the field of respiratory monitoring. Selected papers cover work in pulse oximetry monitoring, acoustic monitoring, respiratory system mechanics, monitoring during surgery, electrical impedance tomography, respiratory rate monitoring, lung ultrasound and detection of patient-ventilator asynchrony.

Design of effective self-powered SnS2/halide perovskite photo-detection system based on triboelectric nanogenerator by regarding circuit impedance

Self-powered detectors based on triboelectric nanogenerators (TENG) have been considered because of their capability to convert ambient mechanical energy to electrical out-put signal, instead of conventional usage of electrochemical batteries as power sources. In this regard, the self-powered photodetectors have been designed through totally two lay out called passive and active circuit. in former model, impedance matching between the TENG and the resistance of the circuit's elements is crucial, which is not investigated systematically till now. In this paper, a cost effective novel planar photodetector (PD) based on heterojunction of SnS2 sheets and Cs0.05(FA0.83 MA0.17)0.95Pb(I0.83Br0.17)3 three cationic lead iodide based perovskite (PVK) layer fabricated which powered by graphene oxide (GO) paper and Kapton based contact-separated TENG (CS-TENG). To achieve the high performance of this device, the proper range of the load resistances in the circuit regards to TENG's characterization has been studied. In the next steps, the integrated self-powered photo-detection system was designed by applying Kapton/FTO and hand/FTO TENG, separately, in the proposed impedance matching circuit. The calculated D* of integrated self-powered SnS2/PVK supplied by tapping the Kapton and hand on FTO is 2.83 × 1010 and 1.10 × 1013 Jones under the 10 mW/cm2 of white light intensity, the investigations determine that for designing significate performance of self-powered PD supplied by TENG, the existence of the load resistance with the well match amount to the utilized TENG is crucial. Our results which can be generalized to other types of passive self-powered sensors, are substantial to both academia and industry concepts.

Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera

We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS.

Detection of acute ventilatory problems via magnetic induction in a newborn animal model

BACKGROUND

Magnetic induction measurement (MIM) is a noninvasive method for the contactless registration of respiration in newborn piglets by using measurement coils positioned at the bottom of an incubator. Acute pulmonary problems may be determinants of poor neurological and psychomotor outcomes in preterm infants. The current study tested the detection of pulmonary ventilation disorders via MIM in 11 newborn piglets.

METHODS

Six measurement coils determined changes in magnetic induction, depending on the ventilation of the lung, in comparison with flow resistance. Contactless registration of induced acute pulmonary ventilation disorders (apnea, atelectasis, pneumothorax, and aspiration) was detected by MIM.

RESULTS

All pathologies except aspiration were detected by MIM. Significant changes occurred after induction of apnea (three coils), malposition of the tube (one coil), and pneumothorax (three coils) (p ≤ 0.05). No significant changes occurred after induction of aspiration (p = 0.12).

CONCLUSIONS

MIM seems to have some potential to detect acute ventilation disorders in newborn piglets. The location of the measurement coil related to the animal's position plays a critical role in this process. In addition to an early detection of acute pulmonary problems, potential information pointing to a therapeutic intervention, for example, inhalations or medical respiratory analepsis, may be conceivable with MIM in the future.

IMPACT

MIM seems to be a method in which noncontact ventilation disorders of premature and mature infants can be detected. This study is an extension of the experimental setup to obtain preliminary evidence for detection of respiratory activity in neonatal piglets. For the first time, MIM is used to register acute ventilation problems of neonates. The possibility of an early detection of acute ventilation problems via MIM may provide an opportunity to receive patient-side information for therapeutical interventions like inhalations or medical respiratory analepsis.

Driver Cardiovascular Disease Detection Using Seismocardiogram

This article deals with the treatment and application of cardiac biosignals, an excited accelerometer, and a gyroscope in the prevention of accidents on the road. Previously conducted studies say that the seismocardiogram is a measure of cardiac microvibration signals that allows for detecting rhythms, heart valve opening and closing disorders, and monitoring of patients' breathing. This article refers to the seismocardiogram hypothesis that the measurements of a seismocardiogram could be used to identify drivers' heart problems before they reach a critical condition and safely stop the vehicle by informing the relevant departments in a nonclinical manner. The proposed system works without an electrocardiogram, which helps to detect heart rhythms more easily. The estimation of the heart rate (HR) is calculated through automatically detected aortic valve opening (AO) peaks. The system is composed of two micro-electromechanical systems (MEMSs) to evaluate physiological parameters and eliminate the effects of external interference on the entire system. The few digital filtering methods are discussed and benchmarked to increase seismocardiogram efficiency. As a result, the fourth adaptive filter obtains the estimated HR = 65 beats per min (bmp) in a still noisy signal (SNR = −11.32 dB). In contrast with the low processing benefit (3.39 dB), 27 AO peaks were detected with a 917.56-ms peak interval mean over 1.11 s, and the calculated root mean square error (RMSE) was 0.1942 m/s2 when the adaptive filter order is 50 and the adaptation step is equal to 0.933.

Validation Study of a Contactless Monitoring Device for Vital Signs During Sleep and Sleep Architecture in Adults With Sleep-Disordered Breathing

Background and Objective Few clinical studies have investigated the accuracy of non-contact monitoring devices for vital signs during sleep and sleep architecture in adults with sleep-disordered breathing (SDB). The purpose of this study was to assess the accuracy of a contactless monitoring device for 1) heart rate, respiratory rate, and body temperature during sleep and 2) sleep architecture in adults with SDB.Methods Thirty-five consecutive adults, who visited a tertiary university hospital due to suspected SDB, underwent a complete physical examination and standard (level 1) polysomnography plus body temperature measurement with a contactless monitoring device (HoneyCube System).Results A total of 30 subjects (mean age = 46.43 ± 12.9 years; male: female = 22: 8) were finally included, and five subjects were excluded due to inadequate data in this study. The intraclass correlation coefficient values of heart rate, respiratory rate, and body temperature measured using the contactless monitoring device were 0.91 (95% confidence interval [CI]: 0.892, 0.928), 0.937 (95% CI: 0.919, 0.954), and 0.918 (95% CI: 0.895, 0.941), respectively. The mean kappa value for sleep architecture was 0.562 (95% CI: 0.529, 0.596).Conclusions The contactless monitoring device showed good (almost perfect) agreement in terms of heart rate, respiratory rate, and body temperature and moderate agreement in sleep architecture with contact measurements. These results suggest that the HoneyCube System is a good candidate device for sleep monitoring at home and in multiple accommodations.

Sensor Fusion for Robust Heartbeat Detection during Driving

Private spaces like apartments and vehicles are not yet fully exploited for health monitoring, which includes continuous measurement of biosignals. This work proposes sensor fusion for robust heartbeat detection in the noisy and dynamic driving environment. We use four sensors: electrocardiography (ECG), ballistocardiography (BCG), photoplethysmography (PPG), and image-based PPG (iPPG). As ground truth, we record a 3-lead ECG with wet electrodes attached to the chest. Twelve healthy volunteers are monitored in rest and during driving, each for 11 min. We propose sensor fusion using convolutional neural networks to detect the sensor combination delivering the most accurate heart rate measurement. For rest, we achieve scores of 95.16% (BCG + iPPG), 96.08% (ECG + iPPG), 96.35% (ECG + BCG), 96.53% (ECG + PPG), 96.58% (PPG + iPPG), and 97.15% (BCG + PPG). In motion, the highest scores are 92.46% (BCG + iPPG, PPG + iPPG, ECG + iPPG), 92.83% (ECG + PPG), 93.03% (BCG + PPG), and 93.08% (ECG + BCG). Fusing all four signals with the best fusion approach results in scores of 97.24% (rest) and 94.38% (motion). We conclude that sensor fusion allows robust heartbeat measurement of car drivers to support continuous and unobtrusive health monitoring for early disease detection.

Modified Camera Setups for Day-and-Night Pulse-rate Monitoring

Camera systems have been studied as a means for ubiquitous remote photoplethysmography. It was first considered for daytime applications using ambient light. However, main applications for continuous monitoring are in dark or low-light conditions (e.g. sleep monitoring) and, more recently, suitable light sources and simple camera adaptations have been considered for infrared-based solutions. This paper explores suitable camera configurations for pulse-rate monitoring during both day and night (24/7). Various configurations differing in the recorded spectral range are defined, i.e. straight-forward adaptations of a standard RGB camera by choosing proper optical filters. These systems have been studied in a benchmark involving day and night monitoring with various degrees of motion disturbances. The results indicate that, for the 24/7 monitoring, it is best to deploy the full spectral band of an RGB camera, and this can be done without compromising the monitoring performance at night.

Using in-seat electrical potential sensors for non-contact monitoring of heart rate, heart rate variability, and heart rate recovery

Detecting transient changes in heart rate and heart rate variability during experimental simulated autonomous driving scenarios can indicate participant arousal and cognitive load, providing valuable insights into the relationship between human and vehicle autonomy. Successfully detecting such parameters unobtrusively may assist these experimental situations as well as naturalistic driver monitoring systems within an autonomous vehicle. However, non-contact sensors must collect reliable and accurate signals. This study aims to compare the in-seat, non-contact Plessey EPIC sensor to the gold standard, contact Biopac sensor. Thirty participants took part in five-minute simulated autonomous vehicle journeys in a city environment and a rural environment, and a five-minute resting condition. To ensure the seat sensor was sensitive to elevated heart rate values, heart rate was also collected following the energetic Harvard Step Test. Lin concordance coefficients and Bland-Altman analyses were employed to assess the level of agreement between the non-contact Plessey EPIC sensor and the contact Biopac sensor for heart rate and heart rate variability. Analyses revealed a high level of agreement (r_c > 0.96) between both sensors for one-minute averaged heart rate and five-minute averaged heart rate variability during simulated autonomous driving and rest, and one-minute averaged heart rate following the Harvard Step Test. In addition, the non-contact sensor was sensitive to significant differences during tasks. This proof of principle study demonstrates the feasibility of using the non-contact Plessey EPIC sensor to accurately detect heart rate and heart rate variability during simulated autonomous driving environments.

Survey and Synthesis of State of the Art in Driver Monitoring

Road vehicle accidents are mostly due to human errors, and many such accidents could be avoided by continuously monitoring the driver. Driver monitoring (DM) is a topic of growing interest in the automotive industry, and it will remain relevant for all vehicles that are not fully autonomous, and thus for decades for the average vehicle owner. The present paper focuses on the first step of DM, which consists of characterizing the state of the driver. Since DM will be increasingly linked to driving automation (DA), this paper presents a clear view of the role of DM at each of the six SAE levels of DA. This paper surveys the state of the art of DM, and then synthesizes it, providing a unique, structured, polychotomous view of the many characterization techniques of DM. Informed by the survey, the paper characterizes the driver state along the five main dimensions—called here “(sub)states”—of drowsiness, mental workload, distraction, emotions, and under the influence. The polychotomous view of DM is presented through a pair of interlocked tables that relate these states to their indicators (e.g., the eye-blink rate) and the sensors that can access each of these indicators (e.g., a camera). The tables factor in not only the effects linked directly to the driver, but also those linked to the (driven) vehicle and the (driving) environment. They show, at a glance, to concerned researchers, equipment providers, and vehicle manufacturers (1) most of the options they have to implement various forms of advanced DM systems, and (2) fruitful areas for further research and innovation.

Design of Smart Steering Wheel for Unobtrusive Health and Drowsiness Monitoring

This article describes the design of a smart steering wheel intended for use in unobtrusive health and drowsiness monitoring. The aging population, cardiovascular disease, personalized medicine, and driver fatigue were significant motivations for developing a monitoring platform in cars because people spent much time in cars. The purpose was to create a unique, comprehensive monitoring system for the driver. The crucial parameters in health or drowsiness monitoring, such as heart rate, heart rate variability, and blood oxygenation, are measured by an electrocardiograph and oximeter integrated into the steering wheel. In addition, an inertial unit was integrated into the steering wheel to record and analyze the movement patterns performed by the driver while driving. The developed steering wheel was tested under laboratory and real-life conditions. The measured signals were verified by commercial devices to confirm data correctness and accuracy. The resulting signals show the applicability of the developed platform in further detecting specific cardiovascular diseases (especially atrial fibrillation) and drowsiness.

Automatic Information Exchange in the Early Rescue Chain Using the International Standard Accident Number (ISAN)

Thus far, emergency calls are answered by human operators who interview the calling person in order to obtain all relevant information. In the near future-based on the Internet of (Medical) Things (IoT, IoMT)-accidents, emergencies, or adverse health events will be reported automatically by smart homes, smart vehicles, or smart wearables, without any human in the loop. Several parties are involved in this communication: the alerting system, the rescue service (responding system), and the emergency department in the hospital (curing system). In many countries, these parties use isolated information and communication technology (ICT) systems. Previously, the International Standard Accident Number (ISAN) has been proposed to securely link the data in these systems. In this work, we propose an ISAN-based communication platform that allows semantically interoperable information exchange. Our aims are threefold: (i) to enable data exchange between the isolated systems, (ii) to avoid data misinterpretation, and (iii) to integrate additional data sources. The suggested platform is composed of an alerting, responding, and curing system manager, a workflow manager, and a communication manager. First, the ICT systems of all parties in the early rescue chain register with their according system manager, which tracks the keep-alive. In case of emergency, the alerting system sends an ISAN to the platform. The responsible rescue services and hospitals are determined and interconnected for platform-based communication. Next to the conceptual design of the platform, we evaluate a proof-of-concept implementation according to (1) the registration, (2) channel establishment, (3) data encryption, (4) event alert, and (5) information exchange. Our concept meets the requirements for scalability, error handling, and information security. In the future, it will be used to implement a virtual accident registry.

Non-Contact Respiratory Monitoring Using an RGB Camera for Real-World Applications

Respiratory monitoring is receiving growing interest in different fields of use, ranging from healthcare to occupational settings. Only recently, non-contact measuring systems have been developed to measure the respiratory rate (fR) over time, even in unconstrained environments. Promising methods rely on the analysis of video-frames features recorded from cameras. In this work, a low-cost and unobtrusive measuring system for respiratory pattern monitoring based on the analysis of RGB images recorded from a consumer-grade camera is proposed. The system allows (i) the automatized tracking of the chest movements caused by breathing, (ii) the extraction of the breathing signal from images with methods based on optical flow (FO) and RGB analysis, (iii) the elimination of breathing-unrelated events from the signal, (iv) the identification of possible apneas and, (v) the calculation of fR value every second. Unlike most of the work in the literature, the performances of the system have been tested in an unstructured environment considering user-camera distance and user posture as influencing factors. A total of 24 healthy volunteers were enrolled for the validation tests. Better performances were obtained when the users were in sitting position. FO method outperforms in all conditions. In the fR range 6 to 60 breaths/min (bpm), the FO allows measuring fR values with bias of −0.03 ± 1.38 bpm and −0.02 ± 1.92 bpm when compared to a reference wearable system with the user at 2 and 0.5 m from the camera, respectively.

Transcranial Auricular Vagus Nerve Stimulation (taVNS) and Ear-EEG: Potential for Closed-Loop Portable Non-invasive Brain Stimulation

No matter how hard we concentrate, our attention fluctuates – a fact that greatly affects our success in completing a current task. Here, we review work from two methods that, in a closed-loop manner, have the potential to ameliorate these fluctuations. Ear-EEG can measure electric brain activity from areas in or around the ear, using small and thus portable hardware. It has been shown to capture the state of attention with high temporal resolution. Transcutaneous auricular vagus nerve stimulation (taVNS) comes with the same advantages (small and light) and critically current research suggests that it is possible to influence ongoing brain activity that has been linked to attention. Following the review of current work on ear-EEG and taVNS we suggest that a combination of the two methods in a closed-loop system could serve as a potential application to modulate attention.

Development of Nanosensors Based Intelligent Packaging Systems: Food Quality and Medicine

The issue of medication noncompliance has resulted in major risks to public safety and financial loss. The new omnipresent medicine enabled by the Internet of things offers fascinating new possibilities. Additionally, an in-home healthcare station (IHHS), it is necessary to meet the rapidly increasing need for routine nursing and on-site diagnosis and prognosis. This article proposes a universal and preventive strategy to drug management based on intelligent and interactive packaging (I2Pack) and IMedBox. The controlled delamination material (CDM) seals and regulates wireless technologies in novel medicine packaging. As such, wearable biomedical sensors may capture a variety of crucial parameters via wireless communication. On-site treatment and prediction of these critical factors are made possible by high-performance architecture. The user interface is also highlighted to make surgery easier for the elderly, disabled, and patients. Land testing incorporates and validates an approach for prototyping I2Pack and iMedBox. Additionally, sustainability, increased product safety, and quality standards are crucial throughout the life sciences. To achieve these standards, intelligent packaging is also used in the food and pharmaceutical industries. These technologies will continuously monitor the quality of a product and communicate with the user. Data carriers, indications, and sensors are the three most important groups. They are not widely used at the moment, although their potential is well understood. Intelligent packaging should be used in these sectors and the functionality of the systems and the values presented in this analysis.

Detection of Ventricular Fibrillation Based on Ballistocardiography by Constructing an Effective Feature Set

Ventricular fibrillation (VF) is a type of fatal arrhythmia that can cause sudden death within minutes. The study of a VF detection algorithm has important clinical significance. This study aimed to develop an algorithm for the automatic detection of VF based on the acquisition of cardiac mechanical activity-related signals, namely ballistocardiography (BCG), by non-contact sensors. BCG signals, including VF, sinus rhythm, and motion artifacts, were collected through electric defibrillation experiments in pigs. Through autocorrelation and S transform, the time-frequency graph with obvious information of cardiac rhythmic activity was obtained, and a feature set of 13 elements was constructed for each 7 s segment after statistical analysis and hierarchical clustering. Then, the random forest classifier was used to classify VF and non-VF, and two paradigms of intra-patient and inter-patient were used to evaluate the performance. The results showed that the sensitivity and specificity were 0.965 and 0.958 under 10-fold cross-validation, and they were 0.947 and 0.946 under leave-one-subject-out cross-validation. In conclusion, the proposed algorithm combining feature extraction and machine learning can effectively detect VF in BCG, laying a foundation for the development of long-term self-cardiac monitoring at home and a VF real-time detection and alarm system.

A Heart Rate Monitoring Framework for Real-World Drivers Using Remote Photoplethysmography

Remote photoplethysmography (rPPG) is an unobtrusive solution to heart rate monitoring in drivers. However, disturbances that occur during driving such as driver behavior, motion artifacts, and illuminance variation complicate the monitoring of heart rate. Faced with disturbance, one commonly used assumption is heart rate periodicity (or spectrum sparsity). Several methods improve stability at the expense of tracking sensitivity for heart rate variation. Based on statistical signal processing (SSP) and Monte Carlo simulations, the outlier probability is derived and ADaptive spectral filter banks (AD) is proposed as a new algorithm which provides an explicable tuning option for spectral filter banks to strike a balance between robustness and sensitivity in remote monitoring for driving scenarios. Moreover, we construct a driving database containing over 23 hours of data to verify the proposed algorithm. The influence on rPPG from driver habits (both amateurs and professionals), vehicle types (compact cars and buses), and routes are also evaluated. In comparison to state-of-the-art rPPG for driving scenarios, the mean absolute error in the Passengers, Compact Cars, and Buses scenarios is 3.43, 7.85, and 5.02 beats per minute, respectively. Moreover, AD also won the top third place in the first challenge on remote physiological signal sensing (RePSS) with relative low computational complexity.

Photoplethysmography in Normal and Pathological Sleep

This article presents an overview of the advancements that have been made in the use of photoplethysmography (PPG) for unobtrusive sleep studies. PPG is included in the quickly evolving and very popular landscape of wearables but has specific interesting properties, particularly the ability to capture the modulation of the autonomic nervous system during sleep. Recent advances have been made in PPG signal acquisition and processing, including coupling it with accelerometry in order to construct hypnograms in normal and pathologic sleep and also to detect sleep-disordered breathing (SDB). The limitations of PPG (e.g., oxymetry signal failure, motion artefacts, signal processing) are reviewed as well as technical solutions to overcome these issues. The potential medical applications of PPG are numerous, including home-based detection of SDB (for triage purposes), and long-term monitoring of insomnia, circadian rhythm sleep disorders (to assess treatment effects), and treated SDB (to ensure disease control). New contact sensor combinations to improve future wearables seem promising, particularly tools that allow for the assessment of brain activity. In this way, in-ear EEG combined with PPG and actigraphy could be an interesting focus for future research.

Spatio-temporal and -spectral feature maps in photoplethysmography imaging and infrared thermograph

BACKGROUND

Only a small fraction of the information available is generally used in the majority of camera-based sensing approaches for vital sign monitoring. Dedicated skin pixels, for example, fall into this category while other regions are often disregarded early in the processing chain.

METHODS

We look at a simple processing chain for imaging where a video stream is converted to several other streams to investigate whether other image regions should also be considered. These streams are generated by mapping spatio-temporal and -spectral features of video segments and, thus, compressing the information contained in several seconds of video and encoding these in a new image. Two typical scenarios are provided as examples to study the applicability of these maps: face videos in a laboratory setting and measurements of a baby in the neonatal intensive care unit. Each measurement consists of the synchronous recording of photoplethysmography imaging (PPGI) and infrared thermography (IRT). We report the results of a visual inspection of those maps, evaluate the root mean square (RMS) contrast of foreground and background regions, and use histogram intersections as a tool for similarity measurements.

RESULTS

The maps allow us to distinguish visually between pulsatile foreground objects and an image background, which is found to be a noisy pattern. Distortions in the maps could be localized and the origin could be discovered. The IRT highlights subject contours for the heart frequency band, while silhouettes show strong signals in PPGI. Reflections and shadows were found to be sources of signals and distortions. We can testify advantages for the use of near-infrared light for PPGI. Furthermore, a difference in RMS contrast for pulsatile and non-pulsatile regions could be demonstrated. Histogram intersections allowed us to differentiate between the background and foreground.

CONCLUSIONS

We introduced new maps for the two sensing modalities and presented an overview for three different wavelength ranges. The maps can be used as a tool for visualizing aspects of the dynamic information hidden in video streams without automation. We propose focusing on an indirect method to detect pulsatile regions by using the noisy background pattern characteristic, for example, based on the histogram approach introduced.

The Importance of Respiratory Rate Monitoring: From Healthcare to Sport and Exercise

Respiratory rate is a fundamental vital sign that is sensitive to different pathological conditions (e.g., adverse cardiac events, pneumonia, and clinical deterioration) and stressors, including emotional stress, cognitive load, heat, cold, physical effort, and exercise-induced fatigue. The sensitivity of respiratory rate to these conditions is superior compared to that of most of the other vital signs, and the abundance of suitable technological solutions measuring respiratory rate has important implications for healthcare, occupational settings, and sport. However, respiratory rate is still too often not routinely monitored in these fields of use. This review presents a multidisciplinary approach to respiratory monitoring, with the aim to improve the development and efficacy of respiratory monitoring services. We have identified thirteen monitoring goals where the use of the respiratory rate is invaluable, and for each of them we have described suitable sensors and techniques to monitor respiratory rate in specific measurement scenarios. We have also provided a physiological rationale corroborating the importance of respiratory rate monitoring and an original multidisciplinary framework for the development of respiratory monitoring services. This review is expected to advance the field of respiratory monitoring and favor synergies between different disciplines to accomplish this goal.

Vital Sign Monitoring in Car Seats Based on Electrocardiography, Ballistocardiography and Seismocardiography: A Review

This paper focuses on a thorough summary of vital function measuring methods in vehicles. The focus of this paper is to summarize and compare already existing methods integrated into car seats with the implementation of inter alia capacitive electrocardiogram (cECG), mechanical motion analysis Ballistocardiography (BCG) and Seismocardiography (SCG). In addition, a comprehensive overview of other methods of vital sign monitoring, such as camera-based systems or steering wheel sensors, is also presented in this article. Furthermore, this work contains a very thorough background study on advanced signal processing methods and their potential application for the purpose of vital sign monitoring in cars, which is prone to various disturbances and artifacts occurrence that have to be eliminated.

Photoplethysmography imaging:camera performance evaluation by means of an optoelectronic skin perfusion phantom

OBJECTIVE

Photoplethysmography imaging (PPGI) is a promising contactless camera-based method of non-invasive cardiovascular diagnostics. To achieve the best results, it is important to choose the most suitable camera for a specific application. The settings of the camera influence the quality of the detected signal.

APPROACH

The standard (European Machine Vision Association 2016 EMVA Standard 1288-Standard for Characterization of Image Sensors and Cameras pp 1-39 (available at: https://www.emva.org/wp-content/uploads/EMVA1288-3.1a.pdf)) for evaluating the imaging performance of machine vision cameras (MVC) helps at the initial decision of the sensor, but the camera should always be tested in terms of usability for a specific application. So far, PPGI lacks a standardized measurement scenario for evaluating the performance of individual cameras. For this, we realized a controllable optoelectronic phantom with artificial silicone skin allowing reproducible tests of cameras to verify their suitability for PPGI. The entire system is housed in a light-tight box. We tested an MVC, a digital single-lens reflex camera (DSLR) camera and a webcam. Each camera varies in used technology and price.

MAIN RESULTS

We simulated real PPGI measurement conditions simulating the ratio of pulse (AC) and non-pulse (DC) component of the photoplethysmographic signal and achieved AC/DC ratios of 0.5 % on average. An additional OLED panel ensures proper DC providing reproducible measurement conditions. We evaluated the signal morphological features, amplitude spectrum, signal-to-noise ratio (SNR) and spatially dependent changes of simulated subcutaneous perfusion. Here, the MVC proved to be the most suitable device. A DSLR is also suitable for PPGI, but a larger smoothing kernel is required to obtain a perfusion map. The webcam, as the weakest contender, proved to be very susceptible to any inhomogeneous illumination of the examined artificial skin surface. However, it is still able to detect cardiac rhythm.

SIGNIFICANCE

The result of our work is an optoelectronic phantom for reproducible testing of PPGI camera performance in terms of signal quality and measurement equipment costs.

Continuous-Spectrum Infrared Illuminator for Camera-PPG in Darkness

Many camera-based remote photoplethysmography (PPG) applications require sensing in near infrared (NIR). The performance of PPG systems benefits from multi-wavelength processing. The illumination source in such system is explored in this paper. We demonstrate that multiple narrow-band LEDs have inferior color homogeneity compared to broadband light sources. Therefore, we consider the broadband option based on phosphor material excited by LEDs. A first prototype was realized and its details are discussed. It was tested within a remote-PPG monitoring scenario in darkness and the full system demonstrates robust pulse-rate measurement. Given its accuracy in pulse rate extraction, the proposed illumination principle is considered a valuable asset for large-scale NIR-PPG applications as it enables multi-wavelength processing, lightweight set-ups with relatively low-power infrared light sources.

Noise-Resistant CECG Using Novel Capacitive Electrodes

For years, capacitive electrocardiogram (CECG) has been known to be susceptible to ambient interference. In light of this, a novel capacitive electrode was developed as an effective way to reduce the interference effect. This was done by simply introducing the capacitive elector in series with a 1 pF capacitor, and the 60 Hz common mode noise induced by AC power lines was cancelled using a capacitive right leg (CRL) circuit. The proposed electrode did as expected outperform two counterparts in terms of SNR, and particularly gave an up to 99.8% correlation between RRIs extracted from an ECG and a CECG signal, a figure far beyond 52% and 63% using the two counterparts. This capacitive electrode was originally designed for long-term noncontact monitoring of heart rate, and hopefully can be integrated to portable devices for other medical care services in the near future.

Unobtrusive Health Monitoring in Private Spaces: The Smart Vehicle

Unobtrusive in-vehicle health monitoring has the potential to use the driving time to perform regular medical check-ups. This work intends to provide a guide to currently proposed sensor systems for in-vehicle monitoring and to answer, in particular, the questions: (1) Which sensors are suitable for in-vehicle data collection? (2) Where should the sensors be placed? (3) Which biosignals or vital signs can be monitored in the vehicle? (4) Which purposes can be supported with the health data? We reviewed retrospective literature systematically and summarized the up-to-date research on leveraging sensor technology for unobtrusive in-vehicle health monitoring. PubMed, IEEE Xplore, and Scopus delivered 959 articles. We firstly screened titles and abstracts for relevance. Thereafter, we assessed the entire articles. Finally, 46 papers were included and analyzed. A guide is provided to the currently proposed sensor systems. Through this guide, potential sensor information can be derived from the biomedical data needed for respective purposes. The suggested locations for the corresponding sensors are also linked. Fifteen types of sensors were found. Driver-centered locations, such as steering wheel, car seat, and windscreen, are frequently used for mounting unobtrusive sensors, through which some typical biosignals like heart rate and respiration rate are measured. To date, most research focuses on sensor technology development, and most application-driven research aims at driving safety. Health-oriented research on the medical use of sensor-derived physiological parameters is still of interest.

Impact of makeup on remote-PPG monitoring

Camera-based remote photoplethysmography (remote-PPG) enables contactless measurement of blood volume pulse from the human skin. Skin visibility is essential to remote-PPG as the camera needs to capture the light reflected from the skin that penetrates deep into skin tissues and carries blood pulsation information. The use of facial makeup may jeopardize this measurement by reducing the amount of light penetrating into and reflecting from the skin. In this paper, we conduct an empirical study to thoroughly investigate the impact of makeup on remote-PPG monitoring, in both the visible (RGB) and invisible (Near Infrared, NIR) lighting conditions. The experiment shows that makeup has negative influence on remote-PPG, which reduces the relative PPG strength (AC/DC) at different wavelengths and changes the normalized PPG signature across multiple wavelengths. It makes (i) the pulse-rate extraction more difficult in both the RGB and NIR, although NIR is less affected than RGB, and (ii) the blood oxygen saturation extraction in NIR impossible. To the best of our knowledge, this is the first work that systematically investigate the impact of makeup on camera-based remote-PPG monitoring.

Modified RGB Cameras for Infrared Remote-PPG

Multi-wavelength cameras play an essential role in remote photoplethysmography (PPG). Whereas these are readily available for visible light, this is not the case for near infrared (NIR). We propose to modify existing RGB cameras to make them suited for NIR-PPG. In particular, we exploit the spectral leakage of the RGB channels in infrared in combination with a narrow dual-band optical filter. Such camera modification is simple, cost-effective, easy to implement, and it is shown to attain a pulse-rate extraction performance comparable to that of multiple narrow-band NIR cameras.

Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection

Drowsy driving imposes a high safety risk. Current systems often use driving behavior parameters for driver drowsiness detection. The continuous driving automation reduces the availability of these parameters, therefore reducing the scope of such methods. Especially, techniques that include physiological measurements seem to be a promising alternative. However, in a dynamic environment such as driving, only non- or minimal intrusive methods are accepted, and vibrations from the roadbed could lead to degraded sensor technology. This work contributes to driver drowsiness detection with a machine learning approach applied solely to physiological data collected from a non-intrusive retrofittable system in the form of a wrist-worn wearable sensor. To check accuracy and feasibility, results are compared with reference data from a medical-grade ECG device. A user study with 30 participants in a high-fidelity driving simulator was conducted. Several machine learning algorithms for binary classification were applied in user-dependent and independent tests. Results provide evidence that the non-intrusive setting achieves a similar accuracy as compared to the medical-grade device, and high accuracies (>92%) could be achieved, especially in a user-dependent scenario. The proposed approach offers new possibilities for human-machine interaction in a car and especially for driver state monitoring in the field of automated driving.

Delineation of Electrocardiograms Using Multiscale Parameter Estimation

The continuing interest in unobtrusive electrocardiography requires the development of algorithms, compensating for an increased number of artifacts. In previous work, we proposed a framework for robust parameter estimation of signals following a piecewise Gaussian derivative model, well suited for describing all waves of a heartbeat. The framework is based on a numeric and analytic representation of applying the Wavelet Transform at arbitrary scale to the input model. For robustly estimating model parameters, it processes lines of zero-crossings in scale-space, showing high accuracy for various noise models in synthetic signals. An initial evaluation with electrocardiography signals revealed that our basic classifier for identifying the correct lines often fails, leading to false parameter estimates. In this work, we propose a general delineation method based on a solid mathematical framework that treats each heartbeat, wave and fiducial point in the same way, tailored only by intuitive parameters and not relying on any heuristically found decision rules. The steps include a novel line classifier based on pre-filtering using domain knowledge, followed by an exhaustive search among all possible combinations of zero-crossing lines and an error-measure quantifying their agreement with the model. The combination with highest agreement is processed by the parameter estimation framework, customized to the computation of all nine fiducial points. Evaluation using the expert-annotated QT database, shows high sensitivity (P: 99.91%, QRS: 99.92%, T: 99.89%) and mean errors below 1 ms for all onset and offset fiducial points. The proposed combination of line classification and parameter estimation is well suited for delineating electrocardiograms.