Noncontact Monitoring of Heart Rate and Heart Rate Variability in Geriatric Patients Using Photoplethysmography Imaging

Fusion of ballistocardiography and imaging for improved non-contact heart rate monitoring

Camera based imaging is the most widely used technique for non-contact heart rate monitoring (HRM). However, it's robustness to motion artifacts and the absence of reliability when the patient's face is not in the camera's field of view still persists. This is often addressed with computationally heavy AI algorithms or hardware intensive multi-camera systems. Here, we investigate the improvement in accuracy and reliability of noncontact HRM by augmenting vision with an unobtrusive near field sensing modality such as ballistocardiography. The system seamlessly transitions between the two modalities based on a real time signal quality index (SQI). The SQI parameter was able to discard segments of higher errors and select the data stream with lower error. The proposed system was validated on data collected from 20 subjects, with induced motion and facial occlusions to create artifacts. The accuracy was validated against a contact-based ground truth reference. The findings indicate a notable enhancement in the temporal coverage upto 100 percent. The proposed method had a competent error ranging between 3 to 17 bpm, with a median error of 8 bpm.

Video-based analysis of the blink reflex in Parkinson's disease patients

We developed a video-based tool to quantitatively assess the Glabellar Tap Reflex (GTR) in patients with idiopathic Parkinson's disease (iPD) as well as healthy age-matched participants. We also video-graphically assessed the effect of dopaminergic medication on the GTR in iPD patients, as well as the frequency and blinking duration of reflex and non-reflex blinks. The Glabellar Tap Reflex is a clinical sign seen in patients e.g. suffering from iPD. Reliable tools to quantify this sign are lacking.

METHODS

We recorded the GTR in 11 iPD patients and 12 healthy controls (HC) with a consumer-grade camera at a framerate of at least 180 images/s. In these videos, reflex and non-reflex blinks were analyzed for blink count and blinking duration in an automated fashion.

RESULTS

With our setup, the GTR can be extracted from high-framerate cameras using landmarks of the MediaPipe face algorithm. iPD patients did not habituate to the GTR; dopaminergic medication did not alter that response. iPD patients' non-reflex blinks were higher in frequency and higher in blinking duration (width at half prominence); dopaminergic medication decreased the median frequency (Before medication-HC: p < 0.001, After medication-HC: p = 0.0026) and decreased the median blinking duration (Before medication-HC: p = 0.8594, After medication-HC: p = 0.6943)-both in the direction of HC.

CONCLUSION

We developed a quantitative, video-based tool to assess the GTR and other blinking-specific parameters in HC and iPD patients. Further studies could compare the video data to electromyogram (EMG) data for accuracy and comparability, as well as evaluate the specificity of the GTR in patients with other neurodegenerative disorders, in whom the GTR can also be present.

SIGNIFICANCE

The video-based detection of the blinking parameters allows for unobtrusive measurement in patients, a safer and more comfortable option.

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Prediction of Impulsive Aggression Based on Video Images

In response to the subjectivity, low accuracy, and high concealment of existing attack behavior prediction methods, a video-based impulsive aggression prediction method that integrates physiological parameters and facial expression information is proposed. This method uses imaging equipment to capture video and facial expression information containing the subject's face and uses imaging photoplethysmography (IPPG) technology to obtain the subject's heart rate variability parameters. Meanwhile, the ResNet-34 expression recognition model was constructed to obtain the subject's facial expression information. Based on the random forest classification model, the physiological parameters and facial expression information obtained are used to predict individual impulsive aggression. Finally, an impulsive aggression induction experiment was designed to verify the method. The experimental results show that the accuracy of this method for predicting the presence or absence of impulsive aggression was 89.39%. This method proves the feasibility of applying physiological parameters and facial expression information to predict impulsive aggression. This article has important theoretical and practical value for exploring new impulsive aggression prediction methods. It also has significance in safety monitoring in special and public places such as prisons and rehabilitation centers.

Non-contact heart vibration measurement using computer vision-based seismocardiography

Seismocardiography (SCG) is the noninvasive measurement of local vibrations of the chest wall produced by the mechanical activity of the heart and has shown promise in providing clinical information for certain cardiovascular diseases including heart failure and ischemia. Conventionally, SCG signals are recorded by placing an accelerometer on the chest. In this paper, we propose a novel contactless SCG measurement method to extract them from chest videos recorded by a smartphone. Our pipeline consists of computer vision methods including the Lucas-Kanade template tracking to track an artificial target attached to the chest, and then estimate the SCG signals from the tracked displacements. We evaluated our pipeline on 14 healthy subjects by comparing the vision-based SCG[Formula: see text] estimations with the gold-standard SCG[Formula: see text] measured simultaneously using accelerometers attached to the chest. The similarity between SCG[Formula: see text] and SCG[Formula: see text] was measured in the time and frequency domains using the Pearson correlation coefficient, a similarity index based on dynamic time warping (DTW), and wavelet coherence. The average DTW-based similarity index between the signals was 0.94 and 0.95 in the right-to-left and head-to-foot directions, respectively. Furthermore, SCG[Formula: see text] signals were utilized to estimate the heart rate, and these results were compared to the gold-standard heart rate obtained from ECG signals. The findings indicated a good agreement between the estimated heart rate values and the gold-standard measurements (bias = 0.649 beats/min). In conclusion, this work shows promise in developing a low-cost and widely available method for remote monitoring of cardiovascular activity using smartphone videos.

Contactless Technologies, Sensors, and Systems for Cardiac and Respiratory Measurement during Sleep: A Systematic Review

Sleep is essential to physical and mental health. However, the traditional approach to sleep analysis-polysomnography (PSG)-is intrusive and expensive. Therefore, there is great interest in the development of non-contact, non-invasive, and non-intrusive sleep monitoring systems and technologies that can reliably and accurately measure cardiorespiratory parameters with minimal impact on the patient. This has led to the development of other relevant approaches, which are characterised, for example, by the fact that they allow greater freedom of movement and do not require direct contact with the body, i.e., they are non-contact. This systematic review discusses the relevant methods and technologies for non-contact monitoring of cardiorespiratory activity during sleep. Taking into account the current state of the art in non-intrusive technologies, we can identify the methods of non-intrusive monitoring of cardiac and respiratory activity, the technologies and types of sensors used, and the possible physiological parameters available for analysis. To do this, we conducted a literature review and summarised current research on the use of non-contact technologies for non-intrusive monitoring of cardiac and respiratory activity. The inclusion and exclusion criteria for the selection of publications were established prior to the start of the search. Publications were assessed using one main question and several specific questions. We obtained 3774 unique articles from four literature databases (Web of Science, IEEE Xplore, PubMed, and Scopus) and checked them for relevance, resulting in 54 articles that were analysed in a structured way using terminology. The result was 15 different types of sensors and devices (e.g., radar, temperature sensors, motion sensors, cameras) that can be installed in hospital wards and departments or in the environment. The ability to detect heart rate, respiratory rate, and sleep disorders such as apnoea was among the characteristics examined to investigate the overall effectiveness of the systems and technologies considered for cardiorespiratory monitoring. In addition, the advantages and disadvantages of the considered systems and technologies were identified by answering the identified research questions. The results obtained allow us to determine the current trends and the vector of development of medical technologies in sleep medicine for future researchers and research.

Clinical applications of contactless photoplethysmography for monitoring in adults: A systematic review and meta-analysis

Contactless photoplethysmography (cPPG) is a method of physiological monitoring. It differs from conventional monitoring methods (e.g., saturation probe) by ensuring no contact with the subject by use of a camera. The majority of research on cPPG is conducted in a laboratory setting or in healthy populations. This review aims to evaluate the current literature on monitoring using cPPG in adults within a clinical setting. Adhering to the Preferred Items for Systematic Reviews and Meta-analysis (PRISMA, 2020) guidelines, OVID, Webofscience, Cochrane library, and clinicaltrials.org were systematically searched by two researchers. Research articles using cPPG for monitoring purposes in adults within a clinical setting were selected. Twelve studies with a total of 654 individuals were included. Heart rate (HR) was the most investigated vital sign (n = 8) followed by respiratory rate ((n = 2), Sp02 (n = 2), and HR variability (n = 2). Four studies were included in a meta-analysis of HR compared to ECG data which demonstrated a mean bias of -0.13 (95% CI, -1.22-0.96). This study demonstrates cPPG can be a useful tool in the remote monitoring of patients and has demonstrated accuracy for HR. However, further research is needed into the clinical applications of this method.

Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey

Over the last few years, a rich amount of research has been conducted on remote vital sign monitoring of the human body. Remote photoplethysmography (rPPG) is a camera-based, unobtrusive technology that allows continuous monitoring of changes in vital signs and thereby helps to diagnose and treat diseases earlier in an effective manner. Recent advances in computer vision and its extensive applications have led to rPPG being in high demand. This paper specifically presents a survey on different remote photoplethysmography methods and investigates all facets of heart rate analysis. We explore the investigation of the challenges of the video-based rPPG method and extend it to the recent advancements in the literature. We discuss the gap within the literature and suggestions for future directions.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

Dynamic Phase Extraction: Applications in Pulse Rate Variability

Pulse rate variability is a physiological parameter that has been extensively studied and correlated with many physical ailments. However, the phase relationship between inter-beat interval, IBI, and breathing has very rarely been studied. Develop a technique by which the phase relationship between IBI and breathing can be accurately and efficiently extracted from photoplethysmography (PPG) data. A program based on Lock-in Amplifier technology was written in Python to implement a novel technique, Dynamic Phase Extraction. It was tested using a breath pacer and a PPG sensor on 6 subjects who followed a breath pacer at varied breathing rates. The data were then analyzed using both traditional methods and the novel technique (Dynamic Phase Extraction) utilizing a breath pacer. Pulse data was extracted using a PPG sensor. Dynamic Phase Extraction (DPE) gave the magnitudes of the variation in IBI associated with breathing [Formula: see text] measured with photoplethysmography during paced breathing (with premature ventricular contractions, abnormal arrhythmias, and other artifacts edited out). [Formula: see text] correlated well with two standard measures of pulse rate variability: the Standard Deviation of the inter-beat interval (SDNN) (ρ = 0.911) and with the integrated value of the Power Spectral Density between 0.04 and 0.15 Hz (Low Frequency Power or LF Power) (ρ = 0.885). These correlations were comparable to the correlation between the SDNN and the LF Power (ρ = 0.877). In addition to the magnitude [Formula: see text], Dynamic Phase Extraction also gave the phase between the breath pacer and the changes in the inter-beat interval (IBI) due to respiratory sinus arrythmia (RSA), and correlated well with the phase extracted using a Fourier transform (ρ = 0.857). Dynamic Phase Extraction can extract both the phase between the breath pacer and the changes in IBI due to the respiratory sinus arrhythmia component of pulse rate variability ([Formula: see text], but is limited by needing a breath pacer.

Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review

BACKGROUND

Consumer-level cameras have provided an advantage of designing cost-effective, non-contact physiological parameters estimation approaches which is not possible with gold standard estimation techniques. This encourages the development of non-contact estimation methods using camera technology. Therefore, this work aims to present a systematic review summarizing the currently existing face-based non-contact methods along with their performance.

METHODS

This review includes all heart rate (HR) and oxygen saturation (SpO2) studies published in journals and a few reputed conferences, which have compared the proposed estimation methods with one or more standard reference devices. The articles were collected from the following research databases: Institute of Electrical and Electronics Engineers (IEEE), PubMed, Web of Science (WoS), Science Direct, and Association of Computer Machinery (ACM) digital library. All database searches were completed on May 20, 2021. Each study was assessed using a finite set of identified factors for reporting bias.

RESULTS

Out of 332 identified studies, 32 studies were selected for the final review. Additionally, 18 studies were included by thoroughly checking these studies. 3 out of 50 (6%) studies were performed in clinical conditions, while the remaining studies were carried out on a healthy population. 42 out of 50 (84%) studies have estimated HR, while 5/50 (10%) studies have measured SpO₂ only. The remaining three studies have estimated both parameters. The majority of the studies have used 1-3 min videos for estimation. Among the estimation methods, Deep Learning and Independent component analysis (ICA) were used by 11/42 (26.19%) and 9/42 (21.42%) studies, respectively. According to the Bland-Altman analysis, only 8/45 (17.77%) HR studies achieved the clinically accepted error limits whereas, for SpO₂, 4/5 (80%) studies have matched the industry standards (±3%).

DISCUSSION

Deep Learning and ICA have been predominantly used for HR estimations. Among deep learning estimation methods, convolutional neural networks have been employed till date due to their good generalization ability. Most non-contact HR estimation methods need significant improvements to implement these methods in a clinical environment. Furthermore, these methods need to be tested on the subjects suffering from any related disease. SpO₂ estimation studies are challenging and need to be tested by conducting hypoxemic events. The authors would encourage reporting the detailed information about the study population, the use of longer videos, and appropriate performance metrics and testing under abnormal HR and SpO₂ ranges for future estimation studies.

Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review

In recent years, noncontact measurements of vital signs using cameras received a great amount of interest. However, some questions are unanswered: (i) Which vital sign is monitored using what type of camera? (ii) What is the performance and which factors affect it? (iii) Which health issues are addressed by camera-based techniques? Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, we conduct a systematic review of continuous camera-based vital sign monitoring using Scopus, PubMed, and the Association for Computing Machinery (ACM) databases. We consider articles that were published between January 2018 and April 2021 in the English language. We include five vital signs: heart rate (HR), respiratory rate (RR), blood pressure (BP), body skin temperature (BST), and oxygen saturation (SpO2). In total, we retrieve 905 articles and screened them regarding title, abstract, and full text. One hundred and four articles remained: 60, 20, 6, 2, and 1 of the articles focus on HR, RR, BP, BST, and SpO2, respectively, and 15 on multiple vital signs. HR and RR can be measured using red, green, and blue (RGB) and near-infrared (NIR) as well as far-infrared (FIR) cameras. So far, BP and SpO2 are monitored with RGB cameras only, whereas BST is derived from FIR cameras only. Under ideal conditions, the root mean squared error is around 2.60 bpm, 2.22 cpm, 6.91 mm Hg, 4.88 mm Hg, and 0.86 °C for HR, RR, systolic BP, diastolic BP, and BST, respectively. The estimated error for SpO2 is less than 1%, but it increases with movements of the subject and the camera-subject distance. Camera-based remote monitoring mainly explores intensive care, post-anaesthesia care, and sleep monitoring, but also explores special diseases such as heart failure. The monitored targets are newborn and pediatric patients, geriatric patients, athletes (e.g., exercising, cycling), and vehicle drivers. Camera-based techniques monitor HR, RR, and BST in static conditions within acceptable ranges for certain applications. The research gaps are large and heterogeneous populations, real-time scenarios, moving subjects, and accuracy of BP and SpO2 monitoring.

Camera-based heart rate estimation for hospitalized newborns in the presence of motion artifacts

Background

Heart rate (HR) is an important vital sign for evaluating the physiological condition of a newborn infant. Recently, for measuring HR, novel RGB camera-based non-contact techniques have demonstrated their specific superiority compared with other techniques, such as dopplers and thermal cameras. However, they still suffered poor robustness in infants’ HR measurements due to frequent body movement.

Methods

This paper introduces a framework to improve the robustness of infants’ HR measurements by solving motion artifact problems. Our solution is based on the following steps: morphology-based filtering, region-of-interest (ROI) dividing, Eulerian video magnification and majority voting. In particular, ROI dividing improves ROI information utilization. The majority voting scheme improves the statistical robustness by choosing the HR with the highest probability. Additionally, we determined the dividing parameter that leads to the most accurate HR measurements. In order to examine the performance of the proposed method, we collected 4 hours of videos and recorded the corresponding electrocardiogram (ECG) of 9 hospitalized neonates under two different conditions—rest still and visible movements.

Results

Experimental results indicate a promising performance: the mean absolute error during rest still and visible movements are 3.39 beats per minute (BPM) and 4.34 BPM, respectively, which improves at least 2.00 and 1.88 BPM compared with previous works. The Bland-Altman plots also show the remarkable consistency of our results and the HR derived from the ground-truth ECG.

Conclusions

To the best of our knowledge, this is the first study aimed at improving the robustness of neonatal HR measurement under motion artifacts using an RGB camera. The preliminary results have shown the promising prospects of the proposed method, which hopefully reduce neonatal mortality in hospitals.

Non-Contact Monitoring and Classification of Breathing Pattern for the Supervision of People Infected by COVID-19

During the pandemic of coronavirus disease-2019 (COVID-19), medical practitioners need non-contact devices to reduce the risk of spreading the virus. People with COVID-19 usually experience fever and have difficulty breathing. Unsupervised care to patients with respiratory problems will be the main reason for the rising death rate. Periodic linearly increasing frequency chirp, known as frequency-modulated continuous wave (FMCW), is one of the radar technologies with a low-power operation and high-resolution detection which can detect any tiny movement. In this study, we use FMCW to develop a non-contact medical device that monitors and classifies the breathing pattern in real time. Patients with a breathing disorder have an unusual breathing characteristic that cannot be represented using the breathing rate. Thus, we created an Xtreme Gradient Boosting (XGBoost) classification model and adopted Mel-frequency cepstral coefficient (MFCC) feature extraction to classify the breathing pattern behavior. XGBoost is an ensemble machine-learning technique with a fast execution time and good scalability for predictions. In this study, MFCC feature extraction assists machine learning in extracting the features of the breathing signal. Based on the results, the system obtained an acceptable accuracy. Thus, our proposed system could potentially be used to detect and monitor the presence of respiratory problems in patients with COVID-19, asthma, etc.

Contact-free sensor signals as a new digital biomarker for cardiovascular disease: chances and challenges

Multiple sensor systems are used to monitor physiological parameters, activities of daily living and behaviour. Digital biomarkers can be extracted and used as indicators for health and disease. Signal acquisition is either by object sensors, wearable sensors, or contact-free sensors including cameras, pressure sensors, non-contact capacitively coupled electrocardiogram (cECG), radar, and passive infrared motion sensors. This review summarizes contemporary knowledge of the use of contact-free sensors for patients with cardiovascular disease and healthy subjects following the PRISMA declaration. Chances and challenges are discussed. Thirty-six publications were rated to be of medium (31) or high (5) relevance. Results are best for monitoring of heart rate and heart rate variability using cardiac vibration, facial camera, or cECG; for respiration using cardiac vibration, cECG, or camera; and for sleep using ballistocardiography. Early results from radar sensors to monitor vital signs are promising. Contact-free sensors are little invasive, well accepted and suitable for long-term monitoring in particular in patient's homes. A major problem are motion artefacts. Results from long-term use in larger patient cohorts are still lacking, but the technology is about to emerge the market and we can expect to see more clinical results in the near future.