Multiple Physiological Signals Fusion Techniques for Improving Heartbeat Detection: A Review

Robust in-vehicle heartbeat detection using multimodal signal fusion

A medical check-up during driving enables the early detection of diseases. Heartbeat irregularities indicate possible cardiovascular diseases, which can be determined with continuous health monitoring. Therefore, we develop a redundant sensor system based on electrocardiography (ECG) and photoplethysmography (PPG) sensors attached to the steering wheel, a red, green, and blue (RGB) camera behind the steering wheel. For the video, we integrate the face recognition engine SeetaFace to detect landmarks of face segments continuously. Based on the green channel, we derive colour changes and, subsequently, the heartbeat. We record the ECG, PPG, video, and reference ECG with body electrodes of 19 volunteers during different driving scenarios, each lasting 15 min: city, highway, and countryside. We combine early, signal-based late, and sensor-based late fusion with a hybrid convolutional neural network (CNN) and integrated majority voting to deliver the final heartbeats that we compare to the reference ECG. Based on the measured and the reference heartbeat positions, the usable time was 51.75%, 58.62%, and 55.96% for the driving scenarios city, highway, and countryside, respectively, with the hybrid algorithm and combination of ECG and PPG. In conclusion, the findings suggest that approximately half the driving time can be utilised for in-vehicle heartbeat monitoring.

An empirical study of modified beat SQI based majority voting fusion method for heart-rate estimation in noisy multimodal cardiovascular signals

Objectives. Most existing heartbeat-detection algorithms rely heavily on cardiovascular signals, namely electrocardiogram (ECG) and arterial blood pressure (ABP), which are often corrupted by noise, leading to unreliable heart-rate estimates. Simultaneously recorded non-cardiovascular (NC) signals help with reliable heart-rate estimates when both cardiovascular signals are corrupted by noise. This study aims to: (i) propose a modified beat signal quality index-based majority voting fusion (MMVF) method to deal with extremely noisy cardiovascular signals; (ii) generate synthetic noise datasets from standard PhysioNet datasets by adding different types of ECG noises, i.e. baseline wander (bw), electrode motion (em), muscle artifact (ma), and realistic artificial ABP noises in clean or low-noise ECG and ABP signals, respectively; and (iii) analyze the effectiveness of the MMVF method for heart-rate estimation with different combinations of beat detectors.Approach.The modified beat signal quality index in the proposed method can identify the quality of the signal even when it contains long durations of noise. The MMVF method assigns uniform weights to the beats detected from all multimodal physiological signals, thus enabling effective participation of beats from NC signals when both cardiovascular signals are corrupted.Results.Fusion of the NC signals with noisy cardiovascular signals using the MMVF method improves heart-rate estimation accuracy over that of single ECG beat detectors like gqrs, epltd, and slope sum function and Teager-Kaiser energy (SSF-TKE) up to 98.81%, 97.95%, and 87.98%, respectively. This method has yielded robust heart-rate estimation within clinically acceptable error limits in concurrently highly noisy cardiovascular signals (ECG: up to a signal-to-noise ratio (SNR) of -70 dB and ABP: up to 100% noise duration in noisy segments) by their fusion with NC signals.Significance.This study serves as empirical evidence for the robustness of the MMVF method in scenarios where there are extremely noisy cardiovascular signals and NC signals with ECG R-peak artifacts.

Detection of COVID-19 using multimodal data from a wearable device: results from the first TemPredict Study

Early detection of diseases such as COVID-19 could be a critical tool in reducing disease transmission by helping individuals recognize when they should self-isolate, seek testing, and obtain early medical intervention. Consumer wearable devices that continuously measure physiological metrics hold promise as tools for early illness detection. We gathered daily questionnaire data and physiological data using a consumer wearable (Oura Ring) from 63,153 participants, of whom 704 self-reported possible COVID-19 disease. We selected 73 of these 704 participants with reliable confirmation of COVID-19 by PCR testing and high-quality physiological data for algorithm training to identify onset of COVID-19 using machine learning classification. The algorithm identified COVID-19 an average of 2.75 days before participants sought diagnostic testing with a sensitivity of 82% and specificity of 63%. The receiving operating characteristic (ROC) area under the curve (AUC) was 0.819 (95% CI [0.809, 0.830]). Including continuous temperature yielded an AUC 4.9% higher than without this feature. For further validation, we obtained SARS CoV-2 antibody in a subset of participants and identified 10 additional participants who self-reported COVID-19 disease with antibody confirmation. The algorithm had an overall ROC AUC of 0.819 (95% CI [0.809, 0.830]), with a sensitivity of 90% and specificity of 80% in these additional participants. Finally, we observed substantial variation in accuracy based on age and biological sex. Findings highlight the importance of including temperature assessment, using continuous physiological features for alignment, and including diverse populations in algorithm development to optimize accuracy in COVID-19 detection from wearables.

Sensor Fusion for Robust Heartbeat Detection during Driving

Private spaces like apartments and vehicles are not yet fully exploited for health monitoring, which includes continuous measurement of biosignals. This work proposes sensor fusion for robust heartbeat detection in the noisy and dynamic driving environment. We use four sensors: electrocardiography (ECG), ballistocardiography (BCG), photoplethysmography (PPG), and image-based PPG (iPPG). As ground truth, we record a 3-lead ECG with wet electrodes attached to the chest. Twelve healthy volunteers are monitored in rest and during driving, each for 11 min. We propose sensor fusion using convolutional neural networks to detect the sensor combination delivering the most accurate heart rate measurement. For rest, we achieve scores of 95.16% (BCG + iPPG), 96.08% (ECG + iPPG), 96.35% (ECG + BCG), 96.53% (ECG + PPG), 96.58% (PPG + iPPG), and 97.15% (BCG + PPG). In motion, the highest scores are 92.46% (BCG + iPPG, PPG + iPPG, ECG + iPPG), 92.83% (ECG + PPG), 93.03% (BCG + PPG), and 93.08% (ECG + BCG). Fusing all four signals with the best fusion approach results in scores of 97.24% (rest) and 94.38% (motion). We conclude that sensor fusion allows robust heartbeat measurement of car drivers to support continuous and unobtrusive health monitoring for early disease detection.

Hidden Markov model-based heartbeat detector using electrocardiogram and arterial pressure signals

The automatic detection of a heartbeat is commonly performed by detecting the QRS complex in the electrocardiogram (ECG), however, various noise sources and missing data can jeopardize the reliability of the ECG. Therefore, there is a growing interest in combining the information from many physiological signals to accurately detect heartbeats. To this end, hidden Markov models (HMMs) are used in this work to jointly exploit the information from ECG, arterial blood pressure (ABP) and pulmonary arterial pressure (PAP) signals in order to conceive a heartbeat detector. After preprocessing the physiological signals, a sliding window is used to extract an observation sequence to be passed through two HMMs (previously trained on a training dataset) in order to obtain the log-likelihoods of observation and signals a detection if the difference of log-likelihoods exceeds an adaptive threshold. Several HMM-based heartbeat detectors were conceived to exploit the information from the ECG, ABP and PAP signals from the MIT-BIH Arrhythmia, PhysioNet Computing in Cardiology Challenge 2014, and MGH/MF Waveform databases. A grid search methodology was used to optimize the duration of the observation sequence and a multiplicative factor to form the adaptive threshold. Using the optimal parameters found on a training database through 10-fold cross-validation, sensitivity and positive predictivity above 99% were obtained on the MIT-BIH Arrhythmia and PhysioNet Computing in Cardiology Challenge 2014 databases, while they are above 95% in the MGH/MF waveform database using ECG and ABP signals. Our detector approach showed detection performances comparable with the literature in the three databases.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-021-00192-x.

Homology Characteristics of EEG and EMG for Lower Limb Voluntary Movement Intention

In the field of lower limb exoskeletons, besides its electromechanical system design and control, attention has been paid to realizing the linkage of exoskeleton robots to humans via electroencephalography (EEG) and electromyography (EMG). However, even the state of the art performance of lower limb voluntary movement intention decoding still faces many obstacles. In the following work, focusing on the perspective of the inner mechanism, a homology characteristic of EEG and EMG for lower limb voluntary movement intention was conducted. A mathematical model of EEG and EMG was built based on its mechanism, which consists of a neural mass model (NMM), neuromuscular junction model, EMG generation model, decoding model, and musculoskeletal biomechanical model. The mechanism analysis and simulation results demonstrated that EEG and EMG signals were both excited by the same movement intention with a response time difference. To assess the efficiency of the proposed model, a synchronous acquisition system for EEG and EMG was constructed to analyze the homology and response time difference from EEG and EMG signals in the limb movement intention. An effective method of wavelet coherence was used to analyze the internal correlation between EEG and EMG signals in the same limb movement intention. To further prove the effectiveness of the hypothesis in this paper, six subjects were involved in the experiments. The experimental results demonstrated that there was a strong EEG-EMG coherence at 1 Hz around movement onset, and the phase of EEG was leading the EMG. Both the simulation and experimental results revealed that EEG and EMG are homologous, and the response time of the EEG signals are earlier than EMG signals during the limb movement intention. This work can provide a theoretical basis for the feasibility of EEG-based pre-perception and fusion perception of EEG and EMG in human movement detection.

El Kassabi H, Ismail H, Nujum Navaz A. ECG Monitoring Systems: Review, Architecture, Processes, and Key Challenges

Health monitoring and its related technologies is an attractive research area. The electrocardiogram (ECG) has always been a popular measurement scheme to assess and diagnose cardiovascular diseases (CVDs). The number of ECG monitoring systems in the literature is expanding exponentially. Hence, it is very hard for researchers and healthcare experts to choose, compare, and evaluate systems that serve their needs and fulfill the monitoring requirements. This accentuates the need for a verified reference guiding the design, classification, and analysis of ECG monitoring systems, serving both researchers and professionals in the field. In this paper, we propose a comprehensive, expert-verified taxonomy of ECG monitoring systems and conduct an extensive, systematic review of the literature. This provides evidence-based support for critically understanding ECG monitoring systems' components, contexts, features, and challenges. Hence, a generic architectural model for ECG monitoring systems is proposed, an extensive analysis of ECG monitoring systems' value chain is conducted, and a thorough review of the relevant literature, classified against the experts' taxonomy, is presented, highlighting challenges and current trends. Finally, we identify key challenges and emphasize the importance of smart monitoring systems that leverage new technologies, including deep learning, artificial intelligence (AI), Big Data and Internet of Things (IoT), to provide efficient, cost-aware, and fully connected monitoring systems.

Digital Volume Pulse Measured at the Fingertip as an Indicator of Diabetic Peripheral Neuropathy in the Aged and Diabetic

This study investigated the application of a modified percussion entropy index (PEI_PPI) in assessing the complexity of baroreflex sensitivity (BRS) for diabetic peripheral neuropathy prognosis. The index was acquired by comparing the obedience of the fluctuation tendency in the change between the amplitudes of continuous digital volume pulse (DVP) and variations in the peak-to-peak interval (PPI) from a decomposed intrinsic mode function (i.e., IMF6) through ensemble empirical mode decomposition (EEMD). In total, 100 middle-aged subjects were split into 3 groups: healthy subjects (group 1, 48–89 years, n = 34), subjects with type 2 diabetes without peripheral neuropathy within 5 years (group 2, 42–86 years, n = 42, HbA1c ≥ 6.5%), and type 2 diabetic patients with peripheral neuropathy within 5 years (group 3, 37–75 years, n = 24). The results were also found to be very successful at discriminating between PEI_PPI values among the three groups (p < 0.017), and indicated significant associations with the anthropometric (i.e., body weight and waist circumference) and serum biochemical (i.e., triglycerides, glycated hemoglobin, and fasting blood glucose) parameters in all subjects (p < 0.05). The present study, which utilized the DVP signals of aged, overweight subjects and diabetic patients, successfully determined the PPI intervals from IMF6 through EEMD. The PEI_PPI can provide a prognosis of peripheral neuropathy from diabetic patients within 5 years after photoplethysmography (PPG) measurement.