On-Device Reliability Assessment and Prediction of Missing Photoplethysmographic Data Using Deep Neural Networks

Machine learning assisted classification between diabetic polyneuropathy and healthy subjects using plantar pressure and temperature data: a feasibility study

Automated and early detection of diabetics with polyneuropathy in an ambulatory health monitoring setup may reduce the major risk factors for diabetic patients. Increased and localized plantar pressure associated with impaired pain and temperature is a combination of developing foot ulcers in subjects with polyneuropathy. Although many interesting research works have been reported in this area, most of them emphasize on signal acquisition process and plantar pressure distribution in the foot region. In this work, a machine learning assisted low complexity technique was developed using plantar pressure and temperature signals which will classify between diabetic polyneuropathy and healthy subjects. Principal component analysis (PCA) and maximum relevance minimum redundancy (mRMR) methods were used for feature extraction and selection respectively followed by k-NN classifier for binary classification. The proposed technique was evaluated with 100 min of publicly available annotated data from 43 subjects and provides blind test accuracy, sensitivity, precision, F1-score, and area under curve (AUC) of 99.58%, 99.50%, 99.44%, 99.47% and 99.56% respectively. A low resource hardware implementation in ARM v6 controller required an average memory usage of 81.2 kB and latency of 1.31 s to process 9 s pressure and temperature data collected from 16 sensor channels for each of the foot region.

Reconstruction of Missing Electrocardiography Signals from Photoplethysmography Data Using Deep Neural Network

ECG helps in diagnosing heart disease by recording heart activity. During long-term measurements, data loss occurs due to sensor detachment. Therefore, research into the reconstruction of missing ECG data is essential. However, ECG requires user participation and cannot be used for continuous heart monitoring. Continuous monitoring of PPG signals is conversely low-cost and easy to carry out. In this study, a deep neural network model is proposed for the reconstruction of missing ECG signals using PPG data. This model is an end-to-end deep learning neural network utilizing WNet architecture as a basis, on which a bidirectional long short-term memory network is added in establishing a second model. The performance of both models is verified using 146 records from the MIMIC III matched subset. Compared with the reference, the ECG reconstructed using the proposed model has a Pearson's correlation coefficient of 0.851, root mean square error (RMSE) of 0.075, percentage root mean square difference (PRD) of 5.452, and a Fréchet distance (FD) of 0.302. The experimental results demonstrate that it is feasible to reconstruct missing ECG signals from PPG.

Learning-Based Multimodal Information Fusion and Behavior Recognition of Vascular Interventionists' Operating Skills

The operating skills of vascular interventionists have an important impact on the effect of surgery. However, current research on behavior recognition and skills learning of interventionists' operating skills is limited. In this study, an innovative deep learning-based multimodal information fusion architecture is proposed for recognizing and analyzing eight common operating behaviors of interventionists. An experimental platform integrating four modal sensors is used to collect multimodal data from interventionists. The ANOVA and Manner-Whitney tests is used for relevance analysis of the data. The analysis results demonstrate that there is almost no significant difference ( p <0.001) between the actions related to the unimodal data, which cannot be used for accurate behavior recognition. Therefore, a study of the fusion architecture based on the existing machine learning classifier and the proposed deep learning fusion architecture is carried out. The research findings indicate that the proposed deep learning-based fusion architecture achieves an impressive overall accuracy of 98.5%, surpassing both the machine learning classifier (93.51%) and the unimodal data (90.05%). The deep learning-based multimodal information fusion architecture proves the feasibility of behavior recognition and skills learning of interventionist's operating skills. Furthermore, the application of deep learning-based multimodal fusion technology of surgeon's operating skills will help to improve the autonomy and intelligence of surgical robotic systems.

DNN based reliability evaluation for telemedicine data

Telemedicine data are measured directly by untrained patients, which may cause problems in data reliability. Many deep learning-based studies have been conducted to improve the quality of measurement data. However, they could not provide an accurate basis for judgment. Therefore, this study proposed a deep neural network filter-based reliability evaluation system that could present an accurate basis for judgment and verified its reliability by evaluating photoplethysmography signal and change in data quality according to judgment criteria through clinical trials. In the results, the deviation of 3% or more when the oxygen saturation was judged as normal according to each criterion was 0.3% and 0.82% for criteria 1 and 2, respectively, which was very low compared to the abnormal judgment (3.86%). The deviation of diastolic blood pressure (≥ 10 mmHg) according to criterion 3 was reduced by about 4% in the normal judgment compared to the abnormal. In addition, when multiple judgment conditions were satisfied, abnormal data were better discriminated than when only one criterion was satisfied. Therefore, the basis for judging abnormal data can be presented with the system proposed in this study, and the quality of telemedicine data can be improved according to the judgment result.

Multimodal Multiresolution Data Fusion Using Convolutional Neural Networks for IoT Wearable Sensing

With advances in circuit design and sensing technology, the acquisition of data from a large number of Internet of Things (IoT) sensors simultaneously to enable more accurate inferences has become mainstream. In this work, we propose a novel convolutional neural network (CNN) model for the fusion of multimodal and multiresolution data obtained from several sensors. The proposed model enables the fusion of multiresolution sensor data, without having to resort to padding/ resampling to correct for frequency resolution differences even when carrying out temporal inferences like high-resolution event detection. The performance of the proposed model is evaluated for sleep apnea event detection, by fusing three different sensor signals obtained from UCD St. Vincent University Hospital's sleep apnea database. The proposed model is generalizable and this is demonstrated by incremental performance improvements, proportional to the number of sensors used for fusion. A selective dropout technique is used to prevent overfitting of the model to any specific high-resolution input, and increase the robustness of fusion to signal corruption from any sensor source. A fusion model with electrocardiogram (ECG), Peripheral oxygen saturation signal (SpO2), and abdominal movement signal achieved an accuracy of 99.72% and a sensitivity of 98.98%. Energy per classification of the proposed fusion model was estimated to be approximately 5.61 μJ for on-chip implementation. The feasibility of pruning to reduce the complexity of the fusion models was also studied.

Smart Mandibular Advancement Device for Intraoral Monitoring of Cardiorespiratory Parameters and Sleeping Postures

Obstructive sleep apnea (OSA), as a highly prevalent sleep disorder, causes several serious health complaints. It has been proved that using intraoral mandibular advancement devices (MADs) during sleep is an efficient treatment for OSA. However, due to limited number of sleep study laboratories, effectiveness of MAD therapy is not regularly monitored. This paper proposes a smart MAD with the capability of continuously monitoring of cardiorespiratory parameters as well as sleeping postures and breathing routes. In this regard, a flexible hybrid wireless sensing platform based on the intraoral photoplethysmography (PPG), temperature and accelerometry monitoring is developed. It is qualitatively and quantitatively discussed that the intraorally captured PPG signals by the smart MAD have similar features as the ones received from the conventional anatomical position, i.e., the left index fingertip. Extensive experimental measurements indicate that the proposed smart MAD can estimate heart-rate (HR), respiration rate (RR) and blood oxygen saturation (SpO₂) with the maximum mean-absolute-errors of 2.4 bpm, 2.52 breaths/min, and 0.8%, respectively, in comparison to the reference measurements, while such a capability is not dependent on subject's positions and breathing routes. It is also shown that the smart MAD can readily identify different sleeping postures, namely, supine, left, right, and prone and breathing routes. The reliability and stability of the proposed smart MAD's measurements are proved by examining a group of subjects. The proposed smart MAD has potential to monitor the effectiveness of MAD treatment and eliminate untreated OSA without the requirement of attaching an extra monitoring platform to the patient's body.

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.