A Deep Machine Learning Method for Classifying Cyclic Time Series of Biological Signals Using Time-Growing Neural Network

Integrated prediction and control of mobility changes of young talents in the field of science and technology based on convolutional neural network

As the scientific and technological levels continue to rise, the dynamics of young talent within these fields are increasingly significant. Currently, there is a lack of comprehensive models for predicting the movement of young professionals in science and technology. To address this gap, this study introduces an integrated approach to forecasting and managing the flow of these talents, leveraging the power of convolutional neural networks (CNNs). The performance test of the proposed method shows that the prediction accuracy of this method is 76.98%, which is superior to the two comparison methods. In addition, the results showed that the average error of the model was 0.0285 lower than that of the model based on the recurrent prediction error (RPE) algorithm learning algorithm, and the average time was 41.6 s lower than that of the model based on the backpropagation (BP) learning algorithm. In predicting the flow of young talent, the study uses flow characteristics including personal characteristics, occupational characteristics, organizational characteristics and network characteristics. Through the above results, the study found that convolutional neural network can effectively use these features to predict the flow of young talents, and its model is superior to other commonly used models in processing speed and accuracy. The above results indicate that the model can provide organizations and government agencies with useful information about the flow trend of young talents, and help them to formulate better talent management strategies.

Reservoir computing models based on spiking neural P systems for time series classification

Nonlinear spiking neural P (NSNP) systems are neural-like membrane computing models with nonlinear spiking mechanisms. Because of this nonlinear spiking mechanism, NSNP systems can show rich nonlinear dynamics. Reservoir computing (RC) is a novel recurrent neural network (RNN) and can overcome some shortcomings of traditional RNNs. Based on NSNP systems, we developed two RC variants for time series classification, RC-SNP and RC-RMS-SNP, which are without and integrated with reservoir model space (RMS), respectively. The two RC variants use NSNP systems as the reservoirs and can be easily implemented in the RC framework. The proposed two RC variants were evaluated on 17 benchmark time series classification datasets and compared with 16 state-of-the-art or baseline classification models. The comparison results demonstrate the effectiveness of the proposed two RC variants for time series classification tasks.

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Learning time-series representations when only unlabeled data or few labeled samples are available can be a challenging task. Recently, contrastive self-supervised learning has shown great improvement in extracting useful representations from unlabeled data via contrasting different augmented views of data. In this work, we propose a novel Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC) that learns representations from unlabeled data with contrastive learning. Specifically, we propose time-series-specific weak and strong augmentations and use their views to learn robust temporal relations in the proposed temporal contrasting module, besides learning discriminative representations by our proposed contextual contrasting module. Additionally, we conduct a systematic study of time-series data augmentation selection, which is a key part of contrastive learning. We also extend TS-TCC to the semi-supervised learning settings and propose a Class-Aware TS-TCC (CA-TCC) that benefits from the available few labeled data to further improve representations learned by TS-TCC. Specifically, we leverage the robust pseudo labels produced by TS-TCC to realize a class-aware contrastive loss. Extensive experiments show that the linear evaluation of the features learned by our proposed framework performs comparably with the fully supervised training. Additionally, our framework shows high efficiency in few labeled data and transfer learning scenarios.

Somtimes: self organizing maps for time series clustering and its application to serious illness conversations

There is demand for scalable algorithms capable of clustering and analyzing large time series data. The Kohonen self-organizing map (SOM) is an unsupervised artificial neural network for clustering, visualizing, and reducing the dimensionality of complex data. Like all clustering methods, it requires a measure of similarity between input data (in this work time series). Dynamic time warping (DTW) is one such measure, and a top performer that accommodates distortions when aligning time series. Despite its popularity in clustering, DTW is limited in practice because the runtime complexity is quadratic with the length of the time series. To address this, we present a new a self-organizing map for clustering TIME Series, called SOMTimeS, which uses DTW as the distance measure. The method has similar accuracy compared with other DTW-based clustering algorithms, yet scales better and runs faster. The computational performance stems from the pruning of unnecessary DTW computations during the SOM's training phase. For comparison, we implement a similar pruning strategy for K-means, and call the latter K-TimeS. SOMTimeS and K-TimeS pruned 43% and 50% of the total DTW computations, respectively. Pruning effectiveness, accuracy, execution time and scalability are evaluated using 112 benchmark time series datasets from the UC Riverside classification archive, and show that for similar accuracy, a 1.8× speed-up on average for SOMTimeS and K-TimeS, respectively with that rates vary between 1× and 18× depending on the dataset. We also apply SOMTimeS to a healthcare study of patient-clinician serious illness conversations to demonstrate the algorithm's utility with complex, temporally sequenced natural language.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10618-023-00979-9.

A learnable front-end based efficient channel attention network for heart sound classification

Objective. To enhance the accuracy of heart sound classification, this study aims to overcome the limitations of common models which rely on handcrafted feature extraction. These traditional methods may distort or discard crucial pathological information within heart sounds due to their requirement of tedious parameter settings.Approach.We propose a learnable front-end based Efficient Channel Attention Network (ECA-Net) for heart sound classification. This novel approach optimizes the transformation of waveform-to-spectrogram, enabling adaptive feature extraction from heart sound signals without domain knowledge. The features are subsequently fed into an ECA-Net based convolutional recurrent neural network, which emphasizes informative features and suppresses irrelevant information. To address data imbalance, Focal loss is employed in our model.Main results.Using the well-known public PhysioNet challenge 2016 dataset, our method achieved a classification accuracy of 97.77%, outperforming the majority of previous studies and closely rivaling the best model with a difference of just 0.57%.Significance.The learnable front-end facilitates end-to-end training by replacing the conventional heart sound feature extraction module. This provides a novel and efficient approach for heart sound classification research and applications, enhancing the practical utility of end-to-end models in this field.

Accurate detection of paroxysmal atrial fibrillation with certified-GAN and neural architecture search

This paper presents a novel machine learning framework for detecting PxAF, a pathological characteristic of electrocardiogram (ECG) that can lead to fatal conditions such as heart attack. To enhance the learning process, the framework involves a generative adversarial network (GAN) along with a neural architecture search (NAS) in the data preparation and classifier optimization phases. The GAN is innovatively invoked to overcome the class imbalance of the training data by producing the synthetic ECG for PxAF class in a certified manner. The effect of the certified GAN is statistically validated. Instead of using a general-purpose classifier, the NAS automatically designs a highly accurate convolutional neural network architecture customized for the PxAF classification task. Experimental results show that the accuracy of the proposed framework exhibits a high value of 99.0% which not only enhances state-of-the-art by up to 5.1%, but also improves the classification performance of the two widely-accepted baseline methods, ResNet-18, and Auto-Sklearn, by [Formula: see text] and [Formula: see text].

Stochastic Stability of Markovian Neural Networks With Generally Hybrid Transition Rates

This article studies the problem of the stability for Markovian neural networks (MNNs) with time delay. The transition rate is considered to be generally hybrid, which treats those existing ones as its special cases. The introduced generally hybrid transition rates (GHTRs) make these systems more general and practical. Apropos of the GHTRs, a double-boundary approach rather than the traditional estimation method is introduced to make full use of the error information in GHTRs. In order to fully capture system information, a parameter-type-delay-dependent-matrix (PTDDM) approach is proposed, in which the PTDDM approach removes some zero components on slack matrices in previous works. Thus, the PTDDM approach can fully link the relationship among time delay and state-related vectors. Based on these ingredients, a novel stochastic stability condition is proposed for MNNs with GHTRs. A numerical example is illustrated to demonstrate the effectiveness of the proposed approaches.

DeepBarcoding: Deep Learning for Species Classification Using DNA Barcoding

DNA barcodes with short sequence fragments are used for species identification. Because of advances in sequencing technologies, DNA barcodes have gradually been emphasized. DNA sequences from different organisms are easily and rapidly acquired. Therefore, DNA sequence analysis tools play an increasingly crucial role in species identification. This study proposed deep barcoding, a deep learning framework for species classification by using DNA barcodes. Deep barcoding uses raw sequence data as the input to represent one-hot encoding as a one-dimensional image and uses a deep convolutional neural network with a fully connected deep neural network for sequence analysis. It can achieve an average accuracy of >90 percent for both simulation and real datasets. Although deep learning yields outstanding performance for species classification with DNA sequences, its application remains a challenge. The deep barcoding model can be a potential tool for species classification and can elucidate DNA barcode-based species identification.

Self-Supervised Autoregressive Domain Adaptation for Time Series Data

Unsupervised domain adaptation (UDA) has successfully addressed the domain shift problem for visual applications. Yet, these approaches may have limited performance for time series data due to the following reasons. First, they mainly rely on the large-scale dataset (i.e., ImageNet) for source pretraining, which is not applicable for time series data. Second, they ignore the temporal dimension on the feature space of the source and target domains during the domain alignment step. Finally, most of the prior UDA methods can only align the global features without considering the fine-grained class distribution of the target domain. To address these limitations, we propose a SeLf-supervised AutoRegressive Domain Adaptation (SLARDA) framework. In particular, we first design a self-supervised (SL) learning module that uses forecasting as an auxiliary task to improve the transferability of source features. Second, we propose a novel autoregressive domain adaptation technique that incorporates temporal dependence of both source and target features during domain alignment. Finally, we develop an ensemble teacher model to align class-wise distribution in the target domain via a confident pseudo labeling approach. Extensive experiments have been conducted on three real-world time series applications with 30 cross-domain scenarios. The results demonstrate that our proposed SLARDA method significantly outperforms the state-of-the-art approaches for time series domain adaptation. Our source code is available at: https://github.com/mohamedr002/SLARDA.

RBF Sliding Mode Control Method for an Upper Limb Rehabilitation Exoskeleton Based on Intent Recognition

Aiming at the lack of active willingness of patients to participate in the current upper limb exoskeleton rehabilitation training control methods, this study proposed a radial basis function (RBF) sliding mode impedance control method based on surface electromyography (sEMG) to identify the movement intention of upper limb rehabilitation. The proposed control method realizes the process of active and passive rehabilitation training according to the wearer’s movement intention. This study first established a joint angle prediction model based on sEMG for the problem of poor human–machine coupling and used the least-squares support vector machine method (LSSVM) to complete the upper limb joint angle prediction. In addition, in view of the problem of poor compliance in the rehabilitation training process, an adaptive sliding mode controller based on the RBF network approximation system model was proposed. In the process of active training, an impedance model was added based on the position loop control, which could dynamically adjust the motion trajectory according to the interaction force. The experiment results showed that the impedance control method based on the RBF could effectively reduce the interaction force between the human and machine to improve the compliance of the exoskeleton manipulator and achieve the purpose of stabilizing the impedance characteristics of the system.

Cross-Domain Missingness-Aware Time-Series Adaptation With Similarity Distillation in Medical Applications

Medical time series of laboratory tests has been collected in electronic health records (EHRs) in many countries. Machine-learning algorithms have been proposed to analyze the condition of patients using these medical records. However, medical time series may be recorded using different laboratory parameters in different datasets. This results in the failure of applying a pretrained model on a test dataset containing a time series of different laboratory parameters. This article proposes to solve this problem with an unsupervised time-series adaptation method that generates time series across laboratory parameters. Specifically, a medical time-series generation network with similarity distillation is developed to reduce the domain gap caused by the difference in laboratory parameters. The relations of different laboratory parameters are analyzed, and the similarity information is distilled to guide the generation of target-domain specific laboratory parameters. To further improve the performance in cross-domain medical applications, a missingness-aware feature extraction network is proposed, where the missingness patterns reflect the health conditions and, thus, serve as auxiliary features for medical analysis. In addition, we also introduce domain-adversarial networks in both feature level and time-series level to enhance the adaptation across domains. Experimental results show that the proposed method achieves good performance on both private and publicly available medical datasets. Ablation studies and distribution visualization are provided to further analyze the properties of the proposed method.

Exploiting deep neural network and long short-term memory method-ologies in bioacoustic classification of LPC-based features

The research describes the recognition and classification of the acoustic characteristics of amphibians using deep learning of deep neural network (DNN) and long short-term memory (LSTM) for biological applications. First, original data is collected from 32 species of frogs and 3 species of toads commonly found in Taiwan. Secondly, two digital filtering algorithms, linear predictive coding (LPC) and Mel-frequency cepstral coefficient (MFCC), are respectively used to collect amphibian bioacoustic features and construct the datasets. In addition, principal component analysis (PCA) algorithm is applied to achieve dimensional reduction of the training model datasets. Next, the classification of amphibian bioacoustic features is accomplished through the use of DNN and LSTM. The Pytorch platform with a GPU processor (NVIDIA GeForce GTX 1050 Ti) realizes the calculation and recognition of the acoustic feature classification results. Based on above-mentioned two algorithms, the sound feature datasets are classified and effectively summarized in several classification result tables and graphs for presentation. The results of the classification experiment of the different features of bioacoustics are verified and discussed in detail. This research seeks to extract the optimal combination of the best recognition and classification algorithms in all experimental processes.

Respiratory sound classification for crackles, wheezes, and rhonchi in the clinical field using deep learning

Auscultation has been essential part of the physical examination; this is non-invasive, real-time, and very informative. Detection of abnormal respiratory sounds with a stethoscope is important in diagnosing respiratory diseases and providing first aid. However, accurate interpretation of respiratory sounds requires clinician's considerable expertise, so trainees such as interns and residents sometimes misidentify respiratory sounds. To overcome such limitations, we tried to develop an automated classification of breath sounds. We utilized deep learning convolutional neural network (CNN) to categorize 1918 respiratory sounds (normal, crackles, wheezes, rhonchi) recorded in the clinical setting. We developed the predictive model for respiratory sound classification combining pretrained image feature extractor of series, respiratory sound, and CNN classifier. It detected abnormal sounds with an accuracy of 86.5% and the area under the ROC curve (AUC) of 0.93. It further classified abnormal lung sounds into crackles, wheezes, or rhonchi with an overall accuracy of 85.7% and a mean AUC of 0.92. On the other hand, as a result of respiratory sound classification by different groups showed varying degree in terms of accuracy; the overall accuracies were 60.3% for medical students, 53.4% for interns, 68.8% for residents, and 80.1% for fellows. Our deep learning-based classification would be able to complement the inaccuracies of clinicians' auscultation, and it may aid in the rapid diagnosis and appropriate treatment of respiratory diseases.

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Multi-feature gait recognition with DNN based on sEMG signals

This study proposed a gait recognition method based on the deep neural network of surface electromyography (sEMG) signals to improve the stability and accuracy of gait recognition using sEMG signals of the lower limbs. First, we determined the parameters of time domain features, including the mean of absolute value, root mean square, waveform length, the number of zero-crossing points of the sEMG signals after noise elimination, and the frequency domain features, including mean power frequency and median frequency. Second, the time domain feature and frequency domain feature were combined into a multi-feature combination. Then, the classifier was trained and used for gait recognition. Finally, in terms of the recognition rate, the classifier was compared with the support vector machine (SVM) and extreme learning machine (ELM). The results showed the method of deep neural network (DNN) had a better recognition rate than that of SVM and ELM. The experimental results of the participants indicated that the average recognition rate obtained with the method of DNN exceeded 95%. On the other hand, from the statistical results of standard deviation, the difference between subjects ranged from 0.46 to 0.94%, which also proved the robustness and stability of the proposed method.

A Machine Learning decision-making tool for extubation in Intensive Care Unit patients

BACKGROUND AND OBJECTIVE

To increase the success rate of invasive mechanical ventilation weaning in critically ill patients using Machine Learning models capable of accurately predicting the outcome of programmed extubations.

METHODS

The study population was adult patients admitted to the Intensive Care Unit. Target events were programmed extubations, both successful and failed. The working dataset is assembled by combining heterogeneous data including time series from Clinical Information Systems, patient demographics, medical records and respiratory event logs. Three classification learners have been compared: Logistic Discriminant Analysis, Gradient Boosting Method and Support Vector Machines. Standard methodologies have been used for preprocessing, hyperparameter tuning and resampling.

RESULTS

The Support Vector Machine classifier is found to correctly predict the outcome of an extubation with a 94.6% accuracy. Contrary to current decision-making criteria for extubation based on Spontaneous Breathing Trials, the classifier predictors only require monitor data, medical entry records and patient demographics.

CONCLUSIONS

Machine Learning-based tools have been found to accurately predict the extubation outcome in critical patients with invasive mechanical ventilation. The use of this important predictive capability to assess the extubation decision could potentially reduce the rate of extubation failure, currently at 9%. With about 40% of critically ill patients eventually receiving invasive mechanical ventilation during their stay and given the serious potential complications associated to reintubation, the excellent predictive ability of the model presented here suggests that Machine Learning techniques could significantly improve the clinical outcomes of critical patients.

Gated temporal convolutional neural network and expert features for diagnosing and explaining physiological time series: A case study on heart rates

BACKGROUND AND OBJECTIVE

Physiological time series are common data sources in many health applications. Mining data from physiological time series is crucial for promoting healthy living and reducing governmental medical expenditure. Recently, research and applications of deep learning methods on physiological time series have developed rapidly because such data can be continuously recorded by smart wristbands or smartwatches. However, existing deep learning methods suffer from excessive model complexity and a lack of explanation. This paper aims to handle these issues.

METHODS

We propose TEG-net, which is a novel deep learning method for accurately diagnosing and explaining physiological time series. TEG-net constructs T-net (a multi-scale bi-directional temporal convolutional neural network) to model physiological time series directly, E-net (personalized linear model) to model expert features extracted from physiological time series, and G-net (gating neural network) to combine T-net and E-net for diagnosis. The combination of T-net and E-net through G-net improves diagnosis accuracy and E-net can be utilized for explanation.

RESULTS

Experimental results demonstrate that TEG-net outperforms the second-best baseline by 13.68% in terms of area under the receiver operating characteristic curve and 11.49% in terms of area under the precision-recall curve. Additionally, intuitive justifications can be provided to explain model predictions.

CONCLUSIONS

This paper develops an ensemble method to combine expert features and deep learning method for modeling physiological time series. Improvements in diagnostic accuracy and explanation make TEG-net applicable to many real-world health applications.

CNFE-SE: a novel approach combining complex network-based feature engineering and stacked ensemble to predict the success of intrauterine insemination and ranking the features

Background

Intrauterine Insemination (IUI) outcome prediction is a challenging issue which the assisted reproductive technology (ART) practitioners are dealing with. Predicting the success or failure of IUI based on the couples' features can assist the physicians to make the appropriate decision for suggesting IUI to the couples or not and/or continuing the treatment or not for them. Many previous studies have been focused on predicting the in vitro fertilization (IVF) and intracytoplasmic sperm injection (ICSI) outcome using machine learning algorithms. But, to the best of our knowledge, a few studies have been focused on predicting the outcome of IUI. The main aim of this study is to propose an automatic classification and feature scoring method to predict intrauterine insemination (IUI) outcome and ranking the most significant features.

Methods

For this purpose, a novel approach combining complex network-based feature engineering and stacked ensemble (CNFE-SE) is proposed. Three complex networks are extracted considering the patients' data similarities. The feature engineering step is performed on the complex networks. The original feature set and/or the features engineered are fed to the proposed stacked ensemble to classify and predict IUI outcome for couples per IUI treatment cycle. Our study is a retrospective study of a 5-year couples' data undergoing IUI. Data is collected from Reproductive Biomedicine Research Center, Royan Institute describing 11,255 IUI treatment cycles for 8,360 couples. Our dataset includes the couples' demographic characteristics, historical data about the patients' diseases, the clinical diagnosis, the treatment plans and the prescribed drugs during the cycles, semen quality, laboratory tests and the clinical pregnancy outcome.

Results

Experimental results show that the proposed method outperforms the compared methods with Area under receiver operating characteristics curve (AUC) of 0.84 ± 0.01, sensitivity of 0.79 ± 0.01, specificity of 0.91 ± 0.01, and accuracy of 0.85 ± 0.01 for the prediction of IUI outcome.

Conclusions

The most important predictors for predicting IUI outcome are semen parameters (sperm motility and concentration) as well as female body mass index (BMI).

Phonocardiography-based mitral valve prolapse detection with using fractional fourier transform

Mitral Valve Prolapse (MVP) is a common condition among people, which is often benign and does not need any serious treatment. However, this doesn't mean that MVP can't cause any problems. In malignant conditions, MVP can cause mitral failure and also heart failure. Early diagnosis of MVP is significantly important to control and reduce its complications. Since the phonocardiogram signal provides useful information about heart valves function, it can be used for MVP detection. To detect MVP, the signal was denoised and segmented into heart cycles and constant three-second pieces in the first and second approaches, respectively. Next, based on the Fractional Fourier Transform (FrFT), the desired features were extracted. Then, the extracted features were windowed by a Moving Logarithmic Median Window (MLMW) and optimum features were selected using Mahalanobis, Bhattacharyya, Canberra, and Minkowski distance criteria. Finally, using the selected features, classification was performed by using the K-Nearest Neighbor (KNN) and the Suppor Vector Machine (SVM) classifiers to find out whether a segment is prolapsed. The best results of the experiment on the collected database contain 15 prolapsed and 6 non-prolapsed subjects using the A-test method show 96.25 ± 2.43 accuracy, 98.5 ± 3.37 sensitivity, 94.0 ± 5.16 specificity, 96.0 ± 3.44 precision, 92.5 ± 4.86 kappa, and 96.6 ± 2.34 f-score with the SVM classifier.

Deep Adaptive Input Normalization for Time Series Forecasting

Deep learning (DL) models can be used to tackle time series analysis tasks with great success. However, the performance of DL models can degenerate rapidly if the data are not appropriately normalized. This issue is even more apparent when DL is used for financial time series forecasting tasks, where the nonstationary and multimodal nature of the data pose significant challenges and severely affect the performance of DL models. In this brief, a simple, yet effective, neural layer that is capable of adaptively normalizing the input time series, while taking into account the distribution of the data, is proposed. The proposed layer is trained in an end-to-end fashion using backpropagation and leads to significant performance improvements compared to other evaluated normalization schemes. The proposed method differs from traditional normalization methods since it learns how to perform normalization for a given task instead of using a fixed normalization scheme. At the same time, it can be directly applied to any new time series without requiring retraining. The effectiveness of the proposed method is demonstrated using a large-scale limit order book data set, as well as a load forecasting data set.

A Review of Computer-Aided Heart Sound Detection Techniques

Cardiovascular diseases have become one of the most prevalent threats to human health throughout the world. As a noninvasive assistant diagnostic tool, the heart sound detection techniques play an important role in the prediction of cardiovascular diseases. In this paper, the latest development of the computer-aided heart sound detection techniques over the last five years has been reviewed. There are mainly the following aspects: the theories of heart sounds and the relationship between heart sounds and cardiovascular diseases; the key technologies used in the processing and analysis of heart sound signals, including denoising, segmentation, feature extraction and classification; with emphasis, the applications of deep learning algorithm in heart sound processing. In the end, some areas for future research in computer-aided heart sound detection techniques are explored, hoping to provide reference to the prediction of cardiovascular diseases.

Signal Pattern Recognition Based on Fractal Features and Machine Learning

As a typical pattern recognition method, communication signal modulation involves many complicated factors. Fractal theory can be used for signal modulation feature extraction and recognition because of its good ability to express complex information. In this paper, we conduct a systematic research study by using the fractal dimension as the feature of modulation signals. Box fractal dimension, Katz fractal dimension, Higuchi fractal dimension, Petrosian fractal dimension, and Sevcik fractal dimension are extracted from eight different modulation signals for signal pattern recognition. Meanwhile, the anti-noise function, box-diagram, and running time are used to evaluate the noise robustness, separability, and computational complexity of five different fractal features. Finally, Bback-Propagation (BP) neural network, grey relation analysis, random forest, and K-nearest neighbor are proposed to classify the different modulation signals based on these fractal features. The confusion matrices and recognition results are provided in the experimental section. They indicate that random forest had a better recognition performance, which could reach 96% in 10 dB.

Heart Rate Estimated from Body Movements at Six Degrees of Freedom by Convolutional Neural Networks

Cardiac activity has been monitored continuously in daily life by virtue of advanced medical instruments with microelectromechanical system (MEMS) technology. Seismocardiography (SCG) has been considered to be free from the burden of measurement for cardiac activity, but it has been limited in its application in daily life. The most important issues regarding SCG are to overcome the limitations of motion artifacts due to the sensitivity of motion sensor. Although novel adaptive filters for noise cancellation have been developed, they depend on the researcher’s subjective decision. Convolutional neural networks (CNNs) can extract significant features from data automatically without a researcher’s subjective decision, so that signal processing has been recently replaced as CNNs. Thus, this study aimed to develop a novel method to enhance heart rate estimation from thoracic movement by CNNs. Thoracic movement was measured by six-axis accelerometer and gyroscope signals using a wearable sensor that can be worn by simply clipping on clothes. The dataset was collected from 30 participants (15 males, 15 females) using 12 measurement conditions according to two physical conditions (i.e., relaxed and aroused conditions), three body postures (i.e., sitting, standing, and supine), and six movement speeds (i.e., 3.2, 4.5, 5.8, 6.4, 8.5, and 10.3 km/h). The motion data (i.e., six-axis accelerometer and gyroscope) and heart rate (i.e., electrocardiogram (ECG)) were determined as the input data and labels in the dataset, respectively. The CNN model was developed based on VGG Net and optimized by testing according to network depth and data augmentation. The ensemble network of the VGG-16 without data augmentation and the VGG-19 with data augmentation was determined as optimal architecture for generalization. As a result, the proposed method showed higher accuracy than the previous SCG method using signal processing in most measurement conditions. The three main contributions are as follows: (1) the CNN model enhanced heart rate estimation with the benefits of automatic feature extraction from the data; (2) the proposed method was compared with the previous SCG method using signal processing; (3) the method was tested in 12 measurement conditions related to daily motion for a more practical application.