A Hierarchical Learning Approach for Human Action Recognition

A Low-Resolution Infrared Array for Unobtrusive Human Activity Recognition That Preserves Privacy

This research uses a low-resolution infrared array sensor to address real-time human activity recognition while prioritizing the preservation of privacy. The proposed system captures thermal pixels that are represented as a human silhouette. With camera and image processing, it is easy to detect human activity, but that reduces privacy. This work proposes a novel human activity recognition system that uses interpolation and mathematical measures that are unobtrusive and do not involve machine learning. The proposed method directly and efficiently recognizes multiple human states in a real-time environment. This work also demonstrates the accuracy of the outcomes for various scenarios using traditional ML approaches. This low-resolution IR array sensor is effective and would be useful for activity recognition in homes and healthcare centers.

Clinical human activity recognition based on a wearable patch of combined tri-axial ACC and ECG sensors

Background

In digital medicine, human activity recognition (HAR) can be used to track and assess a patient's progress throughout rehabilitation, enhancing the quality of life for the elderly and the disabled.

Methods

A patch-type flexible sensor that integrated dynamic electrocardiogram (ECG) and acceleration signal (ACC) was used to record the signals of the various behavioral activities of 20 healthy volunteers and 25 patients with pneumoconiosis. Seven HAR tasks were then carried out on the data using four different deep learning methods (CNN, LSTM, CNN-LSTM and GRU).

Results

When ECG and ACC were obtained simultaneously, the overall accuracy rates of HAR for healthy group were 0.9371, 0.8829, 0.9843 and 0.9486 by the CNN, LSTM, CNN-LSTM and GRU models, respectively. In contrast, the overall accuracy rates of HAR for the pneumoconiosis patients' group were 0.8850, 0.7975, 0.9425 and 0.8525 by the four corresponding models. The accuracy of HAR for both groups using all four models is higher than when only ACC signal is detected.

Conclusion

The addition of the ECG signal significantly improves HAR outcomes in the group of healthy individuals, while having relatively less enhancing effects on the group of patients with pneumoconiosis. When ECG and ACC signals were combined, the increase in HAR accuracy was notable compared to cases where no ECG data was provided. These results suggest that the combination of ACC and ECG data can represent a novel method for the clinical application of HAR.

ConMLP: MLP-Based Self-Supervised Contrastive Learning for Skeleton Data Analysis and Action Recognition

Human action recognition has drawn significant attention because of its importance in computer vision-based applications. Action recognition based on skeleton sequences has rapidly advanced in the last decade. Conventional deep learning-based approaches are based on extracting skeleton sequences through convolutional operations. Most of these architectures are implemented by learning spatial and temporal features through multiple streams. These studies have enlightened the action recognition endeavor from various algorithmic angles. However, three common issues are observed: (1) The models are usually complicated; therefore, they have a correspondingly higher computational complexity. (2) For supervised learning models, the reliance on labels during training is always a drawback. (3) Implementing large models is not beneficial to real-time applications. To address the above issues, in this paper, we propose a multi-layer perceptron (MLP)-based self-supervised learning framework with a contrastive learning loss function (ConMLP). ConMLP does not require a massive computational setup; it can effectively reduce the consumption of computational resources. Compared with supervised learning frameworks, ConMLP is friendly to the huge amount of unlabeled training data. In addition, it has low requirements for system configuration and is more conducive to being embedded in real-world applications. Extensive experiments show that ConMLP achieves the top one inference result of 96.9% on the NTU RGB+D dataset. This accuracy is higher than the state-of-the-art self-supervised learning method. Meanwhile, ConMLP is also evaluated in a supervised learning manner, which has achieved comparable performance to the state of the art of recognition accuracy.

A Deep Learning and Clustering Extraction Mechanism for Recognizing the Actions of Athletes in Sports

In sports, the essence of a complete technical action is a complete information structure pattern and the athlete's judgment of the action is actually the identification of the movement information structure pattern. Action recognition refers to the ability of the human brain to distinguish a perceived action from other actions and obtain predictive response information when it identifies and confirms it according to the constantly changing motion information on the field. Action recognition mainly includes two aspects: one is to obtain the required action information based on visual observation and the other is to judge the action based on the obtained action information, but the neuropsychological mechanism of this process is still unknown. In this paper, a new key frame extraction method based on the clustering algorithm and multifeature fusion is proposed for sports videos with complex content, many scenes, and rich actions. First, a variety of features are fused, and then, similarity measurement can be used to describe videos with complex content more completely and comprehensively; second, a clustering algorithm is used to cluster sports video sequences according to scenes, eliminating the need for shots in the case of many scenes. It is difficult and complicated to detect segmentation; third, extracting key frames according to the minimum motion standard can more accurately represent the video content with rich actions. At the same time, the clustering algorithm used in this paper is improved to enhance the offline computing efficiency of the key frame extraction system. Based on the analysis of the advantages and disadvantages of the classical convolutional neural network and recurrent neural network algorithms in deep learning, this paper proposes an improved convolutional network and optimization based on the recognition and analysis of human actions under complex scenes, complex actions, and fast motion compared to post-neural network and hybrid neural network algorithm. Experiments show that the algorithm achieves similar human observation of athletes' training execution and completion. Compared with other algorithms, it has been verified that it has very high learning rate and accuracy for the athlete's action recognition.

Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to L2-regularized Collaborative Representation Classifier (L2-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of L2-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.