Human Activity Sensing with Wireless Signals: A Survey

Novel statistical time series data augmentation and machine learning based classification of unobtrusive respiration data for respiration Digital Twin model

Digital Twin (DT), a concept of Healthcare (4.0), represents the subject's biological properties and characteristics in a digital model. DT can help in monitoring respiratory failures, enabling timely interventions, personalized treatment plans to improve healthcare, and decision-support for healthcare professionals. Large-scale implementation of DT technology requires extensive patient data for accurate monitoring and decision-making with Machine Learning (ML) and Deep Learning (DL). Initial respiration data was collected unobtrusively with the ESP32 Wi-Fi Channel State Information (CSI) sensor. Due to limited respiration data availability, the paper proposes a novel statistical time series data augmentation method for generating larger synthetic respiration data. To ensure accuracy and validity in the augmentation method, correlation methods (Pearson, Spearman, and Kendall) are implemented to provide a comparative analysis of experimental and synthetic datasets. Data processing methodologies of denoising (smoothing and filtering) and dimensionality reduction with Principal Component Analysis (PCA) are implemented to estimate a patient's Breaths Per Minute (BPM) from raw respiration sensor data and the synthetic version. The methodology provided the BPM estimation accuracy of 92.3% from raw respiration data. It was observed that out of 27 supervised classifications with k-fold cross-validation, the Bagged Tree ensemble algorithm provided the best ML-supervised classification. In the case of binary-class and multi-class, the Bagged Tree ensemble showed accuracies of 89.2% and 83.7% respectively with combined real and synthetic respiration dataset with the larger synthetic dataset. Overall, this provides a blueprint of methodologies for the development of the respiration DT model.

SenseFi: A library and benchmark on deep-learning-empowered WiFi human sensing

Over the recent years, WiFi sensing has been rapidly developed for privacy-preserving, ubiquitous human-sensing applications, enabled by signal processing and deep-learning methods. However, a comprehensive public benchmark for deep learning in WiFi sensing, similar to that available for visual recognition, does not yet exist. In this article, we review recent progress in topics ranging from WiFi hardware platforms to sensing algorithms and propose a new library with a comprehensive benchmark, SenseFi. On this basis, we evaluate various deep-learning models in terms of distinct sensing tasks, WiFi platforms, recognition accuracy, model size, computational complexity, and feature transferability. Extensive experiments are performed whose results provide valuable insights into model design, learning strategy, and training techniques for real-world applications. In summary, SenseFi is a comprehensive benchmark with an open-source library for deep learning in WiFi sensing research that offers researchers a convenient tool to validate learning-based WiFi-sensing methods on multiple datasets and platforms.

Stretchable Woven Fabric-Based Triboelectric Nanogenerator for Energy Harvesting and Self-Powered Sensing

With the triboelectric nanogenerator developing in recent years, it has gradually become a promising alternative to fossil energy and batteries. Its rapid advancements also promote the combination of triboelectric nanogenerators and textiles. However, the limited stretchability of fabric-based triboelectric nanogenerators hindered their development in wearable electronic devices. Here, in combination with the polyamide (PA) conductive yarn, polyester multifilament, and polyurethane yarn, a highly stretchable woven fabric-based triboelectric nanogenerator (SWF-TENG) with the three elementary weaves is developed. Different from the normal woven fabric without elasticity, the loom tension of the elastic warp yarn is much larger than non-elastic warp yarn in the weaving process, which results in the high elasticity of the woven fabric coming from the loom. Based on the unique and creative woven method, SWF-TENGs are qualified with excellent stretchability (up to 300%), flexibility, comfortability, and excellent mechanical stability. It also exhibits good sensitivity and fast responsibility to the external tensile strain, which can be used as a bend-stretch sensor to detect and identify human gait. Its collected power under pressure mode is capable of lighting up 34 light-emitting diodes (LEDs) by only hand-tapping the fabric. SWF-TENG can be mass-manufactured by using the weaving machine, which decreases fabricating costs and accelerates industrialization. Based on these merits, this work provides a promising direction toward stretchable fabric-based TENGs with wide applications in wearable electronics, including energy harvesting and self-powered sensing.

HIT HAR: Human Image Threshing Machine for Human Activity Recognition Using Deep Learning Models

In recent days, research in human activity recognition (HAR) has played a significant role in healthcare systems. The accurate activity classification results from the HAR enhance the performance of the healthcare system with broad applications. HAR results are useful in monitoring a person's health, and the system predicts abnormal activities based on user movements. The HAR system's abnormal activity predictions provide better healthcare monitoring and reduce users' health issues. The conventional HAR systems use wearable sensors, such as inertial measurement unit (IMU) and stretch sensors for activity recognition. These approaches show remarkable performances to the user's basic activities such as sitting, standing, and walking. However, when the user performs complex activities, such as running, jumping, and lying, the sensor-based HAR systems have a higher degree of misclassification results due to the reading errors from sensors. These sensor errors reduce the overall performance of the HAR system with the worst classification results. Similarly, radiofrequency or vision-based HAR systems are not free from classification errors when used in real time. In this paper, we address some of the existing challenges of HAR systems by proposing a human image threshing (HIT) machine-based HAR system that uses an image dataset from a smartphone camera for activity recognition. The HIT machine effectively uses a mask region-based convolutional neural network (R-CNN) for human body detection, a facial image threshing machine (FIT) for image cropping and resizing, and a deep learning model for activity classification. We demonstrated the effectiveness of our proposed HIT machine-based HAR system through extensive experiments and results. The proposed HIT machine achieved 98.53% accuracy when the ResNet architecture was used as its deep learning model.

Device-Free Multi-Location Human Activity Recognition Using Deep Complex Network

Wi-Fi-based human activity recognition has attracted broad attention for its advantages, which include being device-free, privacy-protected, unaffected by light, etc. Owing to the development of artificial intelligence techniques, existing methods have made great improvements in sensing accuracy. However, the performance of multi-location recognition is still a challenging issue. According to the principle of wireless sensing, wireless signals that characterize activity are also seriously affected by location variations. Existing solutions depend on adequate data samples at different locations, which are labor-intensive. To solve the above concerns, we present an amplitude- and phase-enhanced deep complex network (AP-DCN)-based multi-location human activity recognition method, which can fully utilize the amplitude and phase information simultaneously so as to mine more abundant information from limited data samples. Furthermore, considering the unbalanced sample number at different locations, we propose a perception method based on the deep complex network-transfer learning (DCN-TL) structure, which effectively realizes knowledge sharing among various locations. To fully evaluate the performance of the proposed method, comprehensive experiments have been carried out with a dataset collected in an office environment with 24 locations and five activities. The experimental results illustrate that the approaches can achieve 96.85% and 94.02% recognition accuracy, respectively.

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors

This paper presents a comprehensive dataset intended to evaluate passive Human Activity Recognition (HAR) and localization techniques with measurements obtained from synchronized Radio-Frequency (RF) devices and vision-based sensors. The dataset consists of RF data including Channel State Information (CSI) extracted from a WiFi Network Interface Card (NIC), Passive WiFi Radar (PWR) built upon a Software Defined Radio (SDR) platform, and Ultra-Wideband (UWB) signals acquired via commercial off-the-shelf hardware. It also consists of vision/Infra-red based data acquired from Kinect sensors. Approximately 8 hours of annotated measurements are provided, which are collected across two rooms from 6 participants performing 6 daily activities. This dataset can be exploited to advance WiFi and vision-based HAR, for example, using pattern recognition, skeletal representation, deep learning algorithms or other novel approaches to accurately recognize human activities. Furthermore, it can potentially be used to passively track a human in an indoor environment. Such datasets are key tools required for the development of new algorithms and methods in the context of smart homes, elderly care, and surveillance applications.

A Perspective on Passive Human Sensing with Bluetooth

Passive human sensing approaches based on the analysis of the radio signals emitted by the most common wireless communication technologies have been steadily gaining momentum during the last decade. In this context, the Bluetooth technology, despite its widespread adoption in mobile and IoT applications, so far has not received all the attention it deserves. However, the introduction of the Bluetooth direction finding feature and the application of Artificial Intelligence techniques to the processing and analysis of the wireless signal for passive human sensing pave the way for novel Bluetooth-based passive human sensing applications, which will leverage Bluetooth Low Energy features, such as low power consumption, noise resilience, wide diffusion, and relatively low deployment cost. This paper provides a reasoned analysis of the data preprocessing and classification techniques proposed in the literature on Bluetooth-based remote passive human sensing, which is supported by a comparison of the reported accuracy results. Building on such results, the paper also identifies and discusses the multiple factors and operating conditions that explain the different accuracy values achieved by the considered techniques, and it draws the main research directions for the near future.

Machine learning empowered COVID-19 patient monitoring using non-contact sensing: An extensive review

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which caused the coronavirus disease 2019 (COVID-19) pandemic, has affected more than 400 million people worldwide. With the recent rise of new Delta and Omicron variants, the efficacy of the vaccines has become an important question. The goal of various studies has been to limit the spread of the virus by utilizing wireless sensing technologies to prevent human-to-human interactions, particularly for healthcare workers. In this paper, we discuss the current literature on invasive/contact and non-invasive/non-contact technologies (including Wi-Fi, radar, and software-defined radio) that have been effectively used to detect, diagnose, and monitor human activities and COVID-19 related symptoms, such as irregular respiration. In addition, we focused on cutting-edge machine learning algorithms (such as generative adversarial networks, random forest, multilayer perceptron, support vector machine, extremely randomized trees, and k-nearest neighbors) and their essential role in intelligent healthcare systems. Furthermore, this study highlights the limitations related to non-invasive techniques and prospective research directions.

•

This article describes cutting-edge technology (invasive/non-invasive) and its role in the recognition of COVID-19 symptoms.

•

This article summarizes state-of-art machine-learning algorithms and their roles in modern healthcare systems.

•

This article presents the challenges associated with wireless sensing techniques and potential future research directions.

Device-Free Human Identification Using Behavior Signatures in WiFi Sensing

Wireless sensing can be used for human identification by mining and quantifying individual behavior effects on wireless signal propagation. This work proposes a novel device-free biometric (DFB) system, WirelessID, that explores the joint human fine-grained behavior and body physical signatures embedded in channel state information (CSI) by extracting spatiotemporal features. In addition, the signal fluctuations corresponding to different parts of the body contribute differently to the identification performance. Inspired by the success of the attention mechanism in computer vision (CV), thus, to extract more robust features, we introduce the spatiotemporal attention function into our system. To evaluate the performance, commercial WiFi devices are used for prototyping WirelessID in a real laboratory environment with an average accuracy of 93.14% and a best accuracy of 97.72% for five individuals.

Recognizing and Counting Freehand Exercises Using Ubiquitous Cellular Signals

Freehand exercises help improve physical fitness without any requirements for devices or places. Existing fitness assistant systems are typically restricted to wearable devices or exercising at specific positions, compromising the ubiquitous availability of freehand exercises. In this paper, we develop MobiFit, a contactless freehand exercise assistant using just one cellular signal receiver placed on the ground. MobiFit passively monitors the ubiquitous cellular signals sent by the base station, which frees users from the space constraints and deployment overheads and provides accurate repetition counting, exercise type recognition and workout quality assessment without any attachments to the human body. The design of MobiFit faces new challenges of the uncertainties not only on cellular signal payloads but also on signal propagations because the sender (base station) is beyond the control of MobiFit and located far away. To tackle these challenges, we conducted experimental studies to observe the received cellular signal sequence during freehand exercises. Based on the observations, we constructed the analytic model of the received signals. Guided by the insights derived from the analytic model, MobiFit segments out every repetition and rest interval from one exercise session through spectrogram analysis and extracts low-frequency features from each repetition for type recognition. Extensive experiments were conducted in both indoor and outdoor environments, which collected 22,960 exercise repetitions performed by ten volunteers over six months. The results confirm that MobiFit achieves high counting accuracy of 98.6%, high recognition accuracy of 94.1% and low repetition duration estimation error within 0.3 s. Besides, the experiments show that MobiFit works both indoors and outdoors and supports multiple users exercising together.

Wi-Fi-Based Location-Independent Human Activity Recognition via Meta Learning

Wi-Fi-based device-free human activity recognition has recently become a vital underpinning for various emerging applications, ranging from the Internet of Things (IoT) to Human–Computer Interaction (HCI). Although this technology has been successfully demonstrated for location-dependent sensing, it relies on sufficient data samples for large-scale sensing, which is enormously labor-intensive and time-consuming. However, in real-world applications, location-independent sensing is crucial and indispensable. Therefore, how to alleviate adverse effects on recognition accuracy caused by location variations with the limited dataset is still an open question. To address this concern, we present a location-independent human activity recognition system based on Wi-Fi named WiLiMetaSensing. Specifically, we first leverage a Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) feature representation method to focus on location-independent characteristics. Then, in order to well transfer the model across different positions with limited data samples, a metric learning-based activity recognition method is proposed. Consequently, not only the generalization ability but also the transferable capability of the model would be significantly promoted. To fully validate the feasibility of the presented approach, extensive experiments have been conducted in an office with 24 testing locations. The evaluation results demonstrate that our method can achieve more than 90% in location-independent human activity recognition accuracy. More importantly, it can adapt well to the data samples with a small number of subcarriers and a low sampling rate.

A Systematic Review of Non-Contact Sensing for Developing a Platform to Contain COVID-19

The rapid spread of the novel coronavirus disease, COVID-19, and its resulting situation has garnered much effort to contain the virus through scientific research. The tragedy has not yet fully run its course, but it is already clear that the crisis is thoroughly global, and science is at the forefront in the fight against the virus. This includes medical professionals trying to cure the sick at risk to their own health; public health management tracking the virus and guardedly calling on such measures as social distancing to curb its spread; and researchers now engaged in the development of diagnostics, monitoring methods, treatments and vaccines. Recent advances in non-contact sensing to improve health care is the motivation of this study in order to contribute to the containment of the COVID-19 outbreak. The objective of this study is to articulate an innovative solution for early diagnosis of COVID-19 symptoms such as abnormal breathing rate, coughing and other vital health problems. To obtain an effective and feasible solution from existing platforms, this study identifies the existing methods used for human activity and health monitoring in a non-contact manner. This systematic review presents the data collection technology, data preprocessing, data preparation, features extraction, classification algorithms and performance achieved by the various non-contact sensing platforms. This study proposes a non-contact sensing platform for the early diagnosis of COVID-19 symptoms and monitoring of the human activities and health during the isolation or quarantine period. Finally, we highlight challenges in developing non-contact sensing platforms to effectively control the COVID-19 situation.