Capture, learning, and classification of upper extremity movement primitives in healthy controls and stroke patients

A deep learning wearable-based solution for continuous at-home monitoring of upper limb goal-directed movements

Introduction

Monitoring upper limb function is crucial for tracking progress, assessing treatment effectiveness, and identifying potential problems or complications. Hand goal-directed movements (GDMs) are a crucial aspect of daily life, reflecting planned motor commands with hand trajectories towards specific target locations. Previous studies have shown that GDM tasks can detect early changes in upper limb function in neurodegenerative diseases and can be used to track disease progression over time.

Methods

In this study, we used accelerometer data from stroke survivor participants and controls doing activities of daily living to develop an automated deep learning approach to detect GDMs. The model performance for detecting GDM or non-GDM from windowed data achieved an AUC of 0.9, accuracy 0.83, sensitivity 0.81, specificity 0.84 and F1 0.82.

Results

We further validated the utility of detecting GDM by extracting features from GDM periods and using these features to classify whether the measurements are collected from a stroke survivor or a control participant, and to predict the Fugl-Meyer assessment score from stroke survivors.

Discussion

This study presents a promising and reliable tool for monitoring upper limb function in a real-world setting, and assessing biomarkers related to upper limb health in neurological, neuromuscular and muscles disorders.

Hybrid Learning Models for IMU-Based HAR with Feature Analysis and Data Correction

This paper proposes a novel approach to tackle the human activity recognition (HAR) problem. Four classes of body movement datasets, namely stand-up, sit-down, run, and walk, are applied to perform HAR. Instead of using vision-based solutions, we address the HAR challenge by implementing a real-time HAR system architecture with a wearable inertial measurement unit (IMU) sensor, which aims to achieve networked sensing and data sampling of human activity, data pre-processing and feature analysis, data generation and correction, and activity classification using hybrid learning models. Referring to the experimental results, the proposed system selects the pre-trained eXtreme Gradient Boosting (XGBoost) model and the Convolutional Variational Autoencoder (CVAE) model as the classifier and generator, respectively, with 96.03% classification accuracy.

A unified scheme for the benchmarking of upper limb functions in neurological disorders

BACKGROUND

In neurorehabilitation, we are witnessing a growing awareness of the importance of standardized quantitative assessment of limb functions. Detailed assessments of the sensorimotor deficits following neurological disorders are crucial. So far, this assessment has relied mainly on clinical scales, which showed several drawbacks. Different technologies could provide more objective and repeatable measurements. However, the current literature lacks practical guidelines for this purpose. Nowadays, the integration of available metrics, protocols, and algorithms into one harmonized benchmarking ecosystem for clinical and research practice is necessary.

METHODS

This work presents a benchmarking framework for upper limb capacity. The scheme resulted from a multidisciplinary and iterative discussion among several partners with previous experience in benchmarking methodology, robotics, and clinical neurorehabilitation. We merged previous knowledge in benchmarking methodologies for human locomotion and direct clinical and engineering experience in upper limb rehabilitation. The scheme was designed to enable an instrumented evaluation of arm capacity and to assess the effectiveness of rehabilitative interventions with high reproducibility and resolution. It includes four elements: (1) a taxonomy for motor skills and abilities, (2) a list of performance indicators, (3) a list of required sensor modalities, and (4) a set of reproducible experimental protocols.

RESULTS

We proposed six motor primitives as building blocks of most upper-limb daily-life activities and combined them into a set of functional motor skills. We identified the main aspects to be considered during clinical evaluation, and grouped them into ten motor abilities categories. For each ability, we proposed a set of performance indicators to quantify the proposed ability on a quantitative and high-resolution scale. Finally, we defined the procedures to be followed to perform the benchmarking assessment in a reproducible and reliable way, including the definition of the kinematic models and the target muscles.

CONCLUSIONS

This work represents the first unified scheme for the benchmarking of upper limb capacity. To reach a consensus, this scheme should be validated with real experiments across clinical conditions and motor skills. This validation phase is expected to create a shared database of human performance, necessary to have realistic comparisons of treatments and drive the development of new personalized technologies.

IMU-Based Monitoring for Assistive Diagnosis and Management of IoHT: A Review

With the rapid development of Internet of Things (IoT) technologies, traditional disease diagnoses carried out in medical institutions can now be performed remotely at home or even ambient environments, yielding the concept of the Internet of Health Things (IoHT). Among the diverse IoHT applications, inertial measurement unit (IMU)-based systems play a significant role in the detection of diseases in many fields, such as neurological, musculoskeletal, and mental. However, traditional numerical interpretation methods have proven to be challenging to provide satisfying detection accuracies owing to the low quality of raw data, especially under strong electromagnetic interference (EMI). To address this issue, in recent years, machine learning (ML)-based techniques have been proposed to smartly map IMU-captured data on disease detection and progress. After a decade of development, the combination of IMUs and ML algorithms for assistive disease diagnosis has become a hot topic, with an increasing number of studies reported yearly. A systematic search was conducted in four databases covering the aforementioned topic for articles published in the past six years. Eighty-one articles were included and discussed concerning two aspects: different ML techniques and application scenarios. This review yielded the conclusion that, with the help of ML technology, IMUs can serve as a crucial element in disease diagnosis, severity assessment, characteristic estimation, and monitoring during the rehabilitation process. Furthermore, it summarizes the state-of-the-art, analyzes challenges, and provides foreseeable future trends for developing IMU-ML systems for IoHT.

A Residual Movement Classification Based User Interface for Control of Assistive Devices by Persons with Complete Tetraplegia

OBJECTIVE

Complete tetraplegia can deprive a person of hand function. Assistive technologies may improve autonomy but needs for ergonomic interfaces for the user to pilot these devices still persist. Despite the paralysis of their arms, people with tetraplegia may retain residual shoulder movements. In this work we explored these movements as a mean to control assistive devices.

METHODS

We captured shoulder movement with a single inertial sensor and, by training a support vector machine based classifier, we decode such information into user intent.

RESULTS

The setup and training process take only a few minutes and so the classifiers can be user specific. We tested the algorithm with 10 able body and 2 spinal cord injury participants. The average classification accuracy was 80% and 84%, respectively.

CONCLUSION

The proposed algorithm is easy to set up, its operation is fully automated, and achieved results are on par with state-of-the-art systems.

SIGNIFICANCE

Assistive devices for persons without hand function present limitations in their user interfaces. Our work present a novel method to overcome some of these limitations by classifying user movement and decoding it into user intent, all with simple setup and training and no need for manual tuning. We demonstrate its feasibility with experiments with end users, including persons with complete tetraplegia without hand function.

Automatic Identification of Upper Extremity Rehabilitation Exercise Type and Dose Using Body-Worn Sensors and Machine Learning: A Pilot Study

BACKGROUND

Prior studies suggest that participation in rehabilitation exercises improves motor function poststroke; however, studies on optimal exercise dose and timing have been limited by the technical challenge of quantifying exercise activities over multiple days.

OBJECTIVES

The objectives of this study were to assess the feasibility of using body-worn sensors to track rehabilitation exercises in the inpatient setting and investigate which recording parameters and data analysis strategies are sufficient for accurately identifying and counting exercise repetitions.

METHODS

MC10 BioStampRC® sensors were used to measure accelerometer and gyroscope data from upper extremities of healthy controls (n = 13) and individuals with upper extremity weakness due to recent stroke (n = 13) while the subjects performed 3 preselected arm exercises. Sensor data were then labeled by exercise type and this labeled data set was used to train a machine learning classification algorithm for identifying exercise type. The machine learning algorithm and a peak-finding algorithm were used to count exercise repetitions in non-labeled data sets.

RESULTS

We achieved a repetition counting accuracy of 95.6% overall, and 95.0% in patients with upper extremity weakness due to stroke when using both accelerometer and gyroscope data. Accuracy was decreased when using fewer sensors or using accelerometer data alone.

CONCLUSIONS

Our exploratory study suggests that body-worn sensor systems are technically feasible, well tolerated in subjects with recent stroke, and may ultimately be useful for developing a system to measure total exercise "dose" in poststroke patients during clinical rehabilitation or clinical trials.

Towards data-driven stroke rehabilitation via wearable sensors and deep learning

Recovery after stroke is often incomplete, but rehabilitation training may potentiate recovery by engaging endogenous neuroplasticity. In preclinical models of stroke, high doses of rehabilitation training are required to restore functional movement to the affected limbs of animals. In humans, however, the necessary dose of training to potentiate recovery is not known. This ignorance stems from the lack of objective, pragmatic approaches for measuring training doses in rehabilitation activities. Here, to develop a measurement approach, we took the critical first step of automatically identifying functional primitives, the basic building block of activities. Forty-eight individuals with chronic stroke performed a variety of rehabilitation activities while wearing inertial measurement units (IMUs) to capture upper body motion. Primitives were identified by human labelers, who labeled and segmented the associated IMU data. We performed automatic classification of these primitives using machine learning. We designed a convolutional neural network model that outperformed existing methods. The model includes an initial module to compute separate embeddings of different physical quantities in the sensor data. In addition, it replaces batch normalization (which performs normalization based on statistics computed from the training data) with instance normalization (which uses statistics computed from the test data). This increases robustness to possible distributional shifts when applying the method to new patients. With this approach, we attained an average classification accuracy of 70%. Thus, using a combination of IMU-based motion capture and deep learning, we were able to identify primitives automatically. This approach builds towards objectively-measured rehabilitation training, enabling the identification and counting of functional primitives that accrues to a training dose.

The Pragmatic Classification of Upper Extremity Motion in Neurological Patients: A Primer

Recent advances in wearable sensor technology and machine learning (ML) have allowed for the seamless and objective study of human motion in clinical applications, including Parkinson's disease, and stroke. Using ML to identify salient patterns in sensor data has the potential for widespread application in neurological disorders, so understanding how to develop this approach for one's area of inquiry is vital. We previously proposed an approach that combined wearable inertial measurement units (IMUs) and ML to classify motions made by stroke patients. However, our approach had computational and practical limitations. We address these limitations here in the form of a primer, presenting how to optimize a sensor-ML approach for clinical implementation. First, we demonstrate how to identify the ML algorithm that maximizes classification performance and pragmatic implementation. Second, we demonstrate how to identify the motion capture approach that maximizes classification performance but reduces cost. We used previously collected motion data from chronic stroke patients wearing off-the-shelf IMUs during a rehabilitation-like activity. To identify the optimal ML algorithm, we compared the classification performance, computational complexity, and tuning requirements of four off-the-shelf algorithms. To identify the optimal motion capture approach, we compared the classification performance of various sensor configurations (number and location on the body) and sensor type (IMUs vs. accelerometers). Of the algorithms tested, linear discriminant analysis had the highest classification performance, low computational complexity, and modest tuning requirements. Of the sensor configurations tested, seven sensors on the paretic arm and trunk led to the highest classification performance, and IMUs outperformed accelerometers. Overall, we present a refined sensor-ML approach that maximizes both classification performance and pragmatic implementation. In addition, with this primer, we showcase important considerations for appraising off-the-shelf algorithms and sensors for quantitative motion assessment.

A Taxonomy of Functional Upper Extremity Motion

Background: Functional upper extremity (UE) motion enables humans to execute activities of daily living (ADLs). There currently exists no universal language to systematically characterize this type of motion or its fundamental building blocks, called functional primitives. Without a standardized classification approach, pooling mechanistic knowledge and unpacking rehabilitation content will remain challenging.

Methods: We created a taxonomy to characterize functional UE motions occurring during ADLs, classifying them by motion presence, temporal cyclicity, upper body effector, and contact type. We identified five functional primitives by their phenotype and purpose: reach, reposition, transport, stabilize, and idle. The taxonomy was assessed for its validity and interrater reliability in right-paretic chronic stroke patients performing a selection of ADL tasks. We applied the taxonomy to identify the primitive content and motion characteristics of these tasks, and to evaluate the influence of impairment level on these outcomes.

Results: The taxonomy could account for all motions in the sampled activities. Interrater reliability was high for primitive identification (Cohen's kappa = 0.95–0.99). Using the taxonomy, the ADL tasks were found to be composed primarily of transport and stabilize primitives mainly executed with discrete, proximal motions. Compared to mildly impaired patients, moderately impaired patients used more repeated reaches and axial-proximal UE motion to execute the tasks.

Conclusions: The proposed taxonomy yields objective, quantitative data on human functional UE motion. This new method could facilitate the decomposition and quantification of UE rehabilitation, the characterization of functional abnormality after stroke, and the mechanistic examination of shared behavior in motor studies.

Motion Analysis for People with Cerebral Palsy: A Vision Based Approach

We propose a methodology to classify motion of subjects with cerebral palsy based on RGB image sequences and present a new dataset with 2D facial landmark trajectories from RGB images of people with and without disabilities while performing specific types of movements. Depending on these movements, parts of the face can be occluded and we are able to recover the 3D face's shape and its motion based on the Structure from Motion framework. Using the 3D structure and the motion, we propose two different motion descriptors, one is focused on describing the spatial distribution of the motion and the other on the temporal distribution. Finally, we discuss the physical meaning of these descriptors and show that they are very informative about the degree of the subjects' disabilities. Our descriptor can classify people with and without cerebral palsy from 2D image sequences.

Research and Development of Rehabilitation Systems for the Upper Limbs “PLEMO” Series

The human brain has excellent plasticity in terms of recovery from a stroke. It is expected that neurorehabilitation using an apparatus that applies robotic and virtual reality technology has rehabilitation training effects on stroke patients. Rehabilitation of the upper limbs consists of rehabilitation for shoulders and elbows, and rehabilitation of the fingers and wrists. Both rehabilitations are necessary to improve activities of daily living. Many rehabilitation systems for shoulders and elbows have been effective thus far. In this review, we introduce rehabilitation systems for the shoulders, elbows, and wrists of the upper limbs “PLEMO” series using force presentation technology with brakes. Rehabilitation systems using brakes are essentially safe unlike rehabilitation systems using actuators.

Repetitive Transcranial Magnetic Stimulation for Upper Extremity Motor Recovery: Does It Help?

PURPOSE OF REVIEW

Repetitive transcranial magnetic stimulation (rTMS) noninvasively modulates brain excitability in humans and influences mediators of plasticity in animals. When applied in humans in the months to years after stroke, potentiation of motor recovery has been limited. Recently, investigators have shifted rTMS administration into the early weeks following stroke, when injury-induced plasticity could be maximally engaged. This article provides an overview of basic mechanisms of rTMS, consideration of its interaction with various forms of neuroplasticity, and a summary of the highest quality clinical evidence for rTMS given early after stroke.

RECENT FINDINGS

Studies of repetitive magnetic stimulation in vitro and in vivo have found modulation of excitatory and inhibitory neurotransmission and induction of cellular mechanisms supporting plasticity. A handful of clinical studies have shown sustained improvements in grip strength and UE motor impairment when rTMS is delivered in the first weeks after stroke. Though in its infancy, recent research suggests a plasticity-enhancing influence and modest motor recovery potentiation when rTMS is delivered early after stroke.