Automated assessment of upper extremity movement impairment due to stroke

Fusing CNNs and attention-mechanisms to improve real-time indoor Human Activity Recognition for classifying home-based physical rehabilitation exercises

Physical rehabilitation plays a critical role in enhancing health outcomes globally. However, the shortage of physiotherapists, particularly in developing countries where the ratio is approximately ten physiotherapists per million people, poses a significant challenge to effective rehabilitation services. The existing literature on rehabilitation often falls short in data representation and the employment of diverse modalities, limiting the potential for advanced therapeutic interventions. To address this gap, This study integrates Computer Vision and Human Activity Recognition (HAR) technologies to support home-based rehabilitation. The study mitigates this gap by exploring various modalities and proposing a framework for data representation. We introduce a novel framework that leverages both Continuous Wavelet Transform (CWT) and Mel-Frequency Cepstral Coefficients (MFCC) for skeletal data representation. CWT is particularly valuable for capturing the time-frequency characteristics of dynamic movements involved in rehabilitation exercises, enabling a comprehensive depiction of both temporal and spectral features. This dual capability is crucial for accurately modelling the complex and variable nature of rehabilitation exercises. In our analysis, we evaluate 20 CNN-based models and one Vision Transformer (ViT) model. Additionally, we propose 12 hybrid architectures that combine CNN-based models with ViT in bi-model and tri-model configurations. These models are rigorously tested on the UI-PRMD and KIMORE benchmark datasets using key evaluation metrics, including accuracy, precision, recall, and F1-score, with 5-fold cross-validation. Our evaluation also considers real-time performance, model size, and efficiency on low-power devices, emphasising practical applicability. The proposed fused tri-model architectures outperform both single-architectures and bi-model configurations, demonstrating robust performance across both datasets and making the fused models the preferred choice for rehabilitation tasks. Our proposed hybrid model, DenMobVit, consistently surpasses state-of-the-art methods, achieving accuracy improvements of 2.9% and 1.97% on the UI-PRMD and KIMORE datasets, respectively. These findings highlight the effectiveness of our approach in advancing rehabilitation technologies and bridging the gap in physiotherapy services.

Evaluating Joint Angle Data for Clinical Assessment Using Multidimensional Inverse Kinematics with Average Segment Morphometry

Movement analysis is a critical tool in understanding and addressing various disabilities associated with movement deficits. By analyzing movement patterns, healthcare professionals can identify the root causes of these alterations, which is essential for preventing, diagnosing, and rehabilitating a broad spectrum of medical conditions, disabilities, and injuries. With the advent of affordable motion capture technologies, quantitative data on patient movement is more accessible to clinicians, enhancing the quality of care. Nonetheless, it is crucial that these technologies undergo rigorous validation to ensure their accuracy in collecting and monitoring patient movements, particularly for remote healthcare services where direct patient observation is not possible. In this study, motion capture technology was used to track upper extremity movements during a reaching task presented in virtual reality. Kinematic data was then calculated for each participant using a scaled dynamic inertial model. The goal was to evaluate the accuracy of joint angle calculations using inverse kinematics from motion capture relative to the typical movement redundancy. Shoulder, elbow, radioulnar, and wrist joint angles were calculated with models scaled using either direct measurements of each individual's arm segment lengths or those lengths were calculated from individual height using published average proportions. The errors in joint angle trajectories calculated using the two methods of model scaling were compared to the inter-trial variability of those trajectories. The variance of this error was primarily within the normal range of variability between repetitions of the same movements. This suggests that arm joint angles can be inferred with good enough accuracy from motion capture data and individual height to be useful for the clinical assessment of motor deficits.

Neural Networks-Based Approach to Solve Inverse Kinematics Problems for Medical Applications

GOAL

Motion capture is used for recording complex human movements that is increasingly applied in medicine. We describe a novel algorithm of combining a machine learning approach with biomechanics to enable fast and robust analysis of motion capture data to obtain joint angles.

METHODS

A multilayer perceptron and a recurrent neural network were compared in their capacity to estimate the joint angles of the human arm. The networks were pre-trained using data from a kinematic model of the human arm. The data comprised movements of three degrees of freedom, such as wrist flexion/extension, wrist ulnar/radial deviation, and hand pronation/supination.

RESULTS

A recurrent neural network model with long short-term memory architecture can solve the inverse kinematics problem for three rotational degrees of freedom with the least error; it performed faster than real time. The predictions were robust against noise.

CONCLUSIONS

This shows that it is feasible to rely on pre-trained neural networks for real-time calculation of joint angles.

Effects of wearing an upper extremity exoskeleton on measuring joint kinematics during standardized clinical assessment tasks

Evaluating upper extremity (UE) motor impairment post-stroke commonly relies on established clinical assessments, which suffer from inherent subjectivity due to human visual inspection. The sensing capabilities of robotics can facilitate the objective assessment of motor impairment. The robotic device needs to not only precisely measure movement characteristics but also minimally interfere with natural movement to assess motor impairment effectively. However, the effect of wearing an exoskeleton on the performance of motor impairment assessment tasks has yet to be evaluated. Thus, this study aimed to evaluate whether the joint kinematics recorded during Fugl-Meyer Assessment (FMA) tasks performance are comparable between two conditions: 1) while wearing the exoskeleton and 2) without wearing the exoskeleton. Six healthy participants performed six single-degree-of-freedom sub-tasks of the upper extremity subscale of the Fugl-Meyer Assessment (FMA-UE), one of the standardized clinical assessments. We estimated joint angle trajectories in both conditions using the exoskeleton and a motion capture system, respectively. The coefficient of multiple correlation (CMC) was used to evaluate the similarity of joint kinematic trajectories between the two conditions. The range of motion (RoM) between the two conditions was also compared. The calculated CMC indicated a good-to-excellent level of agreement across all tasks between the wearing and non-wearing conditions (CMC > 0.90). The RoMs of all tasks in the two conditions except for shoulder flexion to 180° were not significantly different (p > 0.05). These results revealed the minimal effect of wearing the exoskeleton on joint kinematics during FMA subtask performance. In addition, these results imply that each exoskeleton segment was well-aligned and attached to the corresponding anatomical body segment within the exoskeleton measurement system. Overall, we conclude that the HARMONY exoskeleton can be a feasible measurement tool for clinical assessment tasks.

Impedance-based biomechanical method for robust inverse kinematics from noisy data

This paper presents a novel method for solving the inverse kinematic problem of capturing human reaching movements using a dynamic biomechanical model. The model consists of rigid segments connected by joints and actuated by markers. The method was validated against a rotation matrix-based method using motion capture data recorded during reaching movements performed by healthy human volunteers. The results showed that the proposed method achieved low errors in joint angles and compensated for noise in motion capture data. The angles were comparable to those calculated using the standard marker-based method. The proposed bioinspired method can be used in real-time medical applications for processing noisy marker data with occlusions.

Improved Disabilities of the Arm, Shoulder and Hand scores after myoelectric arm orthosis use at home in chronic stroke: A retrospective study

BACKGROUND

Most stroke survivors have persistent upper limb impairments after completing standard clinical care. The resulting impairments can adversely affect their quality of life and ability to complete self-care tasks and remain employed, leading to increased healthcare and societal costs. A myoelectric arm orthosis can be used effectively to support the affected weak arm and increase an individual's use of that arm.

OBJECTIVE

The study objective was to retrospectively evaluate the outcomes and clinical benefits provided by the MyoPro® orthosis in individuals 65 years and older with upper limb impairment secondary to a stroke.

METHODS

The Disabilities of the Arm, Shoulder and Hand (DASH) questionnaire was administered to individuals who have chronic stroke both before and after receiving their myoelectric orthosis. A Generalized Estimating Equation model was analyzed.

RESULTS

After using the MyoPro, 19 individuals with chronic stroke had a mean improvement (decrease) in DASH score of 18.07, 95% CI = (-25.41, -10.72), adjusted for 8 covariates. This large change in DASH score was statistically significant and clinically meaningful as participants self-reported an improvement with engagement in functional tasks.

CONCLUSIONS

Use of the MyoPro increases independence in functional tasks as reported by the validated DASH outcome measure for older participants with chronic stroke.

Design and Analysis of VARONE a Novel Passive Upper-Limb Exercising Device

Robots have been widely investigated for active and passive rehabilitation therapy of patients with upper limb disabilities. Nevertheless, the rehabilitation assessment process is often ignored or just qualitatively performed by the physiotherapist implementing chart-based ordinal scales or observation-based measures, which tend to rely on professional experience and lack quantitative analysis. In order to objectively quantify the upper limb rehabilitation progress, this paper presents a noVel pAssive wRist motiOn assessmeNt dEvice (VARONE) having three degrees of freedom (DoFs) based on the gimbal mechanical design. VARONE implements a mechanism of three revolute passive joints with controllable passive resistance. An inertial measurement unit (IMU) sensor is used to quantify the wrist orientation and position, and an encoder module is implemented to obtain the arm positions. The proposed VARONE device can also be used in combination with the previously designed two-DoFs device NURSE (cassiNo-qUeretaro uppeR limb aSsistive dEvice) to perform multiple concurrent assessments and rehabilitation tasks. Analyses and experimental tests have been carried out to demonstrate the engineering feasibility of the intended applications of VARONE. The maximum value registered for the IMU sensor is 36.8 degrees, the minimum value registered is −32.3 degrees, and the torque range registered is around −80 and 80 Nmm. The implemented models include kinematics, statics (F.E.M.), and dynamics. Thirty healthy patients participated in an experimental validation. The experimental tests were developed with different goal-defined exercising paths that the participant had to follow.

Setup for the Quantitative Assessment of Motion and Muscle Activity During a Virtual Modified Box and Block Test

The ability to move allows us to interact with the world. When this ability is impaired, it can significantly reduce one's quality of life and independence and may lead to complications. The importance of remote patient evaluation and rehabilitation has recently grown due to limited access to in-person services. For example, the COVID-19 pandemic unexpectedly resulted in strict regulations, reducing access to non-emergent healthcare services. Additionally, remote care offers an opportunity to address healthcare disparities in rural, underserved, and low-income areas where access to services remains limited. Improving accessibility through remote care options would limit the number of hospital or specialist visits and render routine care more affordable. Finally, the use of readily available commercial consumer electronics for at-home care can enhance patient outcomes due to improved quantitative observation of symptoms, treatment efficacy, and therapy dosage. While remote care is a promising means to address these issues, there is a crucial need to quantitatively characterize motor impairment for such applications. The following protocol seeks to address this knowledge gap to enable clinicians and researchers to obtain high-resolution data on complex movement and underlying muscle activity. The ultimate goal is to develop a protocol for remote administration of functional clinical tests. Here, participants were instructed to perform a medically-inspired Box and Block task (BBT), which is frequently used to assess hand function. This task requires subjects to transport standardized cubes between two compartments separated by a barrier. We implemented a modified BBT in virtual reality to demonstrate the potential of developing remote assessment protocols. Muscle activation was captured for each subject using surface electromyography. This protocol allowed for the acquisition of high-quality data to better characterize movement impairment in a detailed and quantitative manner. Ultimately, these data have the potential to be used to develop protocols for virtual rehabilitation and remote patient monitoring.

A deep learning system to monitor and assess rehabilitation exercises in home-based remote and unsupervised conditions

In the domain of physical rehabilitation, the progress in machine learning and the availability of cost-effective motion capture technologies have paved the way for innovative systems capable of capturing human movements, automatically analyzing recorded data, and evaluating movement quality. This study introduces a novel, economically viable system designed for monitoring and assessing rehabilitation exercises. The system enables real-time evaluation of exercises, providing precise insights into deviations from correct execution. The evaluation comprises two significant components: range of motion (ROM) classification and compensatory pattern recognition. To develop and validate the effectiveness of the system, a unique dataset of 6 resistance training exercises was acquired. The proposed system demonstrated impressive capabilities in motion monitoring and evaluation. Notably, we achieved promising results, with mean accuracies of 89% for evaluating ROM-class and 98% for classifying compensatory patterns. By complementing conventional rehabilitation assessments conducted by skilled clinicians, this cutting-edge system has the potential to significantly improve rehabilitation practices. Additionally, its integration in home-based rehabilitation programs can greatly enhance patient outcomes and increase access to high-quality care.

aBnormal motION capture In aCute Stroke (BIONICS): A Low-Cost Tele-Evaluation Tool for Automated Assessment of Upper Extremity Function in Stroke Patients

BACKGROUND

The incidence of stroke and stroke-related hemiparesis has been steadily increasing and is projected to become a serious social, financial, and physical burden on the aging population. Limited access to outpatient rehabilitation for these stroke survivors further deepens the healthcare issue and estranges the stroke patient demographic in rural areas. However, new advances in motion detection deep learning enable the use of handheld smartphone cameras for body tracking, offering unparalleled levels of accessibility.

METHODS

In this study we want to develop an automated method for evaluation of a shortened variant of the Fugl-Meyer assessment, the standard stroke rehabilitation scale describing upper extremity motor function. We pair this technology with a series of machine learning models, including different neural network structures and an eXtreme Gradient Boosting model, to score 16 of 33 (49%) Fugl-Meyer item activities.

RESULTS

In this observational study, 45 acute stroke patients completed at least 1 recorded Fugl-Meyer assessment for the training of the auto-scorers, which yielded average accuracies ranging from 78.1% to 82.7% item-wise.

CONCLUSION

In this study, an automated method was developed for the evaluation of a shortened variant of the Fugl-Meyer assessment, the standard stroke rehabilitation scale describing upper extremity motor function. This novel method is demonstrated with potential to conduct telehealth rehabilitation evaluations and assessments with accuracy and availability.

Impairments of the ipsilesional upper-extremity in the first 6-months post-stroke

BACKGROUND

Ipsilesional motor impairments of the arm are common after stroke. Previous studies have suggested that severity of contralesional arm impairment and/or hemisphere of lesion may predict the severity of ipsilesional arm impairments. Historically, these impairments have been assessed using clinical scales, which are less sensitive than robot-based measures of sensorimotor performance. Therefore, the objective of this study was to characterize progression of ipsilesional arm motor impairments using a robot-based assessment of motor function over the first 6-months post-stroke and quantify their relationship to (1) contralesional arm impairment severity and (2) stroke-lesioned hemisphere.

METHODS

A total of 106 participants with first-time, unilateral stroke completed a unilateral assessment of arm motor impairment (visually guided reaching task) using the Kinarm Exoskeleton. Participants completed the assessment along with a battery of clinical measures with both ipsilesional and contralesional arms at 1-, 6-, 12-, and 26-weeks post-stroke.

RESULTS

Robotic assessment of arm motor function revealed a higher incidence of ipsilesional arm impairment than clinical measures immediately post-stroke. The incidence of ipsilesional arm impairments decreased from 47 to 14% across the study period. Kolmogorov-Smirnov tests revealed that ipsilesional arm impairment severity, as measured by our task, was not related to which hemisphere was lesioned. The severity of ipsilesional arm impairments was variable but displayed moderate significant relationships to contralesional arm impairment severity with some robot-based parameters.

CONCLUSIONS

Ipsilesional arm impairments were variable. They displayed relationships of varying strength with contralesional impairments and were not well predicted by lesioned hemisphere. With standard clinical care, 86% of ipsilesional impairments recovered by 6-months post-stroke.

Automatic Post-Stroke Severity Assessment Using Novel Unsupervised Consensus Learning for Wearable and Camera-Based Sensor Datasets

Stroke survivors often suffer from movement impairments that significantly affect their daily activities. The advancements in sensor technology and IoT have provided opportunities to automate the assessment and rehabilitation process for stroke survivors. This paper aims to provide a smart post-stroke severity assessment using AI-driven models. With the absence of labelled data and expert assessment, there is a research gap in providing virtual assessment, especially for unlabeled data. Inspired by the advances in consensus learning, in this paper, we propose a consensus clustering algorithm, PSA-NMF, that combines various clusterings into one united clustering, i.e., cluster consensus, to produce more stable and robust results compared to individual clustering. This paper is the first to investigate severity level using unsupervised learning and trunk displacement features in the frequency domain for post-stroke smart assessment. Two different methods of data collection from the U-limb datasets-the camera-based method (Vicon) and wearable sensor-based technology (Xsens)-were used. The trunk displacement method labelled each cluster based on the compensatory movements that stroke survivors employed for their daily activities. The proposed method uses the position and acceleration data in the frequency domain. Experimental results have demonstrated that the proposed clustering method that uses the post-stroke assessment approach increased the evaluation metrics such as accuracy and F-score. These findings can lead to a more effective and automated stroke rehabilitation process that is suitable for clinical settings, thus improving the quality of life for stroke survivors.

Data-Driven Quantitation of Movement Abnormality after Stroke

Stroke commonly affects the ability of the upper extremities (UEs) to move normally. In clinical settings, identifying and measuring movement abnormality is challenging due to the imprecision and impracticality of available assessments. These challenges interfere with therapeutic tracking, communication, and treatment. We thus sought to develop an approach that blends precision and pragmatism, combining high-dimensional motion capture with out-of-distribution (OOD) detection. We used an array of wearable inertial measurement units to capture upper body motion in healthy and chronic stroke subjects performing a semi-structured, unconstrained 3D tabletop task. After data were labeled by human coders, we trained two deep learning models exclusively on healthy subject data to classify elemental movements (functional primitives). We tested these healthy subject-trained models on previously unseen healthy and stroke motion data. We found that model confidence, indexed by prediction probabilities, was generally high for healthy test data but significantly dropped when encountering OOD stroke data. Prediction probabilities worsened with more severe motor impairment categories and were directly correlated with individual impairment scores. Data inputs from the paretic UE, rather than trunk, most strongly influenced model confidence. We demonstrate for the first time that using OOD detection with high-dimensional motion data can reveal clinically meaningful movement abnormality in subjects with chronic stroke.

Quantitative Assessment of Upper Limb Movement in Post-Stroke Adults for Identification of Sensitive Measures in Reaching and Lifting Activities

BACKGROUND

The assumption of this work is the achievement of objective results of the movement structure, which forms the basis for in-depth analysis and, consequently, for determining the upper limb movements that are most affected by stroke compared to healthy people.

METHODS

An analysis of relevant and systematically identified features of upper limb movement in post-stroke adults is presented based on scalable hypothesis tests. The basic features were calculated using movements defined by the x, y, and z coordinates (i.e., 3D trajectory time series) and compared to the results of post-stroke patients with healthy controls of similar age.

RESULTS

After automatic feature selection, out of the 1004 common features of upper limb movement, the most differentiated were the upper arm movements in reaching kinematics. In terms of movement type, movements in the frontal plane (shoulder abduction and adduction) were the most sensitive to changes. The largest number of discriminating features was determined on the basis of acceleration time series.

CONCLUSIONS

In the 3D assessment of functional activities of the upper limb, the upper arm turned out to be the most differentiated body segment, especially during abduction and adduction movements. The results indicate a special need to pay attention to abduction and adduction movements to improve the activities of daily living of the upper limbs after a stroke.

A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation

BACKGROUND

Markerless motion capture (MMC) technology has been developed to avoid the need for body marker placement during motion tracking and analysis of human movement. Although researchers have long proposed the use of MMC technology in clinical measurement-identification and measurement of movement kinematics in a clinical population, its actual application is still in its preliminary stages. The benefits of MMC technology are also inconclusive with regard to its use in assessing patients' conditions. In this review we put a minor focus on the method's engineering components and sought primarily to determine the current application of MMC as a clinical measurement tool in rehabilitation.

METHODS

A systematic computerized literature search was conducted in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. The search keywords used in each database were "Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess." Only peer-reviewed articles that applied MMC technology for clinical measurement were included. The last search took place on March 6, 2023. Details regarding the application of MMC technology for different types of patients and body parts, as well as the assessment results, were summarized.

RESULTS

A total of 65 studies were included. The MMC systems used for measurement were most frequently used to identify symptoms or to detect differences in movement patterns between disease populations and their healthy counterparts. Patients with Parkinson's disease (PD) who demonstrated obvious and well-defined physical signs were the largest patient group to which MMC assessment had been applied. Microsoft Kinect was the most frequently used MMC system, although there was a recent trend of motion analysis using video captured with a smartphone camera.

CONCLUSIONS

This review explored the current uses of MMC technology for clinical measurement. MMC technology has the potential to be used as an assessment tool as well as to assist in the detection and identification of symptoms, which might further contribute to the use of an artificial intelligence method for early screening for diseases. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in the disease populations.

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions

The analysis of functional upper extremity (UE) movement kinematics has implications across domains such as rehabilitation and evaluating job-related skills. Using movement kinematics to quantify movement quality and skill is a promising area of research but is currently not being used widely due to issues associated with cost and the need for further methodological validation. Recent developments by computationally-oriented research communities have resulted in potentially useful methods for evaluating UE function that may make kinematic analyses easier to perform, generally more accessible, and provide more objective information about movement quality, the importance of which has been highlighted during the COVID-19 pandemic. This narrative review provides an interdisciplinary perspective on the current state of computer-assisted methods for analyzing UE kinematics with a specific focus on how to make kinematic analyses more accessible to domain experts. We find that a variety of methods exist to more easily measure and segment functional UE movement, with a subset of those methods being validated for specific applications. Future directions include developing more robust methods for measurement and segmentation, validating these methods in conjunction with proposed kinematic outcome measures, and studying how to integrate kinematic analyses into domain expert workflows in a way that improves outcomes.

Feasibility of using the Leap Motion Controller to administer conventional motor tests: a proof-of-concept study

Although upper-limb movement impairments are common, the primary tools for assessing and tracking impairments in clinical settings are limited. Markerless motion capture (MMC) technology has the potential to provide a large amount of quantitative, objective movement data in routine clinical use. Many past studies have focused on whether MMC are sufficiently accurate. However, another necessary step is to create meaningful clinical tests that can be administered via MMC in a robust manner. Four conventional upper-limb motor tests common in clinical assessments (visually guided movement, finger tapping, postural tremor, and reaction time) were modified so they can be administered via a particular MMC sensor, the Leap Motion Controller (LMC). In this proof-of-concept study, we administered these modified tests to 100 healthy subjects and present here the successes and challenges we encountered. Subjects generally found the LMC and the graphical user interfaces of the tests easy to use. The LMC recorded movement with sufficiently high sampling rate (>106 samples/s), and the rate of LMC malfunctions (mainly jumps in time or space) was low, so only 1.9% of data was discarded. However, administration of the tests also revealed some significant weaknesses. The visually guided movement test was easily implemented with the LMC; the modified reaction time test worked reasonably well with the LMC but is likely more easily implemented with other existing technologies; and the modified tremor and finger tapping tests did not work well because of the limited bandwidth of the LMC. Our findings highlight the need to develop and evaluate motor tests specifically suited for MMC. The real strength of MMC may not be in replicating conventional tests but rather in administering new tests or testing conditions not possible with conventional clinical tests or other technologies.

Baseline robot-measured kinematic metrics predict discharge rehabilitation outcomes in individuals with subacute stroke

Background: The literature on upper limb robot-assisted therapy showed that robot-measured metrics can simultaneously predict registered clinical outcomes. However, only a limited number of studies correlated pre-treatment kinematics with discharge motor recovery. Given the importance of predicting rehabilitation outcomes for optimizing physical therapy, a predictive model for motor recovery that incorporates multidirectional indicators of a patient's upper limb abilities is needed. Objective: The aim of this study was to develop a predictive model for rehabilitation outcome at discharge (i.e., muscle strength assessed by the Motricity Index of the affected upper limb) based on multidirectional 2D robot-measured kinematics. Methods: Re-analysis of data from 66 subjects with subacute stroke who underwent upper limb robot-assisted therapy with an end-effector robot was performed. Two least squares error multiple linear regression models for outcome prediction were developed and differ in terms of validation procedure: the Split Sample Validation (SSV) model and the Leave-One-Out Cross-Validation (LOOCV) model. In both models, the outputs were the discharge Motricity Index of the affected upper limb and its sub-items assessing elbow flexion and shoulder abduction, while the inputs were the admission robot-measured metrics. Results: The extracted robot-measured features explained the 54% and 71% of the variance in clinical scores at discharge in the SSV and LOOCV validation procedures respectively. Normalized errors ranged from 22% to 35% in the SSV models and from 20% to 24% in the LOOCV models. In all models, the movement path error of the trajectories characterized by elbow flexion and shoulder extension was the significant predictor, and all correlations were significant. Conclusion: This study highlights that motor patterns assessed with multidirectional 2D robot-measured metrics are able to predict clinical evalutation of upper limb muscle strength and may be useful for clinicians to assess, manage, and program a more specific and appropriate rehabilitation in subacute stroke patients.

A Novel Automated RGB-D Sensor-Based Measurement of Voluntary Items of the Fugl-Meyer Assessment for Upper Extremity: A Feasibility Study

Motor function assessment is essential for post-stroke rehabilitation, while the requirement for professional therapists’ participation in current clinical assessment limits its availability to most patients. By means of sensors that collect the motion data and algorithms that conduct assessment based on such data, an automated system can be built to optimize the assessment process, benefiting both patients and therapists. To this end, this paper proposed an automated Fugl-Meyer Assessment (FMA) upper extremity system covering all 30 voluntary items of the scale. RGBD sensors, together with force sensing resistor sensors were used to collect the patients’ motion information. Meanwhile, both machine learning and rule-based logic classification were jointly employed for assessment scoring. Clinical validation on 20 hemiparetic stroke patients suggests that this system is able to generate reliable FMA scores. There is an extremely high correlation coefficient (r = 0.981, p < 0.01) with that yielded by an experienced therapist. This study offers guidance and feasible solutions to a complete and independent automated assessment system.

A low-cost virtual coach for 2D video-based compensation assessment of upper extremity rehabilitation exercises

Background

The increasing demands concerning stroke rehabilitation and in-home exercise promotion grew the need for affordable and accessible assistive systems to promote patients’ compliance in therapy. These assistive systems require quantitative methods to assess patients’ quality of movement and provide feedback on their performance. However, state-of-the-art quantitative assessment approaches require expensive motion-capture devices, which might be a barrier to the development of low-cost systems.

Methods

In this work, we develop a low-cost virtual coach (VC) that requires only a laptop with a webcam to monitor three upper extremity rehabilitation exercises and provide real-time visual and audio feedback on compensatory motion patterns exclusively from image 2D positional data analysis. To assess compensation patterns quantitatively, we propose a Rule-based (RB) and a Neural Network (NN) based approaches. Using the dataset of 15 post-stroke patients, we evaluated these methods with Leave-One-Subject-Out (LOSO) and Leave-One-Exercise-Out (LOEO) cross-validation and the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 score that measures the accuracy (geometric mean of precision and recall) of a model to assess compensation motions. In addition, we conducted a pilot study with seven volunteers to evaluate system performance and usability.

Results

For exercise 1, the RB approach assessed four compensation patterns with a \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 score of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$76.69 \%$$\end{document}76.69%. For exercises 2 and 3, the NN-based approach achieved a \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 score of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$72.56 \%$$\end{document}72.56% and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$79.87 \%$$\end{document}79.87%, respectively. Concerning the user study, they found that the system is enjoyable (hedonic value of 4.54/5) and relevant (utilitarian value of 4.86/5) for rehabilitation administration. Additionally, volunteers’ enjoyment and interest (Hedonic value perception) were correlated with their perceived VC performance (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho = 0.53$$\end{document}ρ=0.53).

Conclusions

The VC performs analysis on 2D videos from a built-in webcam of a laptop and accurately identifies compensatory movement patterns to provide corrective feedback. In addition, we discuss some findings concerning system performance and usability.

Depth Estimation for Egocentric Rehabilitation Monitoring Using Deep Learning Algorithms

Upper limb impairment is one of the most common problems for people with neurological disabilities, affecting their activity, quality of life (QOL), and independence. Objective assessment of upper limb performance is a promising way to help patients with neurological upper limb disorders. By using wearable sensors, such as an egocentric camera, it is possible to monitor and objectively assess patients’ actual performance in activities of daily life (ADLs). We analyzed the possibility of using Deep Learning models for depth estimation based on a single RGB image to allow the monitoring of patients with 2D (RGB) cameras. We conducted experiments placing objects at different distances from the camera and varying the lighting conditions to evaluate the performance of the depth estimation provided by two deep learning models (MiDaS & Alhashim). Finally, we integrated the best performing model for depth-estimation (MiDaS) with other Deep Learning models for hand (MediaPipe) and object detection (YOLO) and evaluated the system in a task of hand-object interaction. Our tests showed that our final system has a 78% performance in detecting interactions, while the reference performance using a 3D (depth) camera is 84%.

Using Wearable Inertial Sensors to Estimate Clinical Scores of Upper Limb Movement Quality in Stroke

Neurorehabilitation is progressively shifting from purely in-clinic treatment to therapy that is provided in both clinical and home-based settings. This transition generates a pressing need for assessments that can be performed across the entire continuum of care, a need that might be accommodated by application of wearable sensors. A first step toward ubiquitous assessments is to augment validated and well-understood standard clinical tests. This route has been pursued for the assessment of motor functioning, which in clinical research and practice is observation-based and requires specially trained personnel. In our study, 21 patients performed movement tasks of the Action Research Arm Test (ARAT), one of the most widely used clinical tests of upper limb motor functioning, while trained evaluators scored each task on pre-defined criteria. We collected data with just two wrist-worn inertial sensors to guarantee applicability across the continuum of care and used machine learning algorithms to estimate the ARAT task scores from sensor-derived features. Tasks scores were classified with approximately 80% accuracy. Linear regression between summed clinical task scores (across all tasks per patient) and estimates of sum task scores yielded a good fit (R 2 = 0.93; range reported in previous studies: 0.61-0.97). Estimates of the sum scores showed a mean absolute error of 2.9 points, 5.1% of the total score, which is smaller than the minimally detectable change and minimally clinically important difference of the ARAT when rated by a trained evaluator. We conclude that it is feasible to obtain accurate estimates of ARAT scores with just two wrist worn sensors. The approach enables administration of the ARAT in an objective, minimally supervised or remote fashion and provides the basis for a widespread use of wearable sensors in neurorehabilitation.

Kinect v2-Assisted Semi-Automated Method to Assess Upper Limb Motor Performance in Children

The interruption of rehabilitation activities caused by the COVID-19 lockdown has significant health negative consequences for the population with physical disabilities. Thus, measuring the range of motion (ROM) using remotely taken photographs, which are then sent to specialists for formal assessment, has been recommended. Currently, low-cost Kinect motion capture sensors with a natural user interface are the most feasible implementations for upper limb motion analysis. An active range of motion (AROM) measuring system based on a Kinect v2 sensor for upper limb motion analysis using Fugl-Meyer Assessment (FMA) scoring is described in this paper. Two test groups of children, each having eighteen participants, were analyzed in the experimental stage, where upper limbs’ AROM and motor performance were assessed using FMA. Participants in the control group (mean age of 7.83 ± 2.54 years) had no cognitive impairment or upper limb musculoskeletal problems. The study test group comprised children aged 8.28 ± 2.32 years with spastic hemiparesis. A total of 30 samples of elbow flexion and 30 samples of shoulder abduction of both limbs for each participant were analyzed using the Kinect v2 sensor at 30 Hz. In both upper limbs, no significant differences (p < 0.05) in the measured angles and FMA assessments were observed between those obtained using the described Kinect v2-based system and those obtained directly using a universal goniometer. The measurement error achieved by the proposed system was less than ±1° compared to the specialist’s measurements. According to the obtained results, the developed measuring system is a good alternative and an effective tool for FMA assessment of AROM and motor performance of upper limbs, while avoiding direct contact in both healthy children and children with spastic hemiparesis.

Data-Driven Classification of Human Movements in Virtual Reality-Based Serious Games: Preclinical Rehabilitation Study in Citizen Science

BACKGROUND

Sustained engagement is essential for the success of telerehabilitation programs. However, patients' lack of motivation and adherence could undermine these goals. To overcome this challenge, physical exercises have often been gamified. Building on the advantages of serious games, we propose a citizen science-based approach in which patients perform scientific tasks by using interactive interfaces and help advance scientific causes of their choice. This approach capitalizes on human intellect and benevolence while promoting learning. To further enhance engagement, we propose performing citizen science activities in immersive media, such as virtual reality (VR).

OBJECTIVE

This study aims to present a novel methodology to facilitate the remote identification and classification of human movements for the automatic assessment of motor performance in telerehabilitation. The data-driven approach is presented in the context of a citizen science software dedicated to bimanual training in VR. Specifically, users interact with the interface and make contributions to an environmental citizen science project while moving both arms in concert.

METHODS

In all, 9 healthy individuals interacted with the citizen science software by using a commercial VR gaming device. The software included a calibration phase to evaluate the users' range of motion along the 3 anatomical planes of motion and to adapt the sensitivity of the software's response to their movements. During calibration, the time series of the users' movements were recorded by the sensors embedded in the device. We performed principal component analysis to identify salient features of movements and then applied a bagged trees ensemble classifier to classify the movements.

RESULTS

The classification achieved high performance, reaching 99.9% accuracy. Among the movements, elbow flexion was the most accurately classified movement (99.2%), and horizontal shoulder abduction to the right side of the body was the most misclassified movement (98.8%).

CONCLUSIONS

Coordinated bimanual movements in VR can be classified with high accuracy. Our findings lay the foundation for the development of motion analysis algorithms in VR-mediated telerehabilitation.

Quantitative Evaluation System of Upper Limb Motor Function of Stroke Patients Based on Desktop Rehabilitation Robot

Rehabilitation training and movement evaluation after stroke have become a research hotspot as stroke has become a very common and harmful disease. However, traditional rehabilitation training and evaluation are mainly conducted under the guidance of rehabilitation doctors. The evaluation process is time-consuming and the evaluation results are greatly influenced by doctors. In this study, a desktop upper limb rehabilitation robot was designed and a quantitative evaluation system of upper limb motor function for stroke patients was proposed. The kinematics and dynamics data of stroke patients during active training were collected by sensors. Combined with the scores of patients' upper limb motor function by rehabilitation doctors using the Wolf Motor Function Test (WMFT) scale, three different quantitative evaluation models of upper limb motor function based on Back Propagation Neural Network (BPNN), K-Nearest Neighbors (KNN), and Support Vector Regression (SVR) algorithms were established. To verify the effectiveness of the quantitative evaluation system, 10 healthy subjects and 21 stroke patients were recruited for experiments. The experimental results show that the BPNN model has the best evaluation performance among the three quantitative evaluation models. The scoring accuracy of the BPNN model reached up to 87.1%. Moreover, there was a significant correlation between the models' scores and the doctors' scores. The proposed system can help doctors to quantitatively evaluate the upper limb motor function of stroke patients and accurately master the rehabilitation progress of patients.

Motor Ability Evaluation of the Upper Extremity with Point-To-Point Training Movement Based on End-Effector Robot-Assisted Training System

Assessment is critical during the procedure of stroke rehabilitation. However, traditional assessment methods are time-consuming, laborious, and dependent on the skillfulness of the therapist. Moreover, they cannot distinguish whether the improvement comes from the abnormal compensation or the improvement of upper extremity motor function. To make up for the shortcomings of the traditional methods, this study proposes a novel assessment system, which consisted of a rehabilitation robot and motion capture (MoCAP) system. A 9-degree-of-freedom (DOF) kinematic model is established, which consists of the shoulder girdle, shoulder, elbow, and wrist joints. And seven assessment indices are selected for this assessment system, including a range of motion (ROM), shoulder girdle compensation (SGC), trunk compensation (TC), aiming angle (AA), motion error (ME), motion length ratio (MLR), and useful force (UF). For AA, ME, and MLR, all describe the motor ability of the upper extremity, and a linear model was proposed to map these three indices into one index, called motor control ability (MCA). Then, this system can quantitatively evaluate human upper extremity motor function from joint space kinematics, Cartesian space kinematics, and dynamics. Three healthy participants were invited to verify the effectiveness of this system. The preliminary results show that all participants' handedness performs a little better than the nonhandedness. And the performance of the participants and the change of all the upper limb joints can be directly watched from the trajectory of the hand and joint angles' curve. Therefore, this assessment system can evaluate the human upper limb motor function well. Future studies are planned to recruit elderly volunteers or stroke patients to further verify the effectiveness of this system.

Design of a Data Glove for Assessment of Hand Performance Using Supervised Machine Learning

The large number of poststroke recovery patients poses a burden on rehabilitation centers, hospitals, and physiotherapists. The advent of rehabilitation robotics and automated assessment systems can ease this burden by assisting in the rehabilitation of patients with a high level of recovery. This assistance will enable medical professionals to either better provide for patients with severe injuries or treat more patients. It also translates into financial assistance as well in the long run. This paper demonstrated an automated assessment system for in-home rehabilitation utilizing a data glove, a mobile application, and machine learning algorithms. The system can be used by poststroke patients with a high level of recovery to assess their performance. Furthermore, this assessment can be sent to a medical professional for supervision. Additionally, a comparison between two machine learning classifiers was performed on their assessment of physical exercises. The proposed system has an accuracy of 85% (±5.1%) with careful feature and classifier selection.

Proof-of-Concept of a Sensor-Based Evaluation Method for Better Sensitivity of Upper-Extremity Motor Function Assessment

In rehabilitation, the Fugl-Meyer assessment (FMA) is a typical clinical instrument to assess upper-extremity motor function of stroke patients, but it cannot measure fine changes of motor function (both in recovery and deterioration) due to its limited sensitivity. This paper introduces a sensor-based automated FMA system that addresses this limitation with a continuous rating algorithm. The system consists of a depth sensor (Kinect V2) and an algorithm to rate the continuous FM scale based on fuzzy inference. Using a binary logic based classification method developed from a linguistic scoring guideline of FMA, we designed fuzzy input/output variables, fuzzy rules, membership functions, and a defuzzification method for several representative FMA tests. A pilot trial with nine stroke patients was performed to test the feasibility of the proposed approach. The continuous FM scale from the proposed algorithm exhibited a high correlation with the clinician rated scores and the results showed the possibility of more sensitive upper-extremity motor function assessment.

Muscle torques and joint accelerations provide more sensitive measures of poststroke movement deficits than joint angles

The whole repertoire of complex human motion is enabled by forces applied by our muscles and controlled by the nervous system. The impact of stroke on the complex multijoint motor control is difficult to quantify in a meaningful way that informs about the underlying deficit in the active motor control and intersegmental coordination. We tested whether poststroke deficit can be quantified with high sensitivity using motion capture and inverse modeling of a broad range of reaching movements. Our hypothesis is that muscle moments estimated based on active joint torques provide a more sensitive measure of poststroke motor deficits than joint angles. The motion of 22 participants was captured while performing reaching movements in a center-out task, presented in virtual reality. We used inverse dynamic analysis to derive active joint torques that were the result of muscle contractions, termed muscle torques, that caused the recorded multijoint motion. We then applied a novel analysis to separate the component of muscle torque related to gravity compensation from that related to intersegmental dynamics. Our results show that muscle torques characterize individual reaching movements with higher information content than joint angles do. Moreover, muscle torques enable distinguishing the individual motor deficits caused by aging or stroke from the typical differences in reaching between healthy individuals. Similar results were obtained using metrics derived from joint accelerations. This novel quantitative assessment method may be used in conjunction with home-based gaming motion capture technology for remote monitoring of motor deficits and inform the development of evidence-based robotic therapy interventions.NEW & NOTEWORTHY Functional deficits seen in task performance have biomechanical underpinnings, seen only through the analysis of forces. Our study has shown that estimating muscle moments can quantify with high-sensitivity poststroke deficits in intersegmental coordination. An assessment developed based on this method could help quantify less observable deficits in mildly affected stroke patients. It may also bridge the gap between evidence from studies of constrained or robotically manipulated movements and research with functional and unconstrained movements.

Quantification of Motor Function Post-Stroke Using Novel Combination of Wearable Inertial and Mechanomyographic Sensors

Subjective clinical rating scales represent the gold-standard for diagnosis of motor function following stroke. In practice however, they suffer from well-recognized limitations including assessor variance, low inter-rater reliability and low resolution. Automated systems have been proposed for empirical quantification but have not significantly impacted clinical practice. We address translational challenges in this arena through: (1) implementation of a novel sensor suite combining inertial measurement and mechanomyography (MMG) to quantify hand and wrist motor function; and (2) introduction of a new range of signal features extracted from the suite to supplement predicted clinical scores. The wearable sensors, signal features, and machine learning algorithms have been combined to produce classified ratings from the Fugl-Meyer clinical assessment rating scale. Furthermore, we have designed the system to augment clinical rating with several sensor-derived supplementary features encompassing critical aspects of motor dysfunction (e.g. joint angle, muscle activity, etc.). Performance is validated through a large-scale study on a post-stroke cohort of 64 patients. Fugl-Meyer Assessment tasks were classified with 75% accuracy for gross motor tasks and 62% for hand/wrist motor tasks. Of greater import, supplementary features demonstrated concurrent validity with Fugl-Meyer ratings, evidencing their utility as new measures of motor function suited to automated assessment. Finally, the supplementary features also provide continuous measures of sub-components of motor function, offering the potential to complement low accuracy but well-validated clinical rating scales when high-quality motor outcome measures are required. We believe this work provides a basis for widespread clinical adoption of inertial-MMG sensor use for post-stroke clinical motor assessment.

Exploring Movement Impairments in Patients With Parkinson's Disease Using the Microsoft Kinect Sensor: A Feasibility Study

Background: Current assessments of motor symptoms in Parkinson's disease are often limited to clinical rating scales. Objectives: To develop a computer application using the Microsoft Kinect sensor to assess performance-related bradykinesia. Methods: The developed application (Motorgame) was tested in patients with Parkinson's disease and healthy controls. Participants were assessed with the Movement Disorder Society Unified Parkinson's disease Rating Scale (MDS-UPDRS) and standardized clinical side effect rating scales, i.e., UKU Side Effect Rating Scale and Simpson-Angus Scale. Additionally, tests of information processing (Symbol Coding Task) and motor speed (Token Motor Task), together with a questionnaire, were applied. Results: Thirty patients with Parkinson's disease and 33 healthy controls were assessed. In the patient group, there was a statistically significant (p < 0.05) association between prolonged time of motor performance in the Motorgame and upper body rigidity and bradykinesia (MDS-UPDRS) with the strongest effects in the right hand (p < 0.001). In the entire group, prolonged time of motor performance was significantly associated with higher Simson-Angus scale rigidity score and higher UKU hypokinesia scores (p < 0.05). A shortened time of motor performance was significantly associated with higher scores on information processing (p < 0.05). Time of motor performance was not significantly associated with Token Motor Task, duration of illness, or hours of daily physical activity. The Motorgame was well-accepted. Conclusions: In the present feasibility study the Motorgame was able to detect common motor symptoms in Parkinson's disease in a statistically significant and clinically meaningful way, making it applicable for further testing in larger samples.

Equal Opportunities for Stroke Survivors' Rehabilitation: A Study on the Validity of the Upper Extremity Fugl-Meyer Assessment Scale Translated and Adapted into Romanian

Background and objectives: The Upper Extremity Fugl-Meyer Assessment (UEFMA) is one of the most recommended and used methods of clinical evaluation not only for post-stroke motor function disability conditions but also for physiotherapy goal-setting. Up to the present, an official Romanian version has not been officially available. This study aims to carry out a translation, adaptation, and validation of UEFMA in Romanian, thus giving both patients and medical practitioners the equal opportunity of benefiting from its proficiency. Material and methods: The English version of the motor component of UEFMA was back and forth translated in the assent of best practice translation guidelines. The research was performed on a group of 64 post-stroke in-patients regarding psychometric properties for content validation and an exploratory and confirmatory factorial analysis was performed using the Bayesian model. To assess internal consistency and test–retest reliability, we used the Cronbach Alpha index and Intraclass Correlation Coefficient (ICC). We used Pearson correlation with the Functional Independence Measure (FIM) and Modified Rankin Scale (MRS) to determine concurrent validation. Standardized response mean (SRM) was applied to determine the responsiveness of the instrument used. Results: After performing the exploratory factor analysis, a single factor was extracted, with an Eigenvalue of 19.363, which explained 64.543% of the variation. The model was confirmed by Bayesian exploration, with Root Mean Square Residual (RMR) 0.051, Goodness-of-fit Index (GFI) 0.980, Normed-Fit Index (NFI) 0.978 and Relative Fit Index (RFI) 0.977. The Cronbach Alpha value was 0.981, the Intraclass Correlation Coefficient (ICC) index for average measures was 0.992, the Pearson correlation with FIM 0.789, and MRS −0.787, while the SRM was 1.117. Conclusions: The Romanian version of the UEFMA scale is a reliable, responsive and valid tool which can be used as a standardized assessment in post-stroke patients across Romania.

Meaningful change: Defining the interpretability of changes in endpoints derived from interactive and mHealth technologies in healthcare and clinical research

Immersive, interactive and mHealth technologies are increasingly being used in clinical research, healthcare and rehabilitation solutions. Leveraging technology solutions to derive new and novel clinical outcome measures is important to the ongoing assessment of clinical interventions. While demonstrating statistically significant changes is an important element of intervention assessment, understanding whether changes detected reflect changes of a magnitude that are considered meaningful to patients is equally important. We describe methodologies used to determine meaningful change and recommend that these techniques are routinely included in the development and testing of clinical assessment and rehabilitation technology solutions.

Quantitative Assessment of Upper-Limb Motor Function for Post-Stroke Rehabilitation Based on Motor Synergy Analysis and Multi-Modality Fusion

Functional assessment is an essential part of rehabilitation protocols after stroke. Conventionally, the assessment process relies heavily on clinical experience and lacks quantitative analysis. In order to objectively quantify the upper-limb motor impairments in patients with post-stroke hemiparesis, this study proposes a novel assessment approach based on motor synergy quantification and multi-modality fusion. Fifteen post-stroke hemiparetic patients and fifteen age-matched healthy persons participated in this study. During different goal-directed tasks, kinematic data and surface electromyography(sEMG) signals were synchronously collected from these participants, and then motor features extracted from each modal data could be fed into the respective local classifiers. In addition, kinematic synergies and muscle synergies were quantified by principal component analysis (PCA) and k weighted angular similarity ( k WAS) algorithm to provide in-depth analysis of the coactivated features responsible for observable movement impairments. By integrating the outputs of local classifiers and the quantification results of motor synergies, ensemble classifiers can be created to generate quantitative assessment for different modalities separately. In order to further exploit the complementarity between the evaluation results at kinematic and muscular levels, a multi-modal fusion scheme was developed to comprehensively analyze the upper-limb motor function and generate a probability-based function score. Under the proposed assessment framework, three types of machine learning methods were employed to search the optimal performance of each classifier. Experimental results demonstrated that the classification accuracy was respectively improved by 4.86% and 2.78% when the analysis of kinematic and muscle synergies was embedded in the assessment system, and could be further enhanced to 96.06% by fusing the characteristics derived from different modalities. Furthermore, the assessment result of multi-modality fusion framework exhibited a significant correlation with the score of standard clinical tests ( R = - 0.87, P = 1.98e - 5 ). These promising results show the feasibility of applying the proposed method to clinical assessments for post-stroke hemiparetic patients.

Estimating Upper-Limb Impairment Level in Stroke Survivors Using Wearable Inertial Sensors and a Minimally-Burdensome Motor Task

Upper-limb paresis is the most common motor impairment post stroke. Current solutions to automate the assessment of upper-limb impairment impose a number of critical burdens on patients and their caregivers that preclude frequent assessment. In this work, we propose an approach to estimate upper-limb impairment in stroke survivors using two wearable inertial sensors, on the wrist and the sternum, and a minimally-burdensome motor task. Twenty-three stroke survivors with no, mild, or moderate upper-limb impairment performed two repetitions of one-to-two minute-long continuous, random (i.e., patternless), voluntary upper-limb movements spanning the entire range of motion. The three-dimensional time-series of upper-limb movements were segmented into a series of one-dimensional submovements by employing a unique movement decomposition technique. An unsupervised clustering algorithm and a supervised regression model were used to estimate Fugl-Meyer Assessment (FMA) scores based on features extracted from these submovements. Our regression model estimated FMA scores with a normalized root mean square error of 18.2% ( r²=0.70 ) and needed as little as one minute of movement data to yield reasonable estimation performance. These results support the possibility of frequently monitoring stroke survivors' rehabilitation outcomes, ultimately enabling the development of individually-tailored rehabilitation programs.

The Effects of Upper Limb Motor Recovery on Submovement Characteristics among the Patients with Stroke: A Meta-Analysis

OBJECTIVE

To evaluate the evidence related to the effect of upper limb motor recovery on submovement characteristics, including duration, amplitude, overlap, interpeak distance, and the number of submovements in stroke patients using a meta-analysis.

TYPE OF STUDY

Meta-analysis.

LITERATURE SURVEY

The literature search was restricted to articles written in English published from inception to October 2018 in Web of Science, PubMed, Science Direct, IEEE Explore, MEDLINE, CDSR, Scopus, Compendex, Wiley Online Library, Springer Link, and REHABDATA.

METHODOLOGY

Studies were included if they encompassed adult participants with a clinical diagnosis of stroke who underwent upper limb rehabilitation and if they assessed and reported submovement characteristics as the outcome measures in pre- and posttreatment stages. Changes in submovement characteristics between pre- and postinterventions were compared using the standardized mean difference (SMD). Finally, a test for heterogeneity and publication bias was implemented for all meta-analyses.

SYNTHESIS

Among the 188 retrieved articles, seven of them (one randomized controlled trial, six pre-post) involving 259 patients were selected for meta-analysis. Based on the results, the overall observed changes in all meta-analyses were statistically significant. In total, submovement amplitude (SMD 0.624, 95% confidence interval [CI] [0.356, 0.893]), duration (SMD 0.61, 95% CI [0.332, 0.888]), and overlap (SMD 0.928, 95% CI [0.768, 1.088]) increased whereas interpeak distance (SMD -0.278, 95% CI [-0.42, -0.137]), and the total number of submovements (SMD -0.804, 95% CI [-1.069, -0.538]) decreased.

CONCLUSIONS

The submovements appeared to become longer, fewer, and more overlapped with motor recovery. Based on the results, the ability of the neural system to blend submovements increased in both acute/subacute and chronic patients during recovery. Therefore, assessing the submovements during recovery can be a new quantitative measure of motor improvement, providing another means of comparing rehabilitation interventions and individualizing therapy for stroke patients.

Upper Limb Three-Dimensional Reachable Workspace Analysis Using the Kinect Sensor in Hemiplegic Stroke Patients: A Cross-Sectional Observational Study

OBJECTIVE

A reachable workspace evaluation using the Kinect sensor was previously introduced as a novel upper limb outcome measure in neuromuscular and musculoskeletal conditions. This study investigated its usefulness in hemiplegic stroke patients.

DESIGN

Forty-one patients with hemiplegic stroke were included. Kinect-based reachable workspace analysis was performed on both paretic and nonparetic sides. Upper limb impairment was measured using the Fugl-Meyer Assessment and the Motricity Index on the paretic side. Disability was assessed using the shortened Disabilities of the Arm, Shoulder, and Hand questionnaire. Correlations between the relative surface areas, impairment scores, and disability were analyzed.

RESULTS

Quadrants 1, 3, and 4 as well as the total relative surface area of the paretic side were significantly reduced compared with the nonparetic side. The total relative surface area of the paretic side correlated with the Fugl-Meyer Assessment scores, the Motricity Index for Upper Extremity, and the Disabilities of the Arm, Shoulder, and Hand questionnaire score. Furthermore, quadrant 3 was the most important determinant of upper limb impairment and disability.

CONCLUSIONS

A reachable workspace (a sensor-based measure that can be obtained relatively quickly and unobtrusively) could be a useful and alternative outcome measure for upper limb in hemiplegic stroke patients.

Cellphone-Based Automated Fugl-Meyer Assessment to Evaluate Upper Extremity Motor Function After Stroke

The Fugl-Meyer Assessment (FMA) is a widely used evaluation tool for assessing upper extremity motor function during stroke rehabilitation. However, the FMA is a repetitive, time-consuming task that currently must be performed by therapists in a hospital or clinic. We thus propose an alternative automated approach in which patients perform FMA movements while holding a cellphone at the hand and receive automated FMA scores. In the proposed system, features are extracted from cellphone movement data and decision trees are used to automatically score FMA test items. Ten stroke patients with upper extremity dysfunction participated in a validation experiment to compare automated FMA scores with traditional FMA scores from a trained therapist. Results showed that FMA scores from the cellphone-based automated system were highly correlated with FMA scores from the trained therapist (r² = 0.97), and that the average accuracy for individual FMA test items was 85%. These results demonstrate that such a portable, automated FMA system could potentially be used to assess upper extremity function during stroke rehabilitation to remove the repetitive, time-consuming burden from therapists and could potentially be performed in clinic or home-based settings.

Capturing Upper Limb Gross Motor Categories Using the Kinect® Sensor

IMPORTANCE

Along with growth in telerehabilitation, a concurrent need has arisen for standardized methods of tele-evaluation.

OBJECTIVE

To examine the feasibility of using the Kinect sensor in an objective, computerized clinical assessment of upper limb motor categories.

DESIGN

We developed a computerized Mallet classification using the Kinect sensor. Accuracy of computer scoring was assessed on the basis of reference scores determined collaboratively by multiple evaluators from reviewing video recording of movements. In addition, using the reference score, we assessed the accuracy of the typical clinical procedure in which scores were determined immediately on the basis of visual observation. The accuracy of the computer scores was compared with that of the typical clinical procedure.

SETTING

Research laboratory.

PARTICIPANTS

Seven patients with stroke and 10 healthy adult participants. Healthy participants intentionally achieved predetermined scores.

OUTCOMES AND MEASURES

Accuracy of the computer scores in comparison with accuracy of the typical clinical procedure (immediate visual assessment).

RESULTS

The computerized assessment placed participants' upper limb movements in motor categories as accurately as did typical clinical procedures.

CONCLUSIONS AND RELEVANCE

Computerized clinical assessment using the Kinect sensor promises to facilitate tele-evaluation and complement telehealth applications.

WHAT THIS ARTICLE ADDS

Computerized clinical assessment can enable patients to conduct evaluations remotely in their homes without therapists present.

Design and Analysis of Cloud Upper Limb Rehabilitation System Based on Motion Tracking for Post-Stroke Patients

In order to improve the convenience and practicability of home rehabilitation training for post-stroke patients, this paper presents a cloud-based upper limb rehabilitation system based on motion tracking. A 3-dimensional reachable workspace virtual game (3D-RWVG) was developed to achieve meaningful home rehabilitation training. Five movements were selected as the criteria for rehabilitation assessment. Analysis was undertaken of the upper limb performance parameters: relative surface area (RSA), mean velocity (MV), logarithm of dimensionless jerk (LJ) and logarithm of curvature (LC). A two-headed convolutional neural network (TCNN) model was established for the assessment. The experiment was carried out in the hospital. The results show that the RSA, MV, LC and LJ could reflect the upper limb motor function intuitively from the graphs. The accuracy of the TCNN models is 92.6%, 80%, 89.5%, 85.1% and 87.5%, respectively. A therapist could check patient training and assessment information through the cloud database and make a diagnosis. The system can realize home rehabilitation training and assessment without the supervision of a therapist, and has the potential to become an effective home rehabilitation system.

Design and Study of a Smart Cup for Monitoring the Arm and Hand Activity of Stroke Patients

This paper presents a new platform to monitor the arm and hand activity of stroke patients during rehabilitation exercises in the hospital and at home during their daily living activities. The platform provides relevant data to the therapist in order to assess the patients physical state and adapt the rehabilitation program if necessary. The platform consists of a self-contained smart cup that can be used to perform exercises that are similar to everyday tasks such as drinking. The first smart cup prototype, the design of which was based on interviews regarding the needs of therapists, contains various sensors that collect information about its orientation, the liquid level, its position compared to a reference target and tremors. The prototype also includes audio and visual displays that provide feedback to patients about their movements. Two studies were carried out in conjunction with healthcare professionals and patients. The first study focused on collecting feedback from healthcare professionals to assess the functionalities of the cup and to improve the prototype. Based on this paper, we designed an improved prototype and created a visualization tool for therapists. Finally, we carried out a preliminary study involving nine patients who had experienced an ischemic or hemorrhagic stroke in the previous 24 months. This preliminary study focused on assessing the usability and acceptability of the cup to the patients. The results showed that the cup was very well accepted by eight of the nine patients in monitoring their activity within a rehabilitation center or at home. Moreover, these eight patients had almost no concerns about the design of the cup and its usability.

Analysis and Quantification of Repetitive Motion in Long-Term Rehabilitation

Objective assessment in long-term rehabilitation under real-life recording conditions is a challenging task. We propose a data-driven method to evaluate changes in motor function under uncontrolled, long-term conditions with the low-cost Microsoft Kinect sensor. Instead of using human ratings as ground truth data, we propose kinematic features of hand motion, healthy reference trajectories derived by principal component regression, and methods taken from machine learning to analyze the progression of motor function. We demonstrate the capability of this approach on datasets with repetitive unrestrained bi-manual drumming movements in three-dimensional space of stroke survivors, patients suffering of Parkinson's disease, and a healthy control group. We present processing steps to eliminate the influence of varying recording setups under real-life conditions and offer visualization methods to support clinicians in the evaluation of treatment effects.

Technology-Enabled Assessment of Functional Health

The maturation of pervasive computing technologies has dramatically altered the face of healthcare. With the introduction of mobile devices, body area networks, and embedded computing systems, care providers can use continuous, ecologically valid information to overcome geographic and temporal barriers and thus provide more effective and timely health assessments. In this paper, we review recent technological developments that can be harnessed to replicate, enhance, or create methods for assessment of functional performance. Enabling technologies in wearable sensors, ambient sensors, mobile technologies, and virtual reality make it possible to quantify real-time functional performance and changes in cognitive health. These technologies, their uses for functional health assessment, and their challenges for adoption are presented in this paper.

Objective motor assessment for personalized rehabilitation of upper extremity in brain injury patients

BACKGROUND

The increasing number of patients with acquired brain injury and the current subjectivity of the conventional upper extremity (UE) assessment tests require new objective assessment techniques.

OBJECTIVE

This research proposes a novel objective motor assessment (OMA) methodology based on the Fugl-Meyer assessment (FMA). The goals are to automatically calculate the objective scores (OSs) of FMA-UE movements (as well as a global OS) and to interpret the estimated OSs.

METHODS

Fifteen patients participated in the study. The OMA algorithm was designed to detect small-scale variations in UE movements. The OSs for 14 FMA-UE movements and the global OSs were automatically calculated using the algorithm and evaluated by 2 therapists. The interpretation of the global OSs was performed at 3 levels: by item, movement and globally.

RESULTS

The global OSs calculated by our algorithm had a significant correlation with the therapists' scores (0.783 and 0.938, p <  0.01). All OSs for each movement were correlated with the scores given by the therapists. The correlation coefficient can reach values as high as 0.981 (p <  0.01).

CONCLUSIONS

We provide a new objective assessment tool for therapists to help them improve the diagnostic accuracy and to achieve a more personalized and potentially effective physical rehabilitation of brain injury patients.

[Human-robot global Simulink modeling and analysis for an end-effector upper limb rehabilitation robot]

Robot rehabilitation has been a primary therapy method for the urgent rehabilitation demands of paralyzed patients after a stroke. The parameters in rehabilitation training such as the range of the training, which should be adjustable according to each participant's functional ability, are the key factors influencing the effectiveness of rehabilitation therapy. Therapists design rehabilitation projects based on the semiquantitative functional assessment scales and their experience. But these therapies based on therapists' experience cannot be implemented in robot rehabilitation therapy. This paper modeled the global human-robot by Simulink in order to analyze the relationship between the parameters in robot rehabilitation therapy and the patients' movement functional abilities. We compared the shoulder and elbow angles calculated by simulation with the angles recorded by motion capture system while the healthy subjects completed the simulated action. Results showed there was a remarkable correlation between the simulation data and the experiment data, which verified the validity of the human-robot global Simulink model. Besides, the relationship between the circle radius in the drawing tasks in robot rehabilitation training and the active movement degrees of shoulder as well as elbow was also matched by a linear, which also had a remarkable fitting coefficient. The matched linear can be a quantitative reference for the robot rehabilitation training parameters.

Upper Extremity Functional Evaluation by Fugl-Meyer Assessment Scoring Using Depth-Sensing Camera in Hemiplegic Stroke Patients

Virtual home-based rehabilitation is an emerging area in stroke rehabilitation. Functional assessment tools are essential to monitor recovery and provide current function-based rehabilitation. We developed the Fugl-Meyer Assessment (FMA) tool using Kinect (Microsoft, USA) and validated it for hemiplegic stroke patients. Forty-one patients with hemiplegic stroke were enrolled. Thirteen of 33 items were selected for upper extremity motor FMA. One occupational therapist assessed the motor FMA while recording upper extremity motion with Kinect. FMA score was calculated using principal component analysis and artificial neural network learning from the saved motion data. The degree of jerky motion was also transformed to jerky scores. Prediction accuracy for each of the 13 items and correlations between real FMA scores and scores using Kinect were analyzed. Prediction accuracies ranged from 65% to 87% in each item and exceeded 70% for 9 items. Correlations were high for the summed score for the 13 items between real FMA scores and scores obtained using Kinect (Pearson’s correlation coefficient = 0.873, P<0.0001) and those between total upper extremity scores (66 in full score) and scores using Kinect (26 in full score) (Pearson’s correlation coefficient = 0.799, P<0.0001). Log transformed jerky scores were significantly higher in the hemiplegic side (1.81 ± 0.76) compared to non-hemiplegic side (1.21 ± 0.43) and showed significant negative correlations with Brunnstrom stage (3 to 6; Spearman correlation coefficient = -0.387, P = 0.046). FMA using Kinect is a valid way to assess upper extremity function and can provide additional results for movement quality in stroke patients. This may be useful in the setting of unsupervised home-based rehabilitation.

A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks

To extend the use of wearable sensor networks for stroke patients training and assessment in non-clinical settings, this paper proposes a novel remote quantitative Fugl-Meyer assessment (FMA) framework, in which two accelerometer and seven flex sensors were used to monitoring the movement function of upper limb, wrist and fingers. The extreme learning machine based ensemble regression model was established to map the sensor data to clinical FMA scores while the RRelief algorithm was applied to find the optimal features subset. Considering the FMA scale is time-consuming and complicated, seven training exercises were designed to replace the upper limb related 33 items in FMA scale. 24 stroke inpatients participated in the experiments in clinical settings and 5 of them were involved in the experiments in home settings after they left the hospital. Both the experimental results in clinical and home settings showed that the proposed quantitative FMA model can precisely predict the FMA scores based on wearable sensor data, the coefficient of determination can reach as high as 0.917. It also indicated that the proposed framework can provide a potential approach to the remote quantitative rehabilitation training and evaluation.

Motion trajectory analysis for evaluating the performance of functional upper extremity tasks in daily living: a pilot study

Since 1998, tele-rehabilitation has been extensively studied for its potential capacity of saving time and cost for both therapists and patients. However, one gap hindering the deployment of tele-rehabilitation service is the approach to evaluate the outcome after tele-rehabilitation exercises without the presence of professional clinicians. In this paper, we propose an approach to model jerky and jerky-free movement trajectories with hidden Markov models (HMMs). The HMMs are then utilised to identify the jerky characteristics in a motion trajectory, thereby providing the number and amplitude of jerky movements in the specific length of the trajectory. Eventually, the ability of performing functional upper extremity tasks can be evaluated by classifying the motion trajectory into one of the pre-defined ability levels by looking at the number and amplitude of jerky movements. The simulation experiment confirmed that the proposed method is able to correctly classify motion trajectories into various ability levels to a high degree.

Automated extraction and validation of children's gait parameters with the Kinect

BACKGROUND

Gait analysis for therapy regimen prescription and monitoring requires patients to physically access clinics with specialized equipment. The timely availability of such infrastructure at the right frequency is especially important for small children. Besides being very costly, this is a challenge for many children living in rural areas. This is why this work develops a low-cost, portable, and automated approach for in-home gait analysis, based on the Microsoft Kinect.

METHODS

A robust and efficient method for extracting gait parameters is introduced, which copes with the high variability of noisy Kinect skeleton tracking data experienced across the population of young children. This is achieved by temporally segmenting the data with an approach based on coupling a probabilistic matching of stride template models, learned offline, with the estimation of their global and local temporal scaling. A preliminary study conducted on healthy children between 2 and 4 years of age is performed to analyze the accuracy, precision, repeatability, and concurrent validity of the proposed method against the GAITRite when measuring several spatial and temporal children's gait parameters.

RESULTS

The method has excellent accuracy and good precision, with segmenting temporal sequences of body joint locations into stride and step cycles. Also, the spatial and temporal gait parameters, estimated automatically, exhibit good concurrent validity with those provided by the GAITRite, as well as very good repeatability. In particular, on a range of nine gait parameters, the relative and absolute agreements were found to be good and excellent, and the overall agreements were found to be good and moderate.

CONCLUSION

This work enables and validates the automated use of the Kinect for children's gait analysis in healthy subjects. In particular, the approach makes a step forward towards developing a low-cost, portable, parent-operated in-home tool for clinicians assisting young children.