The variable relationship between arm and hand use: a rationale for using finger magnetometry to complement wrist accelerometry when measuring daily use of the upper extremity

Perspectives and expectations of stroke survivors using egocentric cameras for monitoring hand function at home: a mixed methods study

PURPOSE

Most stroke survivors have remaining upper limb impairment six months after stroke and require additional rehabilitation and help from family members to enhance their performance of daily activities. First-person (egocentric) video has been proposed to capture the activities of daily living (ADLs) of stroke survivors in order to assess their hand function at home. This study explored the experiences and expectations of stroke survivors regarding the use of egocentric cameras in daily life for rehabilitation applications.

METHODS

Twenty-one chronic stroke survivors recruited for the study were asked to record three sessions of 1.5 h of video of their ADLs at home over two weeks. Their experiences and expectations after completing the recordings were discussed using a structured questionnaire and a semi-structured interview. The questionnaire and interview data were analysed using descriptive statistics and content analysis, respectively. The results were further integrated using a mixed methods analysis for mutual explanation and elaboration.

RESULTS

The themes generated were Camera Usability, Privacy Concerns Related to Home Recordings, Future Use of the Camera in Public, and Information Usefulness. The participants perceived that the camera was easy to use, the information obtained from the recordings was beneficial, and no major concerns about recording at home. A discreet camera and a solution to privacy issues were prerequisites to recording tasks in public.

CONCLUSIONS

There was high acceptance among stroke survivors regarding the use of wearable cameras for rehabilitation purposes in the future. Concerns to be managed include discomfort, self-consciousness, and the privacy of others.Implications for rehabilitationThe egocentric camera was easy for the stroke survivors to use at home. However, they expressed a preference for cameras to be less noticeable and lighter in the future to minimize self-consciousness and discomfort.Expectations for future use of an egocentric camera for upper limb rehabilitation at home from the perspectives of stroke survivors included receiving feedback on their hand function in daily life and guidance on how to improve function.Privacy concerns of stroke survivors regarding recording activities of daily living were mostly avoidable by planning in advance. However, some personal hygiene tasks and virtual meetings were recorded by accident. A checklist of common activities that may raise privacy issues can be provided along with the camera to serve as a reminder to avoid these issues.

Envisioning the use of in-situ arm movement data in stroke rehabilitation: Stroke survivors' and occupational therapists' perspectives

BACKGROUND

The key for successful stroke upper-limb rehabilitation includes the personalization of therapeutic interventions based on patients' functional ability and performance level. However, therapists often encounter challenges in supporting personalized rehabilitation due to the lack of information about how stroke survivors use their stroke-affected arm outside the clinic. Wearable technologies have been considered as an effective, objective solution to monitor patients' arm use patterns in their naturalistic environments. However, these technologies have remained a proof of concept and have not been adopted as mainstream therapeutic products, and we lack understanding of how key stakeholders perceive the use of wearable technologies in their practice.

OBJECTIVE

We aim to understand how stroke survivors and therapists perceive and envision the use of wearable sensors and arm activity data in practical settings and how we could design a wearable-based performance monitoring system to better support the needs of the stakeholders.

METHODS

We conducted semi-structured interviews with four stroke survivors and 15 occupational therapists (OTs) based on real-world arm use data that we collected for contextualization. To situate our participants, we leveraged a pair of finger-worn accelerometers to collect stroke survivors' arm use data in real-world settings, which we used to create study probes for stroke survivors and OTs, respectively. The interview data was analyzed using the thematic approach.

RESULTS

Our study unveiled a detailed account of (1) the receptiveness of stroke survivors and OTs for using wearable sensors in clinical practice, (2) OTs' envisioned strategies to utilize patient-generated sensor data in the light of providing patients with personalized therapy programs, and (3) practical challenges and design considerations to address for the accelerated integration of wearable systems into their practice.

CONCLUSIONS

These findings offer promising directions for the design of a wearable solution that supports OTs to develop individually-tailored therapy programs for stroke survivors to improve their affected arm use.

Upper Limb Performance in Daily Life Approaches Plateau Around Three to Six Weeks Post-stroke

Background. Wearable sensors allow for direct measurement of upper limb (UL) performance in daily life. Objective. To map the trajectory of UL performance and its relationships to other factors post-stroke. Methods. Participants (n = 67) with first stroke and UL paresis were assessed at 2, 4, 6, 8, 12, 16, 20, and 24 weeks after stroke. Assessments captured UL impairment (Fugl-Meyer), capacity for activity (Action Research Arm Test), and performance of activity in daily life (accelerometer variables of use ratio and hours of paretic limb activity), along with other potential modifying factors. We modeled individual trajectories of change for each measurement level and the moderating effects on UL performance trajectories. Results. Individual trajectories were best fit with a 3-parameter logistic model, capturing the rapid growth early after stroke within the longer data collection period. Plateaus (90% of asymptote) in impairment (bootstrap mean ± SE: 32 ± 4 days post-stroke) preceded those in capacity (41 ± 4 days). Plateau in performance, as measured by the use ratio (24 ± 5 days), tended to precede plateaus in impairment and capacity. Plateau in performance, as measured by hours of paretic activity (41 ± 6 days), occurred at a similar time to that of capacity and slightly lagged impairment. Modifiers of performance trajectories were capacity, concordance, UL rehabilitation, depressive symptomatology, and cognition. Conclusions. Upper limb performance in daily life approached plateau 3 to 6 weeks post-stroke. Individuals with stroke started to achieve a stable pattern of UL use in daily life early, often before neurological impairments and functional capacity started to stabilize.

Recognizing Manual Activities Using Wearable Inertial Measurement Units: Clinical Application for Outcome Measurement

The ability to monitor activities of daily living in the natural environments of patients could become a valuable tool for various clinical applications. In this paper, we show that a simple algorithm is capable of classifying manual activities of daily living (ADL) into categories using data from wrist- and finger-worn sensors. Six participants without pathology of the upper limb performed 14 ADL. Gyroscope signals were used to analyze the angular velocity pattern for each activity. The elaboration of the algorithm was based on the examination of the activity at the different levels (hand, fingers and wrist) and the relationship between them for the duration of the activity. A leave-one-out cross-validation was used to validate our algorithm. The algorithm allowed the classification of manual activities into five different categories through three consecutive steps, based on hands ratio (i.e., activity of one or both hands) and fingers-to-wrist ratio (i.e., finger movement independently of the wrist). On average, the algorithm made the correct classification in 87.4% of cases. The proposed algorithm has a high overall accuracy, yet its computational complexity is very low as it involves only averages and ratios.

Identifying Hand Use and Hand Roles After Stroke Using Egocentric Video

Objective: Upper limb (UL) impairment impacts quality of life, but is common after stroke. UL function evaluated in the clinic may not reflect use in activities of daily living (ADLs) after stroke, and current approaches for assessment at home rely on self-report and lack details about hand function. Wrist-worn accelerometers have been applied to capture UL use, but also fail to reveal details of hand function. In response, a wearable system is proposed consisting of egocentric cameras combined with computer vision approaches, in order to identify hand use (hand-object interactions) and the role of the more-affected hand (as stabilizer or manipulator) in unconstrained environments. Methods: Nine stroke survivors recorded their performance of ADLs in a home simulation laboratory using an egocentric camera. Motion, hand shape, colour, and hand size change features were generated and fed into random forest classifiers to detect hand use and classify hand roles. Leave-one-subject-out cross-validation (LOSOCV) and leave-one-task-out cross-validation (LOTOCV) were used to evaluate the robustness of the algorithms. Results: LOSOCV and LOTOCV F1-scores for more-affected hand use were 0.64 ± 0.24 and 0.76 ± 0.23, respectively. For less-affected hands, LOSOCV and LOTOCV F1-scores were 0.72 ± 0.20 and 0.82 ± 0.22. F1-scores for hand role classification were 0.70 ± 0.19 and 0.68 ± 0.23 in the more-affected hand for LOSOCV and LOTOCV, respectively, and 0.59 ± 0.23 and 0.65 ± 0.28 in the less-affected hand. Conclusion: The results demonstrate the feasibility of predicting hand use and the hand roles of stroke survivors from egocentric videos.

Magnetically Counting Hand Movements: Validation of a Calibration-Free Algorithm and Application to Testing the Threshold Hypothesis of Real-World Hand Use after Stroke

There are few wearable sensors suitable for daily monitoring of wrist and finger movements for hand-related healthcare applications. Here, we describe the development and validation of a novel algorithm for magnetically counting hand movements. We implemented the algorithm on a wristband that senses magnetic field changes produced by movement of a magnetic ring worn on the finger (the “Manumeter”). The “HAND” (Hand Activity estimated by Nonlinear Detection) algorithm assigns a “HAND count” by thresholding the real-time change in magnetic field created by wrist and/or finger movement. We optimized thresholds to achieve a HAND count accuracy of ~85% without requiring subject-specific calibration. Then, we validated the algorithm in a dexterity-impaired population by showing that HAND counts strongly correlate with clinical assessments of upper extremity (UE) function after stroke. Finally, we used HAND counts to test a recent hypothesis in stroke rehabilitation that real-world UE hand use increases only for stroke survivors who achieve a threshold level of UE functional capability. For 29 stroke survivors, HAND counts measured at home did not increase until the participants’ Box and Blocks Test scores exceeded ~50% normal. These results show that a threshold-based magnetometry approach can non-obtrusively quantify hand movements without calibration and also verify a key concept of real-world hand use after stroke.

Systematic review on the application of wearable inertial sensors to quantify everyday life motor activity in people with mobility impairments

BACKGROUND

Recent advances in wearable sensor technologies enable objective and long-term monitoring of motor activities in a patient's habitual environment. People with mobility impairments require appropriate data processing algorithms that deal with their altered movement patterns and determine clinically meaningful outcome measures. Over the years, a large variety of algorithms have been published and this review provides an overview of their outcome measures, the concepts of the algorithms, the type and placement of required sensors as well as the investigated patient populations and measurement properties.

METHODS

A systematic search was conducted in MEDLINE, EMBASE, and SCOPUS in October 2019. The search strategy was designed to identify studies that (1) involved people with mobility impairments, (2) used wearable inertial sensors, (3) provided a description of the underlying algorithm, and (4) quantified an aspect of everyday life motor activity. The two review authors independently screened the search hits for eligibility and conducted the data extraction for the narrative review.

RESULTS

Ninety-five studies were included in this review. They covered a large variety of outcome measures and algorithms which can be grouped into four categories: (1) maintaining and changing a body position, (2) walking and moving, (3) moving around using a wheelchair, and (4) activities that involve the upper extremity. The validity or reproducibility of these outcomes measures was investigated in fourteen different patient populations. Most of the studies evaluated the algorithm's accuracy to detect certain activities in unlabeled raw data. The type and placement of required sensor technologies depends on the activity and outcome measure and are thoroughly described in this review. The usability of the applied sensor setups was rarely reported.

CONCLUSION

This systematic review provides a comprehensive overview of applications of wearable inertial sensors to quantify everyday life motor activity in people with mobility impairments. It summarizes the state-of-the-art, it provides quick access to the relevant literature, and it enables the identification of gaps for the evaluation of existing and the development of new algorithms.

Towards the Design of a Ring Sensor-based mHealth System to Achieve Optimal Motor Function in Stroke Survivors

Maximizing the motor practice in stroke survivors' living environments may significantly improve the functional recovery of their stroke-affected upper-limb. A wearable system that can continuously monitor upper-limb performance has been considered as an effective clinical solution for its potential to provide patient-centered, data-driven feedback to improve the motor dosage. Towards that end, we investigate a system leveraging a pair of finger-worn, ring-type accelerometers capable of monitoring both gross-arm and fine-hand movements that are clinically relevant to the performance of daily activities. In this work, we conduct a mixed-methods study to (1) quantitatively evaluate the efficacy of finger-worn accelerometers in measuring clinically relevant information regarding stroke survivors' upper-limb performance, and (2) qualitatively investigate design requirements for the self-monitoring system, based on data collected from 25 stroke survivors and seven occupational therapists. Our quantitative findings demonstrate strong face and convergent validity of the finger-worn accelerometers, and its responsiveness to changes in motor behavior. Our qualitative findings provide a detailed account of the current rehabilitation process while highlighting several challenges that therapists and stroke survivors face. This study offers promising directions for the design of a self-monitoring system that can encourage the affected limb use during stroke survivors' daily living.

Thumb and finger movement is reduced after stroke: An observational study

Hand motor impairment is common after stroke but there are few comprehensive data on amount of hand movement. This study aimed to compare the amount of thumb and finger movement over an extended period of time in people with stroke and able-bodied people. Fifteen stroke subjects and 15 able-bodied control subjects participated. Stroke subjects had impaired hand function. Movement of the thumb and index finger was recorded using stretch sensors worn on the affected hand (stroke subjects) or the left or right hand (control subjects) for ∼4 hours during the day. A digit movement was defined as a monotonic increase or decrease in consecutive sensor values. Instantaneous digit position was expressed as a percentage of maximal digit flexion. Mixed linear models were used to compare the following outcomes between groups: (1) average amplitude of digit movement, (2) digit cadence and average digit velocity, (3) percentage of digit idle time and longest idle time. Amplitude of digit movement was not different between groups. Cadence at the thumb (between-group mean difference, 95% CI, p value: -0.6 movements/sec, -1.0 to -0.2 movements/sec, p = 0.003) and finger (-0.5 movements/sec, -0.7 to -0.3 movements/sec, p<0.001) was lower in stroke than control subjects. Digit velocity was not different between groups. Thumb idle time was not different between groups, but finger idle time was greater in stroke than control subjects (percentage of idle time: 6%, 1 to 11%, p = 0.02; longest idle time: 375 sec, 29 to 721 sec, p = 0.04). Rehabilitation after stroke should encourage the performance of functional tasks that involve movements at faster cadences, and encourage more frequent movement of the digits with shorter periods of inactivity.

Wearable sensing for rehabilitation after stroke: Bimanual jerk asymmetry encodes unique information about the variability of upper extremity recovery

Wearable sensing is a new tool for quantifying upper extremity (UE) rehabilitation after stroke. However, it is unclear whether it provides information beyond what is available through standard clinical assessments. To investigate this question, people with a chronic stroke (n=9) wore accelerometers on both wrists for 9 hours on a single day during their daily activities. We used principal components analysis (PCA) to characterize how novel kinematic measures of jerk and acceleration asymmetry, along with conventional measures of limb use asymmetry and clinical function, explained the behavioral variance of UE recovery across participants. The first PC explained 55% of the variance and described a strong correlation between standard clinical assessments and limb use asymmetry, as has been observed previously. The second PC explained a further 31% of the variance and described a strong correlation between bimanual magnitude and jerk asymmetry. Because of the nature of PCA, this second PC is mathematically orthogonal to the first and thus uncorrelated with the clinical assessments. Therefore, kinematic metrics obtainable from bimanual accelerometry, including bimanual jerk asymmetry, encoded additional information about UE recovery. One interpretation is that the first PC relates to "functional status" and the second to "movement quality". We also describe a new graphical format for presenting bimanual wrist accelerometry data that facilitates identification of asymmetries.

Views of individuals with spinal cord injury on the use of wearable cameras to monitor upper limb function in the home and community

OBJECTIVE

Hand function impairment after cervical spinal cord injury (SCI) can significantly reduce independence. Unlike current hand function assessments, wearable camera systems could potentially measure functional hand usage at home, and thus benefit the development of neurorehabilitation strategies. The objective of this study was to understand the views of individuals with SCI on the use of wearable cameras to track neurorehabilitation progress and outcomes in the community.

DESIGN

Questionnaires.

SETTING

Home simulation laboratory.

PARTICIPANTS

Fifteen individuals with cervical SCI.

OUTCOME MEASURES

After using wearable cameras in the simulated home environment, participants completed custom questionnaires, comprising open-ended and structured questions.

RESULTS

Participants showed relatively low concerns related to data confidentiality when first-person videos are used by clinicians (1.93 ± 1.28 on a 5-point Likert scale) or researchers (2.00 ± 1.31). Storing only automatically extracted metrics reduced privacy concerns. Though participants reported moderate privacy concerns (2.53 ± 1.51) about wearing a camera in daily life due to certain sensitive situations (e.g. washrooms), they felt that information about their hand usage at home is useful for researchers (4.73 ± 0.59), clinicians (4.47 ± 0.83), and themselves (4.40 ± 0.83). Participants found the system moderately comfortable (3.27 ± 1.44), but expressed low desire to use it frequently (2.87 ± 1.36).

CONCLUSION

Despite some privacy and comfort concerns, participants believed that the information obtained would be useful. With appropriate strategies to minimize the data stored and recording duration, wearable cameras can be a well-accepted tool to track function in the home and community after SCI.

Does Task-Specific Training Improve Upper Limb Performance in Daily Life Poststroke?

BACKGROUND

A common assumption is that changes in upper limb (UL) capacity, or what an individual is capable of doing, translates to improved UL performance in daily life, or what an individual actually does. This assumption should be explicitly tested for individuals with UL paresis poststroke.

OBJECTIVE

To examine changes in UL performance after an intensive, individualized, progressive, task-specific UL intervention for individuals at least 6 months poststroke.

METHODS

Secondary analysis on 78 individuals with UL paresis who participated in a phase II, single-blind, randomized parallel dose-response trial. Participants were enrolled in a task-specific intervention for 8 weeks. Participants were randomized into 1 of 4 treatment groups with each group completing different amounts of UL movement practice. UL performance was assessed with bilateral, wrist-worn accelerometers once a week for 24 hours throughout the duration of the study. The 6 accelerometer variables were tested for change and the influence of potential modifiers using hierarchical linear modeling.

RESULTS

No changes in UL performance were found on any of the 6 accelerometer variables used to quantify UL performance. Neither changes in UL capacity nor the overall amount of movement practice influenced changes in UL performance. Stroke chronicity, baseline UL capacity, concordance, and ADL status significantly increased the baseline starting points but did not influence the rate of change (slopes) for participants.

CONCLUSIONS

Improved motor capacity resulting from an intensive outpatient UL intervention does not appear to translate to increased UL performance outside the clinic.

How Therapists Use Visualizations of Upper Limb Movement Information From Stroke Patients: A Qualitative Study With Simulated Information

Background

Stroke is a leading cause of disability worldwide, with upper limb deficits affecting an estimated 30% to 60% of survivors. The effectiveness of upper limb rehabilitation relies on numerous factors, particularly patient compliance to home programs and exercises set by therapists. However, therapists lack objective information about their patients’ adherence to rehabilitation exercises as well as other uses of the affected arm and hand in everyday life outside the clinic. We developed a system that consists of wearable sensor technology to monitor a patient’s arm movement and a Web-based dashboard to visualize this information for therapists.

Objective

The aim of our study was to evaluate how therapists use upper limb movement information visualized on a dashboard to support the rehabilitation process.

Methods

An interactive dashboard prototype with simulated movement information was created and evaluated through a user-centered design process with therapists (N=8) at a rehabilitation clinic. Data were collected through observations of therapists interacting with an interactive dashboard prototype, think-aloud data, and interviews. Data were analyzed qualitatively through thematic analysis.

Results

Therapists use visualizations of upper limb information in the following ways: (1) to obtain objective data of patients’ activity levels, exercise, and neglect outside the clinic, (2) to engage patients in the rehabilitation process through education, motivation, and discussion of experiences with activities of daily living, and (3) to engage with other clinicians and researchers based on objective data. A major limitation is the lack of contextual data, which is needed by therapists to discern how movement data visualized on the dashboard relate to activities of daily living.

Conclusions

Upper limb information captured through wearable devices provides novel insights for therapists and helps to engage patients and other clinicians in therapy. Consideration needs to be given to the collection and visualization of contextual information to provide meaningful insights into patient engagement in activities of daily living. These findings open the door for further work to develop a fully functioning system and to trial it with patients and clinicians during therapy.

The Reality of Myoelectric Prostheses: Understanding What Makes These Devices Difficult for Some Users to Control

Users of myoelectric prostheses can often find them difficult to control. This can lead to passive-use of the device or total rejection, which can have detrimental effects on the contralateral limb due to overuse. Current clinically available prostheses are “open loop” systems, and although considerable effort has been focused on developing biofeedback to “close the loop,” there is evidence from laboratory-based studies that other factors, notably improving predictability of response, may be as, if not more, important. Interestingly, despite a large volume of research aimed at improving myoelectric prostheses, it is not currently known which aspect of clinically available systems has the greatest impact on overall functionality and everyday usage. A protocol has, therefore, been designed to assess electromyographic (EMG) skill of the user and predictability of the prosthesis response as significant parts of the control chain, and to relate these to functionality and everyday usage. Here, we present the protocol and results from early pilot work. A set of experiments has been developed. First, to characterize user skill in generating the required level of EMG signal, as well as the speed with which users are able to make the decision to activate the appropriate muscles. Second, to measure unpredictability introduced at the skin–electrode interface, in order to understand the effects of the socket-mounted electrode fit under different loads on the variability of time taken for the prosthetic hand to respond. To evaluate prosthesis user functionality, four different outcome measures are assessed. Using a simple upper limb functional task prosthesis users are assessed for (1) success of task completion, (2) task duration, (3) quality of movement, and (4) gaze behavior. To evaluate everyday usage away from the clinic, the symmetricity of their real-world arm use is assessed using activity monitoring. These methods will later be used to assess a prosthesis user cohort to establish the relative contribution of each control factor to the individual measures of functionality and everyday usage (using multiple regression models). The results will support future researchers, designers, and clinicians in concentrating their efforts on the area that will have the greatest impact on improving prosthesis use.

Computational neurorehabilitation: modeling plasticity and learning to predict recovery

Despite progress in using computational approaches to inform medicine and neuroscience in the last 30 years, there have been few attempts to model the mechanisms underlying sensorimotor rehabilitation. We argue that a fundamental understanding of neurologic recovery, and as a result accurate predictions at the individual level, will be facilitated by developing computational models of the salient neural processes, including plasticity and learning systems of the brain, and integrating them into a context specific to rehabilitation. Here, we therefore discuss Computational Neurorehabilitation, a newly emerging field aimed at modeling plasticity and motor learning to understand and improve movement recovery of individuals with neurologic impairment. We first explain how the emergence of robotics and wearable sensors for rehabilitation is providing data that make development and testing of such models increasingly feasible. We then review key aspects of plasticity and motor learning that such models will incorporate. We proceed by discussing how computational neurorehabilitation models relate to the current benchmark in rehabilitation modeling - regression-based, prognostic modeling. We then critically discuss the first computational neurorehabilitation models, which have primarily focused on modeling rehabilitation of the upper extremity after stroke, and show how even simple models have produced novel ideas for future investigation. Finally, we conclude with key directions for future research, anticipating that soon we will see the emergence of mechanistic models of motor recovery that are informed by clinical imaging results and driven by the actual movement content of rehabilitation therapy as well as wearable sensor-based records of daily activity.