Measurements by A LEAP-Based Virtual Glove for the Hand Rehabilitation

Portable Head-Mounted System for Mobile Forearm Tracking

Computer vision (CV)-based systems using cameras and recognition algorithms offer touchless, cost-effective, precise, and versatile hand tracking. These systems allow unrestricted, fluid, and natural movements without the constraints of wearable devices, gaining popularity in human-system interaction, virtual reality, and medical procedures. However, traditional CV-based systems, relying on stationary cameras, are not compatible with mobile applications and demand substantial computing power. To address these limitations, we propose a portable hand-tracking system utilizing the Leap Motion Controller 2 (LMC) mounted on the head and controlled by a single-board computer (SBC) powered by a compact power bank. The proposed system enhances portability, enabling users to interact freely with their surroundings. We present the system's design and conduct experimental tests to evaluate its robustness under variable lighting conditions, power consumption, CPU usage, temperature, and frame rate. This portable hand-tracking solution, which has minimal weight and runs independently of external power, proves suitable for mobile applications in daily life.

Patient-Therapist Cooperative Hand Telerehabilitation through a Novel Framework Involving the Virtual Glove System

Telerehabilitation is important for post-stroke or post-surgery rehabilitation because the tasks it uses are reproducible. When combined with assistive technologies, such as robots, virtual reality, tracking systems, or a combination of them, it can also allow the recording of a patient's progression and rehabilitation monitoring, along with an objective evaluation. In this paper, we present the structure, from actors and functionalities to software and hardware views, of a novel framework that allows cooperation between patients and therapists. The system uses a computer-vision-based system named virtual glove for real-time hand tracking (40 fps), which is translated into a light and precise system. The novelty of this work lies in the fact that it gives the therapist quantitative, not only qualitative, information about the hand's mobility, for every hand joint separately, while at the same time providing control of the result of the rehabilitation by also quantitatively monitoring the progress of the hand mobility. Finally, it also offers a strategy for patient-therapist interaction and therapist-therapist data sharing.

Feasibility and Performance Validation of a Leap Motion Controller for Upper Limb Rehabilitation

The leap motion controller is a commercial low-cost marker-less optical sensor that can track the motion of a human hand by recording various parameters. Upper limb rehabilitation therapy is the treatment of people having upper limb impairments, whose recovery is achieved through continuous motion exercises. However, the repetitive nature of these exercises can be interpreted as boring or discouraging while patient motivation plays a key role in their recovery. Thus, serious games have been widely used in therapies for motivating patients and making the therapeutic process more enjoyable. This paper explores the feasibility, accuracy, and repeatability of a leap motion controller (LMC) to be applied in combination with a serious game for upper limb rehabilitation. Experimental feasibility tests are carried out by using an industrial robot that replicates the upper limb motions and is tracked by using an LMC. The results suggest a satisfactory performance in terms of tracking accuracy although some limitations are identified and discussed in terms of measurable workspace.

An Innovative Approach for Online Neuroanatomy and Neurorrehabilitation Teaching Based on 3D Virtual Anatomical Models Using Leap Motion Controller During COVID-19 Pandemic

After the World Health Organization had declared a pandemic of coronavirus disease (COVID-19) on March 11, 2020 many governments, including the Government of Spain, declared the state of alarm enforcing a quarantine that have left millions of students confined to their homes. This home confinement has affected students of all levels, including university students, and has forced faculties to adapt online teaching strategies. Thus, traditional classroom face-to-face teaching has suddenly been replaced by online classes. This has revealed particularly challenging for medical courses. For such purpose we have designed an online teaching proposal addressed to the Degree in Physiotherapy and the Double Degree in Nursing and Physiotherapy of the University of Jaén (Spain). The objective is to implement an online virtual teaching protocol through the use of Virtual Reality. For such a goal, the Leap Motion Controller (LMC) will be used to teach the neuroanatomy of the brain and spinal cord and to teach and practice neurorehabilitation exercises. Along with devices like the LMC students will be asked to use Health Sciences databases in order to achieve a significative learning of the course topics. The project is structured in two phases. First, students will learn neuroanatomy and neurophysiology of the most relevant neurological conditions using LMC-based models. Then, they will learn to combine LMC games and conventional physiotherapy for neurorehabilitation purposes. The work of students will include the recording of videoreports demonstrating the acquisition of neuroanatomy concepts and simulating a clinical case. With this project we will assess the usability of LMC as an educative tool, the perception, satisfaction and self-regulated learning of physiotherapy students.

RehabHand: Oriented-tasks serious games for upper limb rehabilitation by using Leap Motion Controller and target population in spinal cord injury

BACKGROUND

There is a growing interest in the use of technology in the field of neurorehabilitation in order to quantify and generate knowledge about sensorimotor disorders after neurological diseases, understanding that the technology has a high potential for its use as therapeutic tools. Taking into account that the rehabilitative process of motor disorders should extend beyond the inpatient condition, it's necessary to involve low-cost technology, in order to have technological solutions that can approach the outpatient period at home.

OBJECTIVE

to present the virtual applications-based RehabHand prototype for the rehabilitation of manipulative skills of the upper limbs in patients with neurological conditions and to determine the target population with respect to spinal cord injured patients.

METHODS

Seven virtual reality applications have been designed and developed with a therapeutic sense, manipulated by means of Leap Motion Controller. The target population was determined from a sample of 40 people, healthy and patients, analyzing hand movements and gestures.

RESULTS

The hand movements and gestures were estimated with a fitting rate between the range 0.607-0.953, determining the target population by cervical levels and upper extremity motor score.

CONCLUSIONS

Leap Motion is suitable for a determined sample of cervical patients with a rehabilitation purpose.

Data integration by two-sensors in a LEAP-based Virtual Glove for human-system interaction

Virtual Glove (VG) is a low-cost computer vision system that utilizes two orthogonal LEAP motion sensors to provide detailed 4D hand tracking in real–time. VG can find many applications in the field of human-system interaction, such as remote control of machines or tele-rehabilitation. An innovative and efficient data-integration strategy, based on the velocity calculation, for selecting data from one of the LEAPs at each time, is proposed for VG. The position of each joint of the hand model, when obscured to a LEAP, is guessed and tends to flicker. Since VG uses two LEAP sensors, two spatial representations are available each moment for each joint: the method consists of the selection of the one with the lower velocity at each time instant. Choosing the smoother trajectory leads to VG stabilization and precision optimization, reduces occlusions (parts of the hand or handling objects obscuring other hand parts) and/or, when both sensors are seeing the same joint, reduces the number of outliers produced by hardware instabilities. The strategy is experimentally evaluated, in terms of reduction of outliers with respect to a previously used data selection strategy on VG, and results are reported and discussed. In the future, an objective test set has to be imagined, designed, and realized, also with the help of an external precise positioning equipment, to allow also quantitative and objective evaluation of the gain in precision and, maybe, of the intrinsic limitations of the proposed strategy. Moreover, advanced Artificial Intelligence-based (AI-based) real-time data integration strategies, specific for VG, will be designed and tested on the resulting dataset.

Validity of Hololens Augmented Reality Head Mounted Display for Measuring Gait Parameters in Healthy Adults and Children with Cerebral Palsy

Serious games are a promising approach to improve gait rehabilitation for people with gait disorders. Combined with wearable augmented reality headset, serious games for gait rehabilitation in a clinical setting can be envisaged, allowing to evolve in a real environment and provide fun and feedback to enhance patient’s motivation. This requires a method to obtain accurate information on the spatiotemporal gait parameters of the playing patient. To this end, we propose a new algorithm called HoloStep that computes spatiotemporal gait parameters using only the head pose provided by an augmented reality headset (Hololens). It is based on the detection of peaks associated to initial contact event, and uses a combination of locking distance, locking time, peak amplitude detection with custom thresholds for children with CP. The performance of HoloStep was compared during a walking session at comfortable speed to Zeni’s reference algorithm, which is based on kinematics and a full 3D motion capture system. Our study included 62 children with cerebral palsy (CP), classified according to Gross Motor Function Classification System (GMFCS) between levels I and III, and 13 healthy participants (HP). Metrics such as sensitivity, specificity, accuracy and precision for step detection with HoloStep were above 96%. The Intra-Class Coefficient between steps length calculated with HoloStep and the reference was 0.92 (GMFCS I), 0.86 (GMFCS II/III) and 0.78 (HP). HoloStep demonstrated good performance when applied to a wide range of gait patterns, including children with CP using walking aids. Findings provide important insights for future gait intervention using augmented reality games for children with CP.

Hand Motion Capture from a 3D Leap Motion Controller for a Musculoskeletal Dynamic Simulation

The AnyBody Modeling System™ (AMS) is a musculoskeletal software simulation solution using inverse dynamics analysis. It enables the determination of muscle and joint forces for a given bodily motion. The recording of the individual movement and the transfer into the AMS is a complex and protracted process. Researches indicated that the contactless, visual Leap Motion Controller (LMC) provides clinically meaningful motion data for hand tracking. Therefore, the aim of this study was to integrate the LMC hand motion data into the AMS in order to improve the process of recording a hand movement. A Python-based interface between the LMC and the AMS, termed ROSE Motion, was developed. This solution records and saves the data of the movement as Biovision Hierarchy (BVH) data and AnyScript vector files that are imported into the AMS simulation. Setting simulation parameters, initiating the calculation automatically, and fetching results is implemented by using the AnyPyTools library from AnyBody. The proposed tool offers a rapid and easy-to-use recording solution for elbow, hand, and finger movements. Features include animation, cutting/editing, exporting the motion, and remote controlling the AMS for the analysis and presentation of musculoskeletal simulation results. Comparing the motion tracking results with previous studies, covering problems when using the LMC limit the correctness of the motion data. However, fast experimental setup and intuitive and rapid motion data editing strengthen the use of marker less systems as the herein presented compared to marker based motion capturing.

A Gesture-Based Teleoperation System for Compliant Robot Motion

Currently, the gesture-based teleoperation system cannot generate precise and compliant robot motions because human motions have the characteristics of uncertainty and low-resolution. In this paper, a novel, gesture-based teleoperation system for compliant robot motion is proposed. By using the left hand as the commander and the right hand as a positioner, different operation modes and scaling ratios can be tuned on-the-fly to meet the accuracy and efficiency requirements. Moreover, a vibration-based force feedback system was developed to provide the operator with a telepresence capability. The pick-and-place and peg-in-hole tasks were used to test the effectiveness of the teleoperation system we developed. The experiment results prove that the gesture-based teleoperation system is effective at handling compliant robot motions.

Support Vector Machine-Based Classifier for the Assessment of Finger Movement of Stroke Patients Undergoing Rehabilitation

Purpose

Traditionally, clinical evaluation of motor paralysis following stroke has been of value to physicians and therapists because it allows for immediate pathophysiological assessment without the need for specialized tools. However, current clinical methods do not provide objective quantification of movement; therefore, they are of limited use to physicians and therapists when assessing responses to rehabilitation. The present study aimed to create a support vector machine (SVM)-based classifier to analyze and validate finger kinematics using the leap motion controller. Results were compared with those of 24 stroke patients assessed by therapists.

Methods

A non-linear SVM was used to classify data according to the Brunnstrom recovery stages of finger movements by focusing on peak angle and peak velocity patterns during finger flexion and extension. One thousand bootstrap data values were generated by randomly drawing a series of sample data from the actual normalized kinematics-related data. Bootstrap data values were randomly classified into training (940) and testing (60) datasets. After establishing an SVM classification model by training with the normalized kinematics-related parameters of peak angle and peak velocity, the testing dataset was assigned to predict classification of paralytic movements.

Results

High separation accuracy was obtained (mean 0.863; 95% confidence interval 0.857–0.869; p = 0.006).

Conclusion

This study highlights the ability of artificial intelligence to assist physicians and therapists evaluating hand movement recovery of stroke patients.

Directional Force Feedback: Mechanical Force Concentration for Immersive Experience in Virtual Reality

In recent years, consumer-level virtual-reality (VR) devices and content have become widely available. Notably, establishing a sense of presence is a key objective of VR and an immersive interface with haptic feedback for VR applications has long been in development. Despite the state-of-the-art force feedback research being conducted, a study on directional feedback, based on force concentration, has not yet been reported. Therefore, we developed directional force feedback (DFF), a device that generates directional sensations for virtual-reality (VR) applications via mechanical force concentrations. DFF uses the rotation of motors to concentrate force and deliver directional sensations to the user. To achieve this, we developed a novel method of force concentration for directional sensation; by considering both rotational rebound and gravity, the optimum rotational motor speeds and rotation angles were identified. Additionally, we validated the impact of DFF in a virtual environment, showing that the users’ presence and immersion within VR were higher with DFF than without. The result of the user studies demonstrated that the device significantly improves immersivity of virtual applications.

A Game Changer: 'The Use of Digital Technologies in the Management of Upper Limb Rehabilitation'

Hemiparesis is a symptom of residual weakness in half of the body, including the upper extremity, which affects the majority of post stroke survivors. Upper limb function is essential for daily life and reduction in movements can lead to tremendous decline in quality of life and independence. Current treatments, such as physiotherapy, aim to improve motor functions, however due to increasing NHS pressure, growing recognition on mental health, and close scrutiny on disease spending there is an urgent need for new approaches to be developed rapidly and sufficient resources devoted to stroke disease. Fortunately, a range of digital technologies has led to revived rehabilitation techniques in captivating and stimulating environments. To gain further insight, a meta-analysis literature search was carried out using the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) method. Articles were categorized and pooled into the following groups; pro/anti/neutral for the use of digital technology. Additionally, most literature is rationalised by quantitative and qualitative findings. Findings displayed, the majority of the inclusive literature is supportive of the use of digital technologies in the rehabilitation of upper extremity following stroke. Overall, the review highlights a wide understanding and promise directed into introducing devices into a clinical setting. Analysis of all four categories; (1) Digital Technology, (2) Virtual Reality, (3) Robotics and (4) Leap Motion displayed varying qualities both-pro and negative across each device. Prevailing developments on use of these technologies highlights an evolutionary and revolutionary step into utilizing digital technologies for rehabilitation purposes due to the vast functional gains and engagement levels experienced by patients. The influx of more commercialised and accessible devices could alter stroke recovery further with initial recommendations for combination therapy utilizing conventional and digital resources.

A Novel Setup and Protocol to Measure the Range of Motion of the Wrist and the Hand

The human hand is important for the performance of activities of daily living which are directly related to quality of life. Various conditions, such as Duchenne muscular dystrophy (DMD) can affect the function of the human hand and wrist. The ability to assess the impairment in the hand and the wrist by measuring the range of motion (ROM), is essential for the development of effective rehabilitation protocols. Currently the clinical standard is the goniometer. In this study we explore the feasibility and reliability of an optical sensor (Leap motion sensor) in measuring active hand/wrist ROM. We measured the hand/wrist ROM of 20 healthy adults with the goniometer and the Leap motion sensor, in order to check the agreement between the two methods and additionally, we performed a test-retest of the Leap motion sensor with 12 of them, to assess its reliability. The results suggest low agreement between the goniometer and the leap motion sensor, yet showing a large decrease in measurement time and high reliability when using the later. Despite the low agreement between the two methods, we believe that the Leap motion sensor shows potential to contribute to the development of hand rehabilitation protocols and be used with patients in a clinical setting.