Portable Head-Mounted System for Mobile Forearm Tracking

Computer vision (CV)-based systems using cameras and recognition algorithms offer touchless, cost-effective, precise, and versatile hand tracking. These systems allow unrestricted, fluid, and natural movements without the constraints of wearable devices, gaining popularity in human-system interaction, virtual reality, and medical procedures. However, traditional CV-based systems, relying on stationary cameras, are not compatible with mobile applications and demand substantial computing power. To address these limitations, we propose a portable hand-tracking system utilizing the Leap Motion Controller 2 (LMC) mounted on the head and controlled by a single-board computer (SBC) powered by a compact power bank. The proposed system enhances portability, enabling users to interact freely with their surroundings. We present the system's design and conduct experimental tests to evaluate its robustness under variable lighting conditions, power consumption, CPU usage, temperature, and frame rate. This portable hand-tracking solution, which has minimal weight and runs independently of external power, proves suitable for mobile applications in daily life.

Semi-Immersive Virtual Reality Exercise Therapy for Upper Limb Rehabilitation in Patients With Spinal Cord Injury Using the Leap Motion Controller

In this article, we present a case study involving a patient with spinal cord injury (SCI), resulting in tetraplegia and subsequent loss of upper limb function. The subject of our study was a 23-year-old woman with incomplete tetraplegia stemming from a cervical spinal cord injury. Our primary objective was to enhance hand function and grip strength. Throughout the intervention, we observed substantial enhancements in hand function, range of motion, and muscle power. Notably, the patient exhibited a favorable response to the therapy, demonstrating commendable adherence and active participation. To create an optimal training environment tailored to the patient's needs, we employed the Unity 3D game engine in conjunction with a Leap Motion controller sensor. This combination facilitated the development of a semi-immersive virtual training environment. The utilization of this technology aimed to simulate a conducive training atmosphere for the rehabilitation of hand function. Based on our study outcomes, we advocate for the incorporation of leap motion-related exercises in the treatment of hand functional loss and weakness. The promising results observed in this case study prompt the recommendation for further large-scale studies to validate and substantiate our findings. Such investigations would contribute to the establishment of evidence-based practices and enhance the understanding of the efficacy of Leap Motion technology in addressing upper limb impairments associated with spinal cord injuries.

4D dynamic system for visual-motor integration analysis

It is very important to evaluate visual-motor integration (VMI), as it is used as an index to evaluate cognitive abilities. However, it is difficult to use the existing paper-based tests to evaluate the dynamic process, including spatial and depth perception abilities. Therefore, this study aims to extract kinematic and dynamic features for dynamic assessment for VMI. We propose a 4D dynamic analysis system that implements a VMI test in a virtual space using Leap motion controller and Unity3D and acquires the time-series data of hand joints and traces. We selected three categories of features: postural control ability, spatial and depth perception ability, and 4D analysis. The degree and patterns of postural maintenance differed between subjects in the VMI and MC tests. In addition, the personal patterns were identified with dynamic features, including their fluency and hesitation in relation to the task figures of the VMI test tool. As such, this system enables dynamic feature extraction and analysis which were previously impossible and presents performance results for healthy adults.

Investigating the Effect of Leap Motion on Upper Extremity Rehabilitation in Children with Cerebral Palsy: A Randomized Controlled Trial

In this randomized clinical trial, we look for the following questions' answer: How does the integration of LMC affect (1) upper extremity (UE) function, (2) grip strength, and (3) lateral and palmar pinch strength in children with cerebral palsy (CP), in comparison with conventional rehabilitation methods? Twenty patients were randomly assigned to LMC (case) or conventional (control) groups. The grip, lateral and palmar pinch forces increased in case group patients more than control group both at the end of intervention (P < .001 for all three), and at 20 weeks' follow-up (P values 0.035, 0.002, and 0.002). The Quality of Upper Extremity Skills Test (QUEST) score changes were similar between two groups, except for grasp score at the end of step 2 and 3 (P = .04 and 0.01, respectively). The addition of LMC to the rehabilitation program of patients with CP may improve the UE motor function outcomes.

Analysis of the Leap Motion Controller's Performance in Measuring Wrist Rehabilitation Tasks Using an Industrial Robot Arm Reference

The Leap Motion Controller (LMC) is a low-cost markerless optical sensor that performs measurements of various parameters of the hands that has been investigated for a wide range of different applications. Research attention still needs to focus on the evaluation of its precision and accuracy to fully understand its limitations and widen its range of applications. This paper presents the experimental validation of the LMC device to verify the feasibility of its use in assessing and tailoring wrist rehabilitation therapy for the treatment of physical disabilities through continuous exercises and integration with serious gaming environments. An experimental set up and analysis is proposed using an industrial robot as motion reference. The high repeatability of the selected robot is used for comparisons with the measurements obtained via a leap motion controller while performing the basic movements needed for rehabilitation exercises of the human wrist. Experimental tests are analyzed and discussed to demonstrate the feasibility of using the leap motion controller for wrist rehabilitation.

An Innovative Approach for Online Neuroanatomy and Neurorrehabilitation Teaching Based on 3D Virtual Anatomical Models Using Leap Motion Controller During COVID-19 Pandemic

After the World Health Organization had declared a pandemic of coronavirus disease (COVID-19) on March 11, 2020 many governments, including the Government of Spain, declared the state of alarm enforcing a quarantine that have left millions of students confined to their homes. This home confinement has affected students of all levels, including university students, and has forced faculties to adapt online teaching strategies. Thus, traditional classroom face-to-face teaching has suddenly been replaced by online classes. This has revealed particularly challenging for medical courses. For such purpose we have designed an online teaching proposal addressed to the Degree in Physiotherapy and the Double Degree in Nursing and Physiotherapy of the University of Jaén (Spain). The objective is to implement an online virtual teaching protocol through the use of Virtual Reality. For such a goal, the Leap Motion Controller (LMC) will be used to teach the neuroanatomy of the brain and spinal cord and to teach and practice neurorehabilitation exercises. Along with devices like the LMC students will be asked to use Health Sciences databases in order to achieve a significative learning of the course topics. The project is structured in two phases. First, students will learn neuroanatomy and neurophysiology of the most relevant neurological conditions using LMC-based models. Then, they will learn to combine LMC games and conventional physiotherapy for neurorehabilitation purposes. The work of students will include the recording of videoreports demonstrating the acquisition of neuroanatomy concepts and simulating a clinical case. With this project we will assess the usability of LMC as an educative tool, the perception, satisfaction and self-regulated learning of physiotherapy students.

RehabHand: Oriented-tasks serious games for upper limb rehabilitation by using Leap Motion Controller and target population in spinal cord injury

BACKGROUND

There is a growing interest in the use of technology in the field of neurorehabilitation in order to quantify and generate knowledge about sensorimotor disorders after neurological diseases, understanding that the technology has a high potential for its use as therapeutic tools. Taking into account that the rehabilitative process of motor disorders should extend beyond the inpatient condition, it's necessary to involve low-cost technology, in order to have technological solutions that can approach the outpatient period at home.

OBJECTIVE

to present the virtual applications-based RehabHand prototype for the rehabilitation of manipulative skills of the upper limbs in patients with neurological conditions and to determine the target population with respect to spinal cord injured patients.

METHODS

Seven virtual reality applications have been designed and developed with a therapeutic sense, manipulated by means of Leap Motion Controller. The target population was determined from a sample of 40 people, healthy and patients, analyzing hand movements and gestures.

RESULTS

The hand movements and gestures were estimated with a fitting rate between the range 0.607-0.953, determining the target population by cervical levels and upper extremity motor score.

CONCLUSIONS

Leap Motion is suitable for a determined sample of cervical patients with a rehabilitation purpose.

A Systematic Review with Meta-Analysis

Leap Motion Controller (LMC) is a virtual reality device that can be used in the rehabilitation of central nervous system disease (CNSD) motor impairments. This review aimed to evaluate the effect of video game-based therapy with LMC on the recovery of upper extremity (UE) motor function in patients with CNSD. A systematic review with meta-analysis was performed in PubMed Medline, Web of Science, Scopus, CINAHL, and PEDro. We included five randomized controlled trials (RCTs) of patients with CNSD in which LMC was used as experimental therapy compared to conventional therapy (CT) to restore UE motor function. Pooled effects were estimated with Cohen's standardized mean difference (SMD) and its 95% confidence interval (95% CI). At first, in patients with stroke, LMC showed low-quality evidence of a large effect on UE mobility (SMD = 0.96; 95% CI = 0.47, 1.45). In combination with CT, LMC showed very low-quality evidence of a large effect on UE mobility (SMD = 1.34; 95% CI = 0.49, 2.19) and the UE mobility-oriented task (SMD = 1.26; 95% CI = 0.42, 2.10). Second, in patients with non-acute CNSD (cerebral palsy, multiple sclerosis, and Parkinson's disease), LMC showed low-quality evidence of a medium effect on grip strength (GS) (SMD = 0.47; 95% CI = 0.03, 0.90) and on gross motor dexterity (GMD) (SMD = 0.73; 95% CI = 0.28, 1.17) in the most affected UE. In combination with CT, LMC showed very low-quality evidence of a high effect in the most affected UE on GMD (SMD = 0.80; 95% CI = 0.06, 1.15) and fine motor dexterity (FMD) (SMD = 0.82; 95% CI = 0.07, 1.57). In stroke, LMC improved UE mobility and UE mobility-oriented tasks, and in non-acute CNSD, LMC improved the GS and GMD of the most affected UE and FMD when it was used with CT.

Hand Motion Capture from a 3D Leap Motion Controller for a Musculoskeletal Dynamic Simulation

The AnyBody Modeling System™ (AMS) is a musculoskeletal software simulation solution using inverse dynamics analysis. It enables the determination of muscle and joint forces for a given bodily motion. The recording of the individual movement and the transfer into the AMS is a complex and protracted process. Researches indicated that the contactless, visual Leap Motion Controller (LMC) provides clinically meaningful motion data for hand tracking. Therefore, the aim of this study was to integrate the LMC hand motion data into the AMS in order to improve the process of recording a hand movement. A Python-based interface between the LMC and the AMS, termed ROSE Motion, was developed. This solution records and saves the data of the movement as Biovision Hierarchy (BVH) data and AnyScript vector files that are imported into the AMS simulation. Setting simulation parameters, initiating the calculation automatically, and fetching results is implemented by using the AnyPyTools library from AnyBody. The proposed tool offers a rapid and easy-to-use recording solution for elbow, hand, and finger movements. Features include animation, cutting/editing, exporting the motion, and remote controlling the AMS for the analysis and presentation of musculoskeletal simulation results. Comparing the motion tracking results with previous studies, covering problems when using the LMC limit the correctness of the motion data. However, fast experimental setup and intuitive and rapid motion data editing strengthen the use of marker less systems as the herein presented compared to marker based motion capturing.

Development of a Diagnosis and Evaluation System for Hemiplegic Patients Post-Stroke Based on Motion Recognition Tracking and Analysis of Wrist Joint Kinematics

An inexperienced therapist lacks the analysis of a patient's movement. In addition, the patient does not receive objective feedback from the therapist due to the visual subjective judgment. The aim is to provide a guide for in-depth rehabilitation therapy in virtual space by continuously tracking the user's wrist joint during Leap Motion Controller (LMC) activities and present the basic data to confirm steady therapy results in real-time. The conventional Box and Block Test (BBT) is commonly used in upper extremity rehabilitation therapy. It was modeled in proportion to the actual size and Auto Desk Inventor was used to perform the 3D modeling work. The created 3D object was then implemented in C # through Unity5.6.2p4 based on LMC. After obtaining a wrist joint motion value, the motion was analyzed by 3D graph. Healthy subjects (23 males and 25 females, n = 48) were enrolled in this study. There was no statistically significant counting difference between conventional BBT and system BBT. This indicates the possibility of effective diagnosis and evaluation of hemiplegic patients post-stroke. We can keep track of wrist joints, check real-time continuous feedback in the implemented virtual space, and provide the basic data for an LMC-based quantitative rehabilitation therapy guide.

Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller

Modern hardware and software development has led to an evolution of user interfaces from command-line to natural user interfaces for virtual immersive environments. Gestures imitating real-world interaction tasks increasingly replace classical two-dimensional interfaces based on Windows/Icons/Menus/Pointers (WIMP) or touch metaphors. Thus, the purpose of this paper is to survey the state-of-the-art Human-Computer Interaction (HCI) techniques with a focus on the special field of three-dimensional interaction. This includes an overview of currently available interaction devices, their applications of usage and underlying methods for gesture design and recognition. Focus is on interfaces based on the Leap Motion Controller (LMC) and corresponding methods of gesture design and recognition. Further, a review of evaluation methods for the proposed natural user interfaces is given.

Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality

Coronary heart disease preoperative diagnosis plays an important role in the treatment of vascular interventional surgery. Actually, most doctors are used to diagnosing the position of the vascular stenosis and then empirically estimating vascular stenosis by selective coronary angiography images instead of using mouse, keyboard and computer during preoperative diagnosis. The invasive diagnostic modality is short of intuitive and natural interaction and the results are not accurate enough. Aiming at above problems, the coronary heart disease preoperative gesture interactive diagnostic system based on Augmented Reality is proposed. The system uses Leap Motion Controller to capture hand gesture video sequences and extract the features which that are the position and orientation vector of the gesture motion trajectory and the change of the hand shape. The training planet is determined by K-means algorithm and then the effect of gesture training is improved by multi-features and multi-observation sequences for gesture training. The reusability of gesture is improved by establishing the state transition model. The algorithm efficiency is improved by gesture prejudgment which is used by threshold discriminating before recognition. The integrity of the trajectory is preserved and the gesture motion space is extended by employing space rotation transformation of gesture manipulation plane. Ultimately, the gesture recognition based on SRT-HMM is realized. The diagnosis and measurement of the vascular stenosis are intuitively and naturally realized by operating and measuring the coronary artery model with augmented reality and gesture interaction techniques. All of the gesture recognition experiments show the distinguish ability and generalization ability of the algorithm and gesture interaction experiments prove the availability and reliability of the system.