Grasping Embodiment: Haptic Feedback for Artificial Limbs

A Review of Embodied Grasping

Pre-trained models trained with internet-scale data have achieved significant improvements in perception, interaction, and reasoning. Using them as the basis of embodied grasping methods has greatly promoted the development of robotics applications. In this paper, we provide a comprehensive review of the latest developments in this field. First, we summarize the embodied foundations, including cutting-edge embodied robots, simulation platforms, publicly available datasets, and data acquisition methods, to fully understand the research focus. Then, the embodied algorithms are introduced, starting from pre-trained models, with three main research goals: (1) embodied perception, using data captured by visual sensors to perform point cloud extraction or 3D reconstruction, combined with pre-trained models, to understand the target object and external environment and directly predict the execution of actions; (2) embodied strategy: In imitation learning, the pre-trained model is used to enhance data or as a feature extractor to enhance the generalization ability of the model. In reinforcement learning, the pre-trained model is used to obtain the optimal reward function, which improves the learning efficiency and ability of reinforcement learning; (3) embodied agent: The pre-trained model adopts hierarchical or holistic execution to achieve end-to-end robot control. Finally, the challenges of the current research are summarized, and a perspective on feasible technical routes is provided.

Delivery of UK military upper limb prosthetics: current concepts and future directions

Upper limb prosthetics have a challenging task. A natural upper limb combines strength, coordination and dexterity to accomplish daily activities such as eating, writing, working and social interaction. Artificially replicating these functions requires a prosthetic with composite, synchronous motor function while maintaining sensory feedback and skeletal stability. Achieving these functions requires interfaces between biology and machine across nerve, muscle, bone and skin. This leads to issues related to infection, foreign material encapsulation and implant stability, and electrical signal transduction and interpretation. Over the last 20 years the advent of technologies such as osseointegration, targeted muscle reinnervation, implantable myoelectric sensors, peripheral nerve interfaces and pattern recognition technology has sought to address these problems.Due to many advances in prehospital care, truncated timelines to damage control surgery and improved combat personal protective equipment, the numbers of amputees have increased with more patients surviving injury. From October 2001 to March 2019 there were 333 amputees from Afghanistan and Iraq compared with 457 fatalities over a similar period. Over a third of these were significant multiple amputees. With a functional, robust upper limb prosthetic which mirrors or exceeds normal function, injured service personnel could be returned to an active combat role. This has benefits for their physical and mental health, improves employability prospects and allows Defence to retain some of its most highly motivated and skilled people who represent significant financial investment.

TouchMark: Partial Tactile Feedback Design for Upper Limb Rehabilitation in Virtual Reality

The use of Virtual Reality (VR) technology, especially in medical rehabilitation, has expanded to include tactile cues along with visual stimuli. For patients with upper limb hemiplegia, tangible handles with haptic stimuli could improve their ability to perform daily activities. Traditional VR controllers are unsuitable for patient rehabilitation in VR, necessitating the design of specialized tangible handles with integrated tracking devices. Besides, matching tactile stimulation with corresponding virtual visuals could strengthen users' embodiment (i.e., owning and controlling virtual bodies) in VR, which is crucial for patients' training with virtual hands. Haptic stimuli have been shown to amplify the embodiment in VR, whereas the effect of partial tactile stimulation from tangible handles on embodiment remains to be clarified. This research, including three experiments, aims to investigate how partial tactile feedback of tangible handles impacts users' embodiment, and we proposed a design concept called TouchMark for partial tactile stimuli that could help users quickly connect the physical and virtual worlds. To evaluate users' tactile and comfort perceptions when grasping tangible handles in a non-VR setting, various handles with three partial tactile factors were manipulated in Study 1. In Study 2, we explored the effects of partial feedback using three forms of TouchMark on the embodiment of healthy users in VR, with various tangible handles, while Study 3 focused on similar investigations with patients. These handles were utilized to complete virtual food preparation tasks. The tactile and comfort perceptions of tangible handles and users' embodiment were evaluated in this research using questionnaires and interviews. The results indicate that TouchMark with haptic line and ring forms over no stimulation would significantly enhance users' embodiment, especially for patients. The low-cost and innovative TouchMark approach may assist users, particularly those with limited VR experience, in achieving the embodiment and enhancing their virtual interactive experience.

Experimental Evaluation of a Hybrid Sensory Feedback System for Haptic and Kinaesthetic Perception in Hand Prostheses

This study proposes a new hybrid multi-modal sensory feedback system for prosthetic hands that can provide not only haptic and proprioceptive feedback but also facilitate object recognition without the aid of vision. Modality-matched haptic perception was provided using a mechanotactile feedback system that can proportionally apply the gripping force through the use of a force controller. A vibrotactile feedback system was also employed to distinguish four discrete grip positions of the prosthetic hand. The system performance was evaluated with a total of 32 participants in three different experiments (i) haptic feedback, (ii) proprioceptive feedback and (iii) object recognition with hybrid haptic-proprioceptive feedback. The results from the haptic feedback experiment showed that the participants' ability to accurately perceive applied force depended on the amount of force applied. As the feedback force was increased, the participants tended to underestimate the force levels, with a decrease in the percentage of force estimation. Of the three arm locations (forearm volar, forearm ventral and bicep), and two muscle states (relaxed and tensed) tested, the highest accuracy was obtained for the bicep location in the relaxed state. The results from the proprioceptive feedback experiment showed that participants could very accurately identify four different grip positions of the hand prosthesis (i.e., open hand, wide grip, narrow grip, and closed hand) without a single case of misidentification. In experiment 3, participants could identify objects with different shapes and stiffness with an overall high success rate of 90.5% across all combinations of location and muscle state. The feedback location and muscle state did not have a significant effect on object recognition accuracy. Overall, our study results indicate that the hybrid feedback system may be a very effective way to enrich a prosthetic hand user's experience of the stiffness and shape of commonly manipulated objects.

Evaluating Virtual Hand Illusion through Realistic Appearance and Tactile Feedback

We conducted a virtual reality study to explore virtual hand illusion through three levels of appearance (Appearance dimension: realistic vs. pixelated vs. toon hand appearances) and two levels of tactile feedback (Tactile dimension: no tactile vs. tactile feedback). We instructed our participants to complete a virtual assembly task in this study. Immediately afterward, we asked them to provide self-reported ratings on a survey that captured presence and five embodiment dimensions (hand ownership, touch sensation, agency and motor control, external appearance, and response to external stimuli). The results of our study indicate that (1) tactile feedback generated a stronger sense of presence, touch sensation, and response to external stimuli; (2) the pixelated hand appearance provided the least hand ownership and external appearance; and (3) in the presence of the pixelated hand, prior virtual reality experience of participants impacted their agency and motor control and their response to external stimuli ratings. This paper discusses our findings and provides design considerations for virtual reality applications with respect to the realistic appearance of virtual hands and tactile feedback.