Measuring tactile sensitivity and mixed-reality-assisted exercise for carpal tunnel syndrome by ultrasound mid-air haptics

Introduction

Carpal tunnel syndrome (CTS) is the most common nerve entrapment neuropathy, which causes numbness and pain in the thumb, the index and middle fingers and the radial side of the ring finger. Regular hand exercises may improve the symptoms and prevent carpal tunnel surgery. This study applied a novel ultrasonic stimulation method to test tactile sensitivity in CTS and also a mixed-reality-assisted (MR-assisted) exercise program which measured hand movements and provided haptic feedback for rehabilitation.

Methods

Twenty patients with mild unilateral CTS took part in the experiments. A mid-air haptics device (Ultrahaptics STRATOS Explore) was used to apply amplitude-modulated ultrasound waves (carrier frequency: 40 kHz) onto the skin to create tactile stimulation mechanically. Participants performed a two-alternative forced-choice task for measuring tactile thresholds at 250-Hz modulation frequency. They were tested at the index fingers and the thenar eminences of both hands. Additionally, 15 CTS patients used an MR-assisted program to do hand exercises with haptic feedback. Exercise performance was assessed by calculating errors between target and actual hand configurations. System Usability Scale (SUS) was adopted to verify the practical usability of the program.

Results

Thresholds at the thenar eminences of the affected and healthy hands were not significantly different. While the thresholds at the healthy index fingers could be measured, those of the affected fingers were all higher than the stimulation level produced by the maximum output from the ultrasound device. In the exercise program, a significant positive correlation (ρ = 0.89, p < 0.001) was found between the performance scores and the SUS scores, which were above the criterion value established in the literature.

Discussion

The results show that thenar tactile sensitivity is not affected in mild CTS as expected from the palmar cutaneous branch of the median nerve (PCBm), but index finger threshold is likely to be higher. Overall, this study suggests that mid-air haptics, with certain improvements, may be used as a preliminary test in the clinical setting. Moreover, the device is promising to develop gamified rehabilitation programs and for the treatment follow-up of CTS.

Orientation representation in human visual cortices: contributions of non-visual information and action-related process

Orientation processing in the human brain plays a crucial role in guiding grasping actions toward an object. Remarkably, despite the absence of visual input, the human visual cortex can still process orientation information. Instead of visual input, non-visual information, including tactile and proprioceptive sensory input from the hand and arm, as well as feedback from action-related processes, may contribute to orientation processing. However, the precise mechanisms by which the visual cortices process orientation information in the context of non-visual sensory input and action-related processes remain to be elucidated. Thus, our study examined the orientation representation within the visual cortices by analyzing the blood-oxygenation-level-dependent (BOLD) signals under four action conditions: direct grasp (DG), air grasp (AG), non-grasp (NG), and uninformed grasp (UG). The images of the cylindrical object were shown at +45° or - 45° orientations, corresponding to those of the real object to be grasped with the whole-hand gesture. Participants judged their orientation under all conditions. Grasping was performed without online visual feedback of the hand and object. The purpose of this design was to investigate the visual areas under conditions involving tactile feedback, proprioception, and action-related processes. To address this, a multivariate pattern analysis was used to examine the differences among the cortical patterns of the four action conditions in orientation representation by classification. Overall, significant decoding accuracy over chance level was discovered for the DG; however, during AG, only the early visual areas showed significant accuracy, suggesting that the object's tactile feedback influences the orientation process in higher visual areas. The NG showed no statistical significance in any area, indicating that without the grasping action, visual input does not contribute to cortical pattern representation. Interestingly, only the dorsal and ventral divisions of the third visual area (V3d and V3v) showed significant decoding accuracy during the UG despite the absence of visual instructions, suggesting that the orientation representation was derived from action-related processes in V3d and visual recognition of object visualization in V3v. The processing of orientation information during non-visually guided grasping of objects relies on other non-visual sources and is specifically divided by the purpose of action or recognition.

A 3D-Printed Soft Haptic Device with Built-in Force Sensing Delivering Bio-Mimicked Feedback

Haptics plays a significant role not only in the rehabilitation of neurological disorders, such as stroke, by substituting necessary cognitive information but also in human-computer interfaces (HCIs), which are now an integral part of the recently launched metaverse. This study proposes a unique, soft, monolithic haptic feedback device (SoHapS) that was directly manufactured using a low-cost and open-source fused deposition modeling (FDM) 3D printer by employing a combination of soft conductive and nonconductive thermoplastic polyurethane (TPU) materials (NinjaTek, USA). SoHapS consists of a soft bellow actuator and a soft resistive force sensor, which are optimized using finite element modeling (FEM). SoHapS was characterized both mechanically and electrically to assess its performance, and a dynamic model was developed to predict its force output with given pressure inputs. We demonstrated the efficacy of SoHapS in substituting biofeedback with tactile feedback, such as gripping force, and proprioceptive feedback, such as finger flexion-extension positions, in the context of teleoperation. With its intrinsic properties, SoHapS can be integrated into rehabilitation robots and robotic prostheses, as well as augmented, virtual, and mixed reality (AR/VR/MR) systems, to induce various types of bio-mimicked feedback.

Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis

Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.

Deep Inspiratory Breath-Hold Radiation for Left-Sided Breast Cancer using Novel Frame-based Tactile Feedback

A frame providing tactile feedback for the reproducibility of deep inspiratory breath-hold (DIBH) is described. The frame, fitted across the patient, comprises a horizontal bar, parallel to the patient's long axis, and holds a graduated pointer perpendicular to it. The pointer provides individualized tactile feedback for reproducibility of DIBH. Within the pointer is a movable pencil, bearing a 5 mm coloured strip which becomes visible only during DIBH, and acts as a visual cue to the therapist. The average variation in separation in the planning and pretreatment cone-beam computed tomography of 10 patients was 2 mm (confidence interval 1.95-2.05). Frame-based tactile feedback is a novel, reproducible technique for DIBH.

Perception Accuracy of a Multi-Channel Tactile Feedback System for Assistive Technology

Assistive technology uses multi-modal feedback devices, focusing on the visual, auditory, and haptic modalities. Tactile devices provide additional information via touch sense. Perception accuracy of vibrations depends on the spectral and temporal attributes of the signal, as well as on the body parts they are attached to. The widespread use of AR/VR devices, wearables, and gaming interfaces requires information about the usability of feedback devices. This paper presents results of an experiment using an 8-channel tactile feedback system with vibrators placed on the wrists, arms, ankles, and forehead. Different vibration patterns were designed and presented using sinusoidal frequency bursts on 2, 4, and 8 channels. In total, 27 subjects reported their sensation formally and informally on questionnaires. Results indicate that 2 and 4 channels could be used simultaneously with high accuracy, and the transducers' optimal placement (best sensitivity) is on the wrists, followed by the ankles. Arm and head positions were inferior and generally inadequate for signal presentation. For optimal performance, signal length should exceed 500 ms. Furthermore, the amplitude level and temporal pattern of the presented signals have to be used for carrying information rather than the frequency of the vibration.

High-Performance Flexible Pressure Sensor with a Self-Healing Function for Tactile Feedback

High-performance flexible pressure sensors have attracted a great deal of attention, owing to its potential applications such as human activity monitoring, man-machine interaction, and robotics. However, most high-performance flexible pressure sensors are complex and costly to manufacture. These sensors cannot be repaired after external mechanical damage and lack of tactile feedback applications. Herein, a high-performance flexible pressure sensor based on MXene/polyurethane (PU)/interdigital electrodes is fabricated by using a low-cost and universal spray method. The sprayed MXene on the spinosum structure PU and other arbitrary flexible substrates (represented by polyimide and membrane filter) act as the sensitive layer and the interdigital electrodes, respectively. The sensor shows an ultrahigh sensitivity (up to 509.8 kPa^-1 ), extremely fast response speed (67.3 ms), recovery speed (44.8 ms), and good stability (10 000 cycles) due to the interaction between the sensitive layer and the interdigital electrodes. In addition, the hydrogen bond of PU endows the device with the self-healing function. The sensor can also be integrated with a circuit, which can realize tactile feedback function. This MXene-based high-performance pressure sensor, along with its designing/fabrication, is expected to be widely used in human activity detection, electronic skin, intelligent robots, and many other aspects.

Design of multi-pad electrotactile system envisioned as a feedback channel for supernumerary robotic limbs

BACKGROUND

Providing real-time haptic feedback is an important, but still not sufficiently explored aspect of use of supernumerary robotic limbs (SRLs). We present a multi-pad electrode for conveying multi-modal proprioceptive and sensory information from SRL to the user's thigh and propose a method for stimuli calibration.

METHODS

Within two pilot tests we investigated return electrode configuration and active electrode discrimination in three healthy subjects to select the appropriate electrode pad topology. Based on the obtained results and anthropometric data from literature, the electrode was designed to have three branches of 10 pads and two additional pads that can be displaced over/under the electrode branches. The electrode was designed to be connected to the stimulator that allows full multiplexing so that specific branches can serve as common return electrode. To define the procedure for application of this system, the sensation, localization and discomfort thresholds applicable for the novel electrode were determined and analysed in ten subjects.

RESULTS

The results showed no overlaps between the three thresholds for individual pads, with significantly different average values, suggesting that the selected electrode positioning and design provide good active range of useful current amplitude. The results of the subsequent analysis suggested that the stimuli intensity level of 200% of sensation threshold is the most probable value of the localization threshold. Furthermore, this level ensures low chance (i.e. 0.7%) of reaching the discomfort.

CONCLUSIONS

We believe that envisioned electrotactile system could serve as a high bandwidth feedback channel that can be easily setup to provide proprioceptive and sensory feedback from supernumerary limbs.

Motor performance in violin bowing: Effects of attentional focus on acoustical, physiological and physical parameters of a sound-producing action

Violin bowing is a specialised sound-producing action, which may be affected by psychological performance techniques. In sport, attentional focus impacts motor performance, but limited evidence for this exists in music. We investigated effects of attentional focus on acoustical, physiological, and physical parameters of violin bowing in experienced and novice violinists. Attentional focus significantly affected spectral centroid, bow contact point consistency, shoulder muscle activity, and novices' violin sway. Performance was most improved when focusing on tactile sensations through the bow (somatic focus), compared to sound (external focus) or arm movement (internal focus). Implications for motor performance theory and pedagogy are discussed.

Feel-Good Requirements: Neurophysiological and Psychological Design Criteria of Affective Touch for (Assistive) Robots

Previous research has shown the value of the sense of embodiment, i.e., being able to integrate objects into one's bodily self-representation, and its connection to (assistive) robotics. Especially, tactile interfaces seem essential to integrate assistive robots into one's body model. Beyond functional feedback, such as tactile force sensing, the human sense of touch comprises specialized nerves for affective signals, which transmit positive sensations during slow and low-force tactile stimulations. Since these signals are extremely relevant for body experience as well as social and emotional contacts but scarcely considered in recent assistive devices, this review provides a requirement analysis to consider affective touch in engineering design. By analyzing quantitative and qualitative information from engineering, cognitive psychology, and neuroscienctific research, requirements are gathered and structured. The resulting requirements comprise technical data such as desired motion or force/torque patterns and an evaluation of potential stimulation modalities as well as their relations to overall user experience, e.g., pleasantness and realism of the sensations. This review systematically considers the very specific characteristics of affective touch and the corresponding parts of the neural system to define design goals and criteria. Based on the analysis, design recommendations for interfaces mediating affective touch are derived. This includes a consideration of biological principles and human perception thresholds which are complemented by an analysis of technical possibilities. Finally, we outline which psychological factors can be satisfied by the mediation of affective touch to increase acceptance of assistive devices and outline demands for further research and development.

Exploring Breaks in Sedentary Behavior of Older Adults Immediately After Receiving Personalized Haptic Feedback: Intervention Study

BACKGROUND

"Push" components of mobile health interventions may be promising to create conscious awareness of habitual sedentary behavior; however, the effect of these components on the near-time, proximal outcome, being breaks in sedentary behavior immediately after receiving a push notification, is still unknown, especially in older adults.

OBJECTIVE

The aims of this study are to examine if older adults break their sedentary behavior immediately after receiving personalized haptic feedback on prolonged sedentary behavior and if the percentage of breaks differs depending on the time of the day when the feedback is provided.

METHODS

A total of 26 Flemish older adults (mean age 64.4 years, SD 3.8) wore a triaxial accelerometer (Activator, PAL Technologies Ltd) for 3 weeks. The accelerometer generated personalized haptic feedback by means of vibrations each time a participant sat for 30 uninterrupted minutes. Accelerometer data on sedentary behavior were used to estimate the proximal outcome, which was sedentary behavior breaks immediately (within 1, 3, and 5 minutes) after receiving personalized haptic feedback. Generalized estimating equations were used to investigate whether or not participants broke up their sedentary behavior immediately after receiving haptic feedback. A time-related variable was added to the model to investigate if the sedentary behavior breaks differed depending on the time of day.

RESULTS

A total of 2628 vibrations were provided to the participants during the 3-week intervention period. Of these 2628 vibrations, 379 (14.4%), 570 (21.7%), and 798 (30.4%) resulted in a sedentary behavior break within 1, 3 and 5 minutes, respectively. Although the 1-minute interval did not reveal significant differences in the percentage of breaks depending on the time at which the haptic feedback was provided, the 3- and 5-minute intervals did show significant differences in the percentage of breaks depending on the time at which the haptic feedback was provided. Concretely, the percentage of sedentary behavior breaks was significantly higher if personalized haptic feedback was provided between noon and 3 PM compared to if the feedback was provided between 6 and 9 AM (odds ratio 1.58, 95% CI 1.01-2.47, within 3 minutes; odds ratio 1.78, 95% CI 1.11-2.84, within 5 minutes).

CONCLUSIONS

The majority of haptic vibrations, especially those in the morning, did not result in a break in the sedentary behavior of older adults. As such, simply bringing habitual sedentary behavior into conscious awareness seems to be insufficient to target sedentary behavior. More research is needed to optimize push components in interventions aimed at the reduction of the sedentary behavior of older adults.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04003324; https://clinicaltrials.gov/ct2/show/NCT04003324.

A Systematic Review of Commercial Smart Gloves: Current Status and Applications

Smart gloves have been under development during the last 40 years to support human-computer interaction based on hand and finger movement. Despite the many devoted efforts and the multiple advances in related areas, these devices have not become mainstream yet. Nevertheless, during recent years, new devices with improved features have appeared, being used for research purposes too. This paper provides a review of current commercial smart gloves focusing on three main capabilities: (i) hand and finger pose estimation and motion tracking, (ii) kinesthetic feedback, and (iii) tactile feedback. For the first capability, a detailed reference model of the hand and finger basic movements (known as degrees of freedom) is proposed. Based on the PRISMA guidelines for systematic reviews for the period 2015-2021, 24 commercial smart gloves have been identified, while many others have been discarded because they did not meet the inclusion criteria: currently active commercial and fully portable smart gloves providing some of the three main capabilities for the whole hand. The paper reviews the technologies involved, main applications and it discusses about the current state of development. Reference models to support end users and researchers comparing and selecting the most appropriate devices are identified as a key need.

Temporal Asynchrony but Not Total Energy Nor Duration Improves the Judgment of Numerosity in Electrotactile Stimulation

Stroke patients suffer from impairments of both motor and somatosensory functions. The functional recovery of upper extremities is one of the primary goals of rehabilitation programs. Additional somatosensory deficits limit sensorimotor function and significantly affect its recovery after the neuromotor injury. Sensory substitution systems, providing tactile feedback, might facilitate manipulation capability, and improve patient's dexterity during grasping movements. As a first step toward this aim, we evaluated the ability of healthy subjects in exploiting electrotactile feedback on the shoulder to determine the number of perceived stimuli in numerosity judgment tasks. During the experiment, we compared four different stimulation patterns (two simultaneous: short and long, intermittent and sequential) differing in total duration, total energy, or temporal synchrony. The experiment confirmed that the subject ability to enumerate electrotactile stimuli decreased with increasing the number of active electrodes. Furthermore, we found that, in electrotactile stimulation, the temporal coding schemes, and not total energy or duration modulated the accuracy in numerosity judgment. More precisely, the sequential condition resulted in significantly better numerosity discrimination than intermittent and simultaneous stimulation. These findings, together with the fact that the shoulder appeared to be a feasible stimulation site to communicate tactile information via electrotactile feedback, can serve as a guide to deliver tactile feedback to proximal areas in stroke survivors who lack sensory integrity in distal areas of their affected arm, but retain motor skills.

Stroke Affected Lower Limbs Rehabilitation Combining Virtual Reality With Tactile Feedback

In our study, we tested a combination of virtual reality (VR) and robotics in the original adjuvant method of post-stroke lower limb walk restoration in acute phase using a simulation with visual and tactile biofeedback based on VR immersion and physical impact to the soles of patients. The duration of adjuvant therapy was 10 daily sessions of 15 min each. The study showed the following significant rehabilitation progress in Control (N = 27) vs. Experimental (N = 35) groups, respectively: 1.56 ± 0.29 (mean ± SD) and 2.51 ± 0.31 points by Rivermead Mobility Index (p = 0.0286); 2.15 ± 0.84 and 6.29 ± 1.20 points by Fugl-Meyer Assessment Lower Extremities scale (p = 0.0127); and 6.19 ± 1.36 and 13.49 ± 2.26 points by Berg Balance scale (p = 0.0163). P-values were obtained by the Mann-Whitney U test. The simple and intuitive mechanism of rehabilitation, including through the use of sensory and semantic components, allows the therapy of a patient with diaschisis and afferent and motor aphasia. Safety of use allows one to apply the proposed method of therapy at the earliest stage of a stroke. We consider the main finding of this study that the application of rehabilitation with implicit interaction with VR environment produced by the robotics action has measurable significant influence on the restoration of the affected motor function of the lower limbs compared with standard rehabilitation therapy.

Using Bionics to Restore Sensation to Reconstructed Breasts

Mastectomy often leads to a complete desensitization of the chest, which in turn can give rise to diminished sexual function and to disembodiment of the breasts. One approach to mitigate the sensory consequences of mastectomy is to leverage technology that has been developed for the restoration of sensation in bionic hands. Specifically, sensors embedded under the skin of the nipple-areolar complex can be used to detect touches. The output of the sensors then drives electrical stimulation of the residual intercostal nerves, delivered through chronically implanted electrode arrays, thereby eliciting tactile sensations experienced on the nipple-areolar complex. The hope is that the bionic breast will restore a woman's sense that her breast belongs to her body so she can experience the pleasure of an embrace and derive the benefit of the sensual touch of her partner.

Tactile Feedback in Closed-Loop Control of Myoelectric Hand Grasping: Conveying Information of Multiple Sensors Simultaneously via a Single Feedback Channel

The appropriate sensory information feedback is important for the success of an object grasping and manipulation task. In many scenarios, the need arises for multiple feedback information to be conveyed to a prosthetic hand user simultaneously. The multiple sets of information may either (1) directly contribute to the performance of the grasping or object manipulation task, such as the feedback of the grasping force, or (2) simply form additional independent set(s) of information. In this paper, the efficacy of simultaneously conveying two independent sets of sensor information (the grasp force and a secondary set of information) through a single channel of feedback stimulation (vibrotactile via bone conduction) to the human user in a prosthetic application is investigated. The performance of the grasping task is not dependent to the second set of information in this study. Subject performance in two tasks: regulating the grasp force and identifying the secondary information, were evaluated when provided with either one corresponding information or both sets of feedback information. Visual feedback is involved in the training stage. The proposed approach is validated on human-subject experiments using a vibrotactile transducer worn on the elbow bony landmark (to realize a non-invasive bone conduction interface) carried out in a virtual reality environment to perform a closed-loop object grasping task. The experimental results show that the performance of the human subjects on either task, whilst perceiving two sets of sensory information, is not inferior to that when receiving only one set of corresponding sensory information, demonstrating the potential of conveying a second set of information through a bone conduction interface in an upper limb prosthetic task.

Grip Stabilization through Independent Finger Tactile Feedback Control

Grip force control during robotic in-hand manipulation is usually modeled as a monolithic task, where complex controllers consider the placement of all fingers and the contact states between each finger and the gripped object in order to compute the necessary forces to be applied by each finger. Such approaches normally rely on object and contact models and do not generalize well to novel manipulation tasks. Here, we propose a modular grip stabilization method based on a proposition that explains how humans achieve grasp stability. In this biomimetic approach, independent tactile grip stabilization controllers ensure that slip does not occur locally at the engaged robot fingers. Local slip is predicted from the tactile signals of each fingertip sensor i.e., BioTac and BioTac SP by Syntouch. We show that stable grasps emerge without any form of central communication when such independent controllers are engaged in the control of multi-digit robotic hands. The resulting grasps are resistant to external perturbations while ensuring stable grips on a wide variety of objects.

Double-Diamond Model-Based Orientation Guidance in Wearable Human-Machine Navigation Systems for Blind and Visually Impaired People

This paper presents the analysis and design of a new, wearable orientation guidance device in modern travel aid systems for blind and visually impaired people. The four-stage double-diamond design model was applied in the design process to achieve human-centric innovation and to ensure technical feasibility and economic viability. Consequently, a sliding tactile feedback wristband was designed and prototyped. Furthermore, a Bezier curve-based adaptive path planner is proposed to guarantee collision-free planned motion. Proof-of-concept experiments on both virtual and real-world scenarios are conducted. The evaluation results confirmed the efficiency and feasibility of the design and imply the design’s remarkable potential in spatial perception rehabilitation.

Bilateral Tactile Feedback-Enabled Training for Stroke Survivors Using Microsoft KinectTM

Rehabilitation and mobility training of post-stroke patients is crucial for their functional recovery. While traditional methods can still help patients, new rehabilitation and mobility training methods are necessary to facilitate better recovery at lower costs. In this work, our objective was to design and develop a rehabilitation training system targeting the functional recovery of post-stroke users with high efficiency. To accomplish this goal, we applied a bilateral training method, which proved to be effective in enhancing motor recovery using tactile feedback for the training. One participant with hemiparesis underwent six weeks of training. Two protocols, “contralateral arm matching” and “both arms moving together”, were carried out by the participant. Each of the protocols consisted of “shoulder abduction” and “shoulder flexion” at angles close to 30 and 60 degrees. The participant carried out 15 repetitions at each angle for each task. For example, in the “contralateral arm matching” protocol, the unaffected arm of the participant was set to an angle close to 30 degrees. He was then requested to keep the unaffected arm at the specified angle while trying to match the position with the affected arm. Whenever the two arms matched, a vibration was given on both brachialis muscles. For the “both arms moving together” protocol, the two arms were first set approximately to an angle of either 30 or 60 degrees. The participant was asked to return both arms to a relaxed position before moving both arms back to the remembered specified angle. The arm that was slower in moving to the specified angle received a vibration. We performed clinical assessments before, midway through, and after the training period using a Fugl-Meyer assessment (FMA), a Wolf motor function test (WMFT), and a proprioceptive assessment. For the assessments, two ipsilateral and contralateral arm matching tasks, each consisting of three movements (shoulder abduction, shoulder flexion, and elbow flexion), were used. Movements were performed at two angles, 30 and 60 degrees. For both tasks, the same procedure was used. For example, in the case of the ipsilateral arm matching task, an experimenter positioned the affected arm of the participant at 30 degrees of shoulder abduction. The participant was requested to keep the arm in that position for ~5 s before returning to a relaxed initial position. Then, after another ~5-s delay, the participant moved the affected arm back to the remembered position. An experimenter measured this shoulder abduction angle manually using a goniometer. The same procedure was repeated for the 60 degree angle and for the other two movements. We applied a low-cost Kinect to extract the participant’s body joint position data. Tactile feedback was given based on the arm position detected by the Kinect sensor. By using a Kinect sensor, we demonstrated the feasibility of the system for the training of a post-stroke user. The proposed system can further be employed for self-training of patients at home. The results of the FMA, WMFT, and goniometer angle measurements showed improvements in several tasks, suggesting a positive effect of the training system and its feasibility for further application for stroke survivors’ rehabilitation.

Fingerpad contact evolution under electrovibration

Displaying tactile feedback through a touchscreen via electrovibration has many potential applications in mobile devices, consumer electronics, home appliances and automotive industry though our knowledge and understanding of the underlying contact mechanics are very limited. An experimental study was conducted to investigate the contact evolution between the human finger and a touch screen under electrovibration using a robotic set-up and an imaging system. The results show that the effect of electrovibration is only present during full slip but not before slip. Hence, the coefficient of friction increases under electrovibration as expected during full slip, but the apparent contact area is significantly smaller during full slip when compared to that of no electrovibration condition. It is suggested that the main cause of the increase in friction during full slip is due to an increase in the real contact area and the reduction in apparent area is due to stiffening of the finger skin in the tangential direction.

Coordinative Motion-Based Bilateral Rehabilitation Training System with Exoskeleton and Haptic Devices for Biomedical Application

According to the neuro-rehabilitation theory, compared with unilateral training, bilateral training is proven to be an effective method for hemiparesis, which affects the most part of stroke patients. In this study, a novel bilateral rehabilitation training system, which incorporates a lightweight exoskeleton device worn on the affected limb; a haptic device (Phantom Premium), which is used for generating a desired tactile feedback for the affected limb; and a VR (virtual reality) graphic interface, has been developed. The use of VR technology during rehabilitation can provide goal directed tasks with rewards and motivate the patient to undertake extended rehabilitation. This paper is mainly focused on elbow joint training, and other independent joints can be trained by easily changing the VR training interface. The haptic device is adopted to enable patients to use their own decision making abilities with a tactical feedback. Integrated with a VR-based graphic interface, the goal-oriented task can help to gradually recovery their motor function with a coordinative motion between two limbs. In particular, the proposed system can accelerate neural plasticity and motor recovery in those patients with little muscle strength by using the exoskeleton device. The exoskeleton device can provide from relatively high joint impedance to near-zero impedance, and can provide a partial assist as the patient requires.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices – such as prostheses, orthoses, exoskeletons, and devices for teleoperation–often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Cutaneous afferent innervation of the human foot sole: what can we learn from single-unit recordings?

Cutaneous afferents convey exteroceptive information about the interaction of the body with the environment and proprioceptive information about body position and orientation. Four classes of low-threshold mechanoreceptor afferents innervate the foot sole and transmit feedback that facilitates the conscious and reflexive control of standing balance. Experimental manipulation of cutaneous feedback has been shown to alter the control of gait and standing balance. This has led to a growing interest in the design of intervention strategies that enhance cutaneous feedback and improve postural control. The advent of single-unit microneurography has allowed the firing and receptive field characteristics of foot sole cutaneous afferents to be investigated. In this review, we consolidate the available cutaneous afferent microneurographic recordings from the foot sole and provide an analysis of the firing threshold, and receptive field distribution and density of these cutaneous afferents. This work enhances the understanding of the foot sole as a sensory structure and provides a foundation for the continued development of sensory augmentation insoles and other tactile enhancement interventions.

Updated Tactile Feedback with a Pin Array Matrix Helps Blind People to Reduce Self-Location Errors

Autonomous navigation in novel environments still represents a challenge for people with visual impairment (VI). Pin array matrices (PAM) are an effective way to display spatial information to VI people in educative/rehabilitative contexts, as they provide high flexibility and versatility. Here, we tested the effectiveness of a PAM in VI participants in an orientation and mobility task. They haptically explored a map showing a scaled representation of a real room on the PAM. The map further included a symbol indicating a virtual target position. Then, participants entered the room and attempted to reach the target three times. While a control group only reviewed the same, unchanged map on the PAM between trials, an experimental group also received an updated map representing, in addition, the position they previously reached in the room. The experimental group significantly improved across trials by having both reduced self-location errors and reduced completion time, unlike the control group. We found that learning spatial layouts through updated tactile feedback on programmable displays outperforms conventional procedures on static tactile maps. This could represent a powerful tool for navigation, both in rehabilitation and everyday life contexts, improving spatial abilities and promoting independent living for VI people.

Human's Capability to Discriminate Spatial Forces at the Big Toe

A key factor for reliable object manipulation is the tactile information provided by the skin of our hands. As this sensory information is so essential in our daily life it should also be provided during teleoperation of robotic devices or in the control of myoelectric prostheses. It is well-known that feeding back the tactile information to the user can lead to a more natural and intuitive control of robotic devices. However, in some applications it is difficult to use the hands as natural feedback channels since they may already be overloaded with other tasks or, e.g., in case of hand prostheses not accessible at all. Many alternatives for tactile feedback to the human hand have already been investigated. In particular, one approach shows that humans can integrate uni-directional (normal) force feedback at the toe into their sensorimotor-control loop. Extending this work, we investigate the human's capability to discriminate spatial forces at the bare front side of their toe. A state-of-the-art haptic feedback device was used to apply forces with three different amplitudes-2 N, 5 N, and 8 N-to subjects' right big toes. During the experiments, different force stimuli were presented, i.e., direction of the applied force was changed, such that tangential components occured. In total the four directions up (distal), down (proximal), left (medial), and right (lateral) were tested. The proportion of the tangential force was varied corresponding to a directional change of 5° to 25° with respect to the normal force. Given these force stimuli, the subjects' task was to identify the direction of the force change. We found the amplitude of the force as well as the proportion of tangential forces to have a significant influence on the success rate. Furthermore, the direction right showed a significantly different successrate from all other directions. The stimuli with a force amplitude of 8 N achieved success rates over 89% in all directions. The results of the user study provide evidence that the subjects were able to discriminate spatial forces at their toe within defined force amplitudes and tangential proportion.

Highly Transparent and Integrable Surface Texture Change Device for Localized Tactile Feedback

Human-machine haptic interaction is typically detected by variations in friction, roughness, hardness, and temperature, which combines to create sensation of surface texture change. Most of the current technologies work to simulate changes in tactile perception (via electrostatic, lateral force fields, vibration motors, etc.) without creating actual topographical transformations. This makes it challenging to provide localized feedback. Here, a new concept for on-demand surface texture augmentation that is capable of physically forming local topographic features in any predesigned pattern is demonstrated. The transparent, flexible, integrable device comprises of a hybrid electrode system with conductive hydrogel, silver nanowires, and conductive polymers with acrylic elastomer as the dielectric layer. Desired surface textures can be controlled by a predesigned pattern of electrodes, which operates as independent or interconnected actuators. Surface features with up to a height of 0.155 mm can be achieved with a transformation time of less than a second for a device area of 18 cm² . High transparency levels of 76% are achieved due to the judicious choice of the electrode and the active elastomer layer. The capability of localized and controlled deformations makes this system highly useful for applications such as display touchscreens, touchpads, braille displays, on-demand buttons, and microfluidic devices.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices - such as prostheses, orthoses, exoskeletons, and devices for teleoperation-often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Instrumental tactile diagnostics in robot-assisted surgery

BACKGROUND

Robotic surgery has gained wide acceptance due to minimizing trauma in patients. However, the lack of tactile feedback is an essential limiting factor for the further expansion. In robotic surgery, feedback related to touch is currently kinesthetic, and it is mainly aimed at the minimization of force applied to tissues and organs. Design and implementation of diagnostic tactile feedback is still an open problem. We hypothesized that a sufficient tactile feedback in robot-assisted surgery can be provided by utilization of Medical Tactile Endosurgical Complex (MTEC), which is a novel specialized tool that is already commercially available in the Russian Federation. MTEC allows registration of tactile images by a mechanoreceptor, real-time visualization of these images, and reproduction of images via a tactile display.

MATERIALS AND METHODS

Nine elective surgeries were performed with da Vinci™ robotic system. An assistant performed tactile examination through an additional port under the guidance of a surgeon during revision of tissues. The operating surgeon sensed registered tactile data using a tactile display, and the assistant inspected the visualization of tactile data. First, surgeries where lesion boundaries were visually detectable were performed. The goal was to promote cooperation between the surgeon and the assistant and to train them in perception of the tactile feedback. Then, instrumental tactile diagnostics was utilized in case of visually undetectable boundaries.

RESULTS

In robot-assisted surgeries where lesion boundaries were not visually detectable, instrumental tactile diagnostics performed using MTEC provided valid identification and localization of lesions. The results of instrumental tactile diagnostics were concordant with the results of intraoperative ultrasound examination. However, in certain cases, for example, thoracoscopy, ultrasound examination is inapplicable, while MTEC-based tactile diagnostics can be efficiently utilized.

CONCLUSION

The study proved that MTEC can be efficiently used in robot-assisted surgery allowing correct localization of visually undetectable lesions and visually undetectable boundaries of pathological changes of tissues.

Tactile feedback improves auditory spatial localization

Our recent studies suggest that congenitally blind adults have severely impaired thresholds in an auditory spatial bisection task, pointing to the importance of vision in constructing complex auditory spatial maps (Gori et al., 2014). To explore strategies that may improve the auditory spatial sense in visually impaired people, we investigated the impact of tactile feedback on spatial auditory localization in 48 blindfolded sighted subjects. We measured auditory spatial bisection thresholds before and after training, either with tactile feedback, verbal feedback, or no feedback. Audio thresholds were first measured with a spatial bisection task: subjects judged whether the second sound of a three sound sequence was spatially closer to the first or the third sound. The tactile feedback group underwent two audio-tactile feedback sessions of 100 trials, where each auditory trial was followed by the same spatial sequence played on the subject’s forearm; auditory spatial bisection thresholds were evaluated after each session. In the verbal feedback condition, the positions of the sounds were verbally reported to the subject after each feedback trial. The no feedback group did the same sequence of trials, with no feedback. Performance improved significantly only after audio-tactile feedback. The results suggest that direct tactile feedback interacts with the auditory spatial localization system, possibly by a process of cross-sensory recalibration. Control tests with the subject rotated suggested that this effect occurs only when the tactile and acoustic sequences are spatially congruent. Our results suggest that the tactile system can be used to recalibrate the auditory sense of space. These results encourage the possibility of designing rehabilitation programs to help blind persons establish a robust auditory sense of space, through training with the tactile modality.

Role of combined tactile and kinesthetic feedback in minimally invasive surgery

BACKGROUND

Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery.

METHODS

To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments.

RESULTS

The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation.

CONCLUSIONS

The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd.

Conveying tactile feedback in sensorized hand neuroprostheses using a biofidelic model of mechanotransduction

One approach to conveying tactile feedback from sensorized neural prostheses is to characterize the neural signals that would normally be produced in an intact limb and reproduce them through electrical stimulation of the residual peripheral nerves. Toward this end, we have developed a model that accurately replicates the neural activity evoked by any dynamic stimulus in the three types of mechanoreceptive afferents that innervate the glabrous skin of the hand. The model takes as input the position of the stimulus as a function of time, along with its first (velocity), second (acceleration), and third (jerk) derivatives. This input is filtered and passed through an integrate-and-fire mechanism to generate a train of spikes as output. The major conclusion of this study is that the timing of individual spikes evoked in mechanoreceptive fibers innervating the hand can be accurately predicted by this model. We discuss how this model can be integrated in a sensorized prosthesis and show that the activity in a population of simulated afferents conveys information about the location, timing, and magnitude of contact between the hand and an object.