Application of psychophysical techniques to haptic research

FW-Touch: A Finger Wearable Haptic Interface With an MR Foam Actuator for Displaying Surface Material Properties on a Touch Screen

The haptic interface plays an increasingly important role in enhancing the realism and immersion of the user's interaction with the touch screen. Inspired by the wearable haptic system, this paper proposes a finger wearable device called FW-Touch for touch screen interaction. The device provides normal force, lateral force, and vibrotactile feedback for the interaction of the finger and the touch screen through three internally integrated actuators. By displaying the hardness, friction, and roughness of a virtual surface, the device is capable of simulating the active exploration and sensing process of the finger on a real surface. This paper describes the design and specifications of the FW-Touch, and details the design process of a magnetorheological (MR) foam actuator that uses a Hall sensor to correct the output force. Through physical measurements and psychophysical experiments, we comprehensively evaluated the force feedback performance of the FW-Touch and its ability in displaying the stiffness and friction of the virtual surface. The results show that improving the accuracy of force feedback is necessary for virtual stiffness display, and the accuracy and effectiveness of the FW-Touch in displaying virtual surface features can be confirmed from the measured stiffness and friction Weber fractions.

Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion

In the real world, our bodies influence how we perceive ourselves and how others perceive us. Our body can also affect estimations of object sizes and distances. But how does our body affect our haptic experience? Here, we examined the modulation of a visuo-haptic illusion of touch on a virtual stick in virtual reality, when participants were embodied in an avatar and when they were not. During the experiments participants (n = 49) received successions of three taps delivered from two independent controllers while they saw visual stimuli presented sequentially along the virtual stick. The stimulation pattern resulted in a robust illusion of tapping directly on the virtual stick. After each trial, participants were asked to report where they perceived the taps. We found that participants in both the body and no-body conditions displaced the second tap toward the center of the stick, and reported similar levels of certainty about their reported location. However, the illusion of touch on the stick, as measured by the reported location of the tap, was significantly stronger for those who had a virtual body than those who did not. Therefore, our study shows that avatar embodiment can change haptic perception.

Unilateral and Bilateral Virtual Springs: Contact Transitions Unmask Device Dynamics

The study of haptic perception often makes use of haptic rendering to display the variety of impedances needed to run an experiment. Unacknowledged in many cases is the influence of the selected device hardware on what the user will feel, particularly in interactions featuring frequencies above the control bandwidth. While human motion is generally limited to 10 Hz, virtual environments with unilateral constraints are subject to excitation of a wider frequency spectrum through contact transitions. We employ the effective impedance decomposition to discuss the effects of parasitics outside the rendering bandwidth. We also introduce an analysis of the admittance and impedance controllers with respect to sensitivity to load cell noise. We explore these effects using a single degree-of-freedom device that can be configured for either a low or high mechanical advantage in a perceptual experiment, with experimental conditions designed through application of the effective impedance decomposition. We find that the excitation of high frequencies through contact transitions negatively impacts humans' ability to distinguish between stiffnesses.

The Perception of Ultrasonic Square Reductions of Friction With Variable Sharpness and Duration

The human perception of square ultrasonic modulation of the finger-surface friction was investigated during active tactile exploration by using short frictional cues of varying duration and sharpness. In a first experiment, we asked participants to discriminate the transition time and duration of short square ultrasonic reductions of friction. They proved very sensitive to discriminate millisecond differences in these two parameters with the average psychophysical thresholds being 2.3-2.4 ms for both parameters. A second experiment focused on the perception of square friction reductions with variable transition times and durations. We found that for durations of the stimulation larger than 90 ms, participants often perceived three or four edges when only two stimulations were presented while they consistently felt two edges for signals shorter than 50 ms. A subsequent analysis of the contact forces induced by these ultrasonic stimulations during slow and fast active exploration showed that two identical consecutive ultrasonic pulses can induce significantly different frictional dynamics especially during fast motion of the finger. These results confirm the human sensitivity to transient frictional cues and suggest that the human perception of square reductions of friction can depend on their sharpness and duration as well as on the speed of exploration.

Perception of Mechanical Impedance During Active Ankle and Knee Movement

During locomotion, energy flow through the legs is governed by the mechanical impedance of each joint. These mechanical properties, including stiffness and damping, have recently been quantified at the ankle joint. However, the relevance of these properties in human sensorimotor control is unclear. An important aspect of sensorimotor control is the ability to sense small changes in stimuli. Thus, we investigated the human ability to detect small changes in the stiffness and damping components of leg joint impedance when interacting with a mechanical system coupled to the ankle or knee. The perception threshold was determined via a psychophysical paradigm that required subjects to compare the mechanical impedance of virtual spring-mass-damper systems. Subjects reliably detected impedance changes of 11% and 12% at the ankle and knee, respectively. Additionally, the perception of stiffness and damping were comparable, indicating that the biomechanical relevance of the stiffness and damping components of impedance may be similar. Finally, these results offer novel insight into the design and control of impedance-based technologies, such as prostheses and exoskeletons.

Improving Haptic Transparency for Uncertain Virtual Environments Using Adaptive Control and Gain-Scheduled Prediction

The realism or transparency of haptic interfaces is becoming more critical as they are applied to training in fields like minimally invasive surgery (MIS). Surgical training simulators must provide a transparent virtual environment (VE) at a high update rate. Complex, deformable, cuttable tissue models have nonlinear dynamics and are computationally expensive, making it difficult to provide sufficient update rates. The objective of this work is to improve the transparency for this type of VE by formulating the unknown nonlinear dynamics as a quasi-linear parameter varying (LPV) system and designing a predictor to provide an output at a much higher update rate. An adaptive controller based on gain-scheduled prediction is considered for a nonlinear haptic device and a nonlinear, delayed, and sampled VE. The predictor uses feedback from the more accurate but slow-updating VE to update a simplified dynamic model. The predictor is designed based on numerical solutions to a linear matrix inequality derived using Lyapunov-based methods. Experimental results demonstrate the effectiveness of the gain-scheduled predictor approach and compare it to previous work using a constant-gain predictor. The gain-scheduled predictor results in significant performance improvements compared to a haptic system without prediction, but less significant improvement compared to the constant-gain approach.

MH-Pen: A Pen-Type Multi-Mode Haptic Interface for Touch Screens Interaction

Touch screen technology supplies a new approach to interact with virtual environments. For haptic interaction on a touch screen, haptic devices that are capable of simultaneously conveying tactile and force information to users are highly desired for enhancing the sense of reality and immersion. To this end, a prototype haptic interface, called MH-Pen, was developed and fabricated to display the virtual interactive information through multi-mode haptic feedback. The MH-Pen is a self-contained system that provides vibrotactile feedback and precise force feedback by integrating three types of actuators. In this paper, MH-Pen's design, specifications, and working principle are described. Subsequently, to accurately display the interaction force, a hybrid actuator was designed by combining a piston-type magnetorheological (MR) actuator and a voice coil motor (VCM), and a closed-loop control scheme was built to manage the hybrid actuator. Finally, we objectively and subjectively evaluated the force feedback performance and the effect of multi-mode haptic display of the MH-Pen through physical measurements and psychophysical experiments of virtual surface stiffness display. The results show that improving the precision of force feedback and using multi-mode haptic display are both useful and necessary to enhance the sense of human-computer interaction realism.

Perception of Ultrasonic Switches Involves Large Discontinuity of the Mechanical Impedance

The distinct tactile feedback provided by mechanical keyboards notifies users that their actions have been successfully recorded. The presence of these subtle yet informative tactile cues is one of the reasons why mechanical keyboards are still preferred to their virtual counterparts. An artificial sensation of pressing a mechanical switch can be produced by varying the coefficient of friction as the user is pressing down on a glass surface using ultrasonic vibration. We examined the factors involved in producing a vivid sensation of a stimulus by measuring the mechanical impedance, the frictional behavior of the fingertip and the perceptual thresholds. Subjects who experienced weaker sensations also showed a weaker sensitivity to friction modulation, which may in turn be attributable to the presence of a larger or a smaller than average impedance. In the second experiment, the user's finger impedance was measured during the click, and it was observed that the successful detection of the stimulus was correlated with the presence of considerable discontinuity in the mechanical impedance added to the plate by the finger. This discontinuity in the evolution of the impedance supports the idea that the skin is being reconfigured towards a new equilibrium state after the change in friction.

Prediction of Perceived Steering Wheel Operation Force by Muscle Activity

Humans feel forces or weights while grasping and manipulating an object. There is a difference between the physical and perceived forces because the physical characteristics of an object and/or human psychophysical characteristics affect perceived force. Sense of effort plays an important role in deciding the movement made by humans. In this study, we propose a computational method to predict the perceived force by evaluating the muscle activity as a function of effort in the operation of a steering wheel based on a 3D-musculoskeletal model simulation. We found that the perceived-force characteristics depend on the driving posture, though the applied force is the same. We evaluated the results, and showed that the mean of the absolute error is 1.78 N for the experiments conducted on four different vehicles in commercially available.

Psychophysical Evaluation of Change in Friction on an Ultrasonically-Actuated Touchscreen

To render tactile cues on a touchscreen by friction modulation, it is important to understand how humans perceive a change in friction. In this study, we investigate the relations between perceived change in friction on an ultrasonically actuated touchscreen and parameters involved in contact between finger and its surface. We first estimate the perceptual thresholds to detect rising and falling friction while a finger is sliding on the touch surface. Then, we conduct intensity scaling experiments and investigate the effect of finger sliding velocity, normal force, and rise/fall time of vibration amplitude (transition time) on the perceived intensity of change in friction. In order to better understand the role of contact mechanics, we also look into the correlations between the perceived intensities of subjects and several parameters involved in contact. The results of our experiments show that the contrast and rate of change in tangential force were best correlated with the perceived intensity. The subjects perceived rising friction more strongly than falling friction, particularly at higher tangential force contrast. We argue that this is due to hysteresis and viscoelastic behavior of fingertip under tangential loading. The results also showed that transition time and normal force have significant effect on our tactile perception.

Dynamic Information Flow Based on EEG and Diffusion MRI in Stroke: A Proof-of-Principle Study

In hemiparetic stroke, functional recovery of paretic limb may occur with the reorganization of neural networks in the brain. Neuroimaging techniques, such as magnetic resonance imaging (MRI), have a high spatial resolution which can be used to reveal anatomical changes in the brain following a stroke. However, low temporal resolution of MRI provides less insight of dynamic changes of brain activity. In contrast, electro-neurophysiological techniques, such as electroencephalography (EEG), have an excellent temporal resolution to measure such transient events, however are hindered by its low spatial resolution. This proof-of-principle study assessed a novel multimodal brain imaging technique namely Variational Bayesian Multimodal Encephalography (VBMEG), which aims to improve the spatial resolution of EEG for tracking the information flow inside the brain and its changes following a stroke. The limitations of EEG are complemented by constraints derived from anatomical MRI and diffusion weighted imaging (DWI). EEG data were acquired from individuals suffering from a stroke as well as able-bodied participants while electrical stimuli were delivered sequentially at their index finger in the left and right hand, respectively. The locations of active sources related to this stimulus were precisely identified, resulting in high Variance Accounted For (VAF above 80%). An accurate estimation of dynamic information flow between sources was achieved in this study, showing a high VAF (above 90%) in the cross-validation test. The estimated dynamic information flow was compared between chronic hemiparetic stroke and able-bodied individuals. The results demonstrate the feasibility of VBMEG method in revealing the changes of information flow in the brain after stroke. This study verified the VBMEG method as an advanced computational approach to track the dynamic information flow in the brain following a stroke. This may lead to the development of a quantitative tool for monitoring functional changes of the cortical neural networks after a unilateral brain injury and therefore facilitate the research into, and the practice of stroke rehabilitation.

The tactile perception of transient changes in friction

When we touch an object or explore a texture, frictional strains are induced by the tactile interactions with the surface of the object. Little is known about how these interactions are perceived, although it becomes crucial for the nascent industry of interactive displays with haptic feedback (e.g. smartphones and tablets) where tactile feedback based on friction modulation is particularly relevant. To investigate the human perception of frictional strains, we mounted a high-fidelity friction modulating ultrasonic device on a robotic platform performing controlled rubbing of the fingertip and asked participants to detect induced decreases of friction during a forced-choice task. The ability to perceive the changes in friction was found to follow Weber's Law of just noticeable differences, as it consistently depended on the ratio between the reduction in tangential force and the pre-stimulation tangential force. The Weber fraction was 0.11 in all conditions demonstrating a very high sensitivity to transient changes in friction. Humid fingers experienced less friction reduction than drier ones for the same intensity of ultrasonic vibration but the Weber fraction for detecting changes in friction was not influenced by the humidity of the skin.

AR Feels "Softer" than VR: Haptic Perception of Stiffness in Augmented versus Virtual Reality

Does it feel the same when you touch an object in Augmented Reality (AR) or in Virtual Reality (VR)? In this paper we study and compare the haptic perception of stiffness of a virtual object in two situations: (1) a purely virtual environment versus (2) a real and augmented environment. We have designed an experimental setup based on a Microsoft HoloLens and a haptic force-feedback device, enabling to press a virtual piston, and compare its stiffness successively in either Augmented Reality (the virtual piston is surrounded by several real objects all located inside a cardboard box) or in Virtual Reality (the same virtual piston is displayed in a fully virtual scene composed of the same other objects). We have conducted a psychophysical experiment with 12 participants. Our results show a surprising bias in perception between the two conditions. The virtual piston is on average perceived stiffer in the VR condition compared to the AR condition. For instance, when the piston had the same stiffness in AR and VR, participants would select the VR piston as the stiffer one in 60% of cases. This suggests a psychological effect as if objects in AR would feel "softer" than in pure VR. Taken together, our results open new perspectives on perception in AR versus VR, and pave the way to future studies aiming at characterizing potential perceptual biases.

Haptic Feedback in Needle Insertion Modeling and Simulation

Needle insertion is the most basic skill in medical care, and training has to be imparted not only for physicians but also for nurses and paramedics. In most needle insertion procedures, haptic feedback from the needle is the main stimulus in which novices need training. For better patient safety, the classical methods of training the haptic skills have to be replaced with simulators based on new robotic and graphics technologies. This paper reviews the current advances in needle insertion modeling, classified into three sections: needle insertion models, tissue deformation models, and needle-tissue interaction models. Although understated in the literature, the classical and dynamic friction models, which are critical for needle insertion modeling, are also discussed. The experimental setup or the needle simulators that have been developed to validate the models are described. The need of psychophysics for needle simulators and psychophysical parameter analysis of human perception in needle insertion are discussed, which are completely ignored in the literature.

Enhancing Variable Friction Tactile Display Using an Ultrasonic Travelling Wave

In Variable Friction Tactile Displays, an ultrasonic standing wave can be used to reduce the friction coefficient between a user's finger sliding and a vibrating surface. However, by principle, the effect is limited by a saturation due to the contact mechanics, and very low friction levels require very high vibration amplitudes. Besides, to be effective, the user's finger has to move. We present a device which uses a travelling wave rather than a standing wave. We present a control that allows to realize such a travelling wave in a robust way, and thus can be implemented on various plane surfaces. We show experimentally that the force produced by the travelling wave has two superimposed contributions. The first one is equal to the friction reduction produced by a standing of the same vibration amplitude. The second produces a driving force in the opposite direction of the travelling wave. As a result, the modulation range of the tangential force on the finger can be extended to zero and even negative values. Moreover, the effect is dependant on the relative direction of exploration with regards to the travelling wave, which is perceivable and confirmed by a psycho-physical study.

Force Perception Thresholds in Cochlear Implantation Surgery

Tissue trauma is a frequent complication of cochlear implantation (CI) surgery, but the relationship between intracochlear trauma, electrode insertion forces, and surgeons' ability to perceive these forces is poorly understood. In this study, we simulated CI surgery using a benchtop apparatus to repeatedly apply small forces to the subjects' hands while reducing the variability in their hand movements. We used a psychophysical testing procedure to estimate the force perception thresholds of 10 otologic surgeons and found a median threshold of 20.4 mN. The results suggest that surgeons have the capability to sense at least some insertion forces and are likely to perceive severe trauma such as occurs when the electrode crosses from one scala to the other.

Non-Colocated Kinesthetic Display Limits Compliance Discrimination in the Absence of Terminal Force Cues

An important goal of haptic display is to make available the action/reaction relationships that define interactions between the body and the physical world. While in physical world interactions reaction cues invariably impinge on the same part of the body involved in action (reaction and action are colocated), a haptic interface is quite capable of rendering feedback to a separate body part than that used for producing exploratory actions (non-colocated action and reaction). This most commonly occurs with the use of vibrotactile display, in which a cutaneous cue has been substituted for a kinesthetic cue (a kind of sensory substitution). In this paper, we investigate whether non-colocated force and displacement cues degrade the perception of compliance. Using a custom non-colocated kinesthetic display in which one hand controls displacement and the other senses force, we ask participants to discriminate between two virtual springs with matched terminal force and adjustable non-linearity. An additional condition includes one hand controlling displacement while the other senses force encoded in a vibrotactile cue. Results show that when the terminal force cue is unavailable, and even when sensory substitution is not involved, non-colocated kinesthetic displays degrade compliance discrimination relative to colocated kinesthetic displays. Compliance discrimination is also degraded with vibrotactile display of force. These findings suggest that non-colocated kinesthetic displays and, likewise, cutaneous sensory substitution displays should be avoided when discrimination of compliance is necessary for task success.

Pseudo-Haptic Feedback in Teleoperation

In this paper, we develop possible realizations of pseudo-haptic feedback in teleoperation systems based on existing works for pseudo-haptic feedback in virtual reality and the intended applications. We derive four potential factors affecting the performance of haptic feedback (calculation operator, maximum displacement, offset force, and scaling factor), which are analyzed in three compliance identification experiments. First, we analyze the principle usability of pseudo-haptic feedback by comparing information transfer measures for teleoperation and direct interaction. Pseudo-haptic interaction yields well above-chance performance, while direct interaction performs almost perfectly. In order to optimize pseudo-haptic feedback, in the second study we perform a full-factorial experimental design with 36 subjects performing 6,480 trials with 36 different treatments. Information transfer ranges from 0.68 bit to 1.72 bit in a task with a theoretical maximum of 2.6 bit, with a predominant effect of the calculation operator and a minor effect of the maximum displacement. In a third study, short- and long-term learning effects are analyzed. Learning effects regarding the performance of pseudo-haptic feedback cannot be observed for single-day experiments. Tests over 10 days show a maximum increase in information transfer of 0.8 bit. The results show the feasibility of pseudo-haptic feedback for teleoperation and can be used as design basis for task-specific systems.

Haptic perception of force magnitude and its relation to postural arm dynamics in 3D

In a previous study, we found the perception of force magnitude to be anisotropic in the horizontal plane. In the current study, we investigated this anisotropy in three dimensional space. In addition, we tested our previous hypothesis that the perceptual anisotropy was directly related to anisotropies in arm dynamics. In experiment 1, static force magnitude perception was studied using a free magnitude estimation paradigm. This experiment revealed a significant and consistent anisotropy in force magnitude perception, with forces exerted perpendicular to the line between hand and shoulder being perceived as 50% larger than forces exerted along this line. In experiment 2, postural arm dynamics were measured using stochastic position perturbations exerted by a haptic device and quantified through system identification. By fitting a mass-damper-spring model to the data, the stiffness, damping and inertia parameters could be characterized in all the directions in which perception was also measured. These results show that none of the arm dynamics parameters were oriented either exactly perpendicular or parallel to the perceptual anisotropy. This means that endpoint stiffness, damping or inertia alone cannot explain the consistent anisotropy in force magnitude perception.

Pseudo-haptic feedback in medical teleoperation

Pseudo-haptic feedback is a haptic illusion based on a mismatch of haptic and visual perception. It is well known from applications in virtual environments. In this work, we discuss the usabiliy of the principle of pseudo-haptic feedback for teleoperation. Using pseudo-haptic feedback can ease the design of haptic medical tele-operation systems.

Thereby a user’s grasping force at an isometric user interface is used to control the closing angle of an end effector of a surgical robot. To provide a realistic haptic feedback, the coupling characteristic of grasping force and end effector closing angle is changed depending on acting end effector interaction forces.

With an experiment, we show the usability of pseudo-haptic feedback for discriminating compliances, comparable to the mechanical characteristic of muscles relaxed and contracted. The provided results base upon the data of 10 subjects, and 300 trails.

A pilot study of a plantar sensory evaluation system for early screening of diabetic neuropathy in a weight-bearing position

The purpose of this study is to develop smart equipment to quantify plantar tactile sensibility for the early diagnosis and tracking of peripheral neuropathy caused by diabetes mellitus. In this paper, we offer a new testing system that is composed of a plantar tactile stimulation platform with a small moving contactor to stretch the skin tangentially, a response switch for each tactile stimulus, a motor control box, and a personal computer (PC) for psychophysical data processing. This quantitative sensory testing system has detailed measurements available and is easy to use compared with the conventional testing devices, such as von Frey monofilaments, pin-prick testing devices, and current perception threshold testers. When using our testing system in a weight-bearing position, we observed that the plantar tactile thresholds for the tangential stretching stimulus on the plantar surface of the foot ranged from approximately 10 um to 30 um for healthy subjects. However, the threshold for a subject with diabetes was nearly three times higher than that for healthy subjects. The significant difference between these values suggests that the plantar sensory evaluation system using the lateral skin stretch stimulation can be used for early diagnosis, for the accurate staging of diabetic neuropathy, and for evaluating its progression noninvasively in a clinic and at home.