Psychophysical Evaluation of Change in Friction on an Ultrasonically-Actuated Touchscreen

Rapid change of friction causes the illusion of touching a receding surface

Shortly after touching an object, humans can tactually gauge the frictional resistance of a surface. The knowledge of surface friction is paramount to tactile perception and the motor control of grasp. While potent correlations between friction and participants' perceptual response have been found, the causal link between the friction of the surface, its evolution and its perceptual experience has yet to be demonstrated. Here, we leverage new experimental apparatus able to modify friction in real time, to show that participants can perceive sudden changes in friction when they are pressing on a surface. Surprisingly, only a reduction of the friction coefficient leads to a robust perception. High-speed imaging data indicate that the sensation is caused by a release of a latent elastic strain over a 20 ms timeframe after the activation of the friction-reduction device. This rapid change of frictional properties during initial contact is interpreted as a normal displacement of the surface, which paves the way for haptic surfaces that can produce illusions of interacting with mechanical buttons.

PCA Model of Fundamental Acoustic Finger Force for Out-of-Plane Ultrasonic Vibration and its Correlation with Friction Reduction

When a finger touches an ultrasonic vibrating plate, a non-sinusoidal contact force is produced. This force is called acoustic finger force. In a setup where closed-loop control is performed on the vibration amplitude, a component of the acoustic finger force can be measured at the fundamental vibration frequency of the plate. This calculation is obtained from the measurement of the variation of the controller voltage between the no-load case and when a finger is present. This calculation is made for a group of twelve participants. From these results a PCA (Principal Component Analysis) model is created. This model permits estimation of the acoustic finger force response of a participant at any vibration amplitude, based on a one or two point measurement. Finally, a linear relation between the PCA coefficients and the friction reduction is proposed. The objective of this relation would be to ultimately provide the means to create an amplitude reference calibration based on the desired friction reduction level, and thus be able to produce a standardized tactile feedback for each user, despite the biomechanical differences in finger pad properties between subjects.

Estimating Friction Modulation From the Ultrasonic Mechanical Impedance

Ultrasonic surface-haptic touchscreens produce compelling tactile sensations directly on the users' fingertips. The tactile sensations stem from the modulation of friction produced by acoustic radiation pressure, which reduces the contact between the skin and the glass plate. During this process, some of the vibrations are partly absorbed by the tissues, resulting in a conspicuous change in the vibration amplitude of the plate upon contact with the finger, which manifests as a net change in the system mechanical impedance. In this article, we leverage the observable change of impedance to estimate the acoustic levitation and the frictional force. The self-sensing method utilizes a model of the first principles governing the physical interaction between the plate and the skin, which relies on multi-scale contact theory. The model accurately describes the experimental influence of the amplitude on the observed impedance (i.e., the amount of energy absorbed and reflected) and can be used to estimate the friction coefficient ($R^2=0.93$). These results provide additional evidence of the partial levitation mechanism at play in ultrasonic friction-modulation. This finding can be useful for designing energy-efficient devices and provide design suggestions for using ultrasonic impedance for self-sensing friction forces.

Tri-Modal Tactile Display and Its Application Into Tactile Perception of Visualized Surfaces

Tactile representation on touchscreens plays an important role in improving realism and richness of users' interaction experience. The dynamic lateral force range and the efficient feedback dimensions are very critical in determining the fidelity of tactile displays. This article develops a tri-modal Electrovibration, Ultrasonic Vibration, and Mechanical Vibration (EUMV) tactile display integrating three types of representative principles, which enhances the dynamic lateral force range by leveraging electrostatic and ultrasonic vibrations stimuli, and induces the normal feedback dimension by utilizing mechanical vibration stimulus. Then, a tactile perception scheme with the EUMV display is proposed for simultaneously rendering contour and texture roughness features of visualized surfaces, in which the contour gradient-lateral force model and the texture gradient-perceived roughness model are determined respectively. Objective and subjective evaluations with 20 participants show that the novel scheme establishes significant improvements in both correct recognition ratios of geometric shapes and tactile perception realism of visualized images than the previous studies.

A Review of Surface Haptics: Enabling Tactile Effects on Touch Surfaces

In this article, we review the current technology underlying surface haptics that converts passive touch surfaces to active ones (machine haptics), our perception of tactile stimuli displayed through active touch surfaces (human haptics), their potential applications (human-machine interaction), and finally, the challenges ahead of us in making them available through commercial systems. This article primarily covers the tactile interactions of human fingers or hands with surface-haptics displays by focusing on the three most popular actuation methods: vibrotactile, electrostatic, and ultrasonic.

Detection of Friction-Modulated Textures is Limited by Vibrotactile Sensitivity

Modulation of the frictional force of a fingertip sliding over a surface-haptic device can produce compelling sensations of texture and relief. The virtual sensation is particularly apparent and feel as fixed in space if the stimulus is rigorously correlated with the displacement of the finger. While frictional textures tactually resemble their real counterparts, some exploratory conditions under which the sharpness of the texture declines exist. We postulate that this decline in sharpness is caused by the perceptual limitation of the attempt to interpret the variation in friction as an out-of-plane sinusoidal topography. To investigate these questions, we measured the detection thresholds of sinusoidal friction-modulated gratings for a wide range of spatial periods explored at two different speeds. We compared the results with the detection thresholds, reported in the literature, of real gratings and vibrotactile stimuli. We found that the detection of spatial friction-modulated textures does not follow the same trend as that of real textures but is more similar to the vibrotactile rendering, which is strongly influenced by the exploratory speed. This article provides a better understanding of the perception of friction-modulated textures and provides insight into how to design impactful stimuli on surface-haptic devices.

Tactile Roughness Perception of Virtual Gratings by Electrovibration

Realistic display of tactile textures on touch screens is a big step forward for haptic technology to reach a wide range of consumers utilizing electronic devices on a daily basis. Since the texture topography cannot be rendered explicitly by electrovibration on touch screens, it is important to understand how we perceive the virtual textures displayed by friction modulation via electrovibration. We investigated the roughness perception of real gratings made of plexiglass and virtual gratings displayed by electrovibration through a touch screen for comparison. In particular, we conducted two psychophysical experiments with ten participants to investigate the effect of spatial period and the normal force applied by finger on roughness perception of real and virtual gratings in macro size. We also recorded the contact forces acting on the participants' finger during the experiments. The results showed that the roughness perception of real and virtual gratings are different. We argue that this difference can be explained by the amount of fingerpad penetration into the gratings. For real gratings, penetration increased tangential forces acting on the finger, whereas for virtual ones where skin penetration is absent, tangential forces decreased with spatial period. Supporting our claim, we also found that increasing normal force increases the perceived roughness of real gratings while it causes an opposite effect for the virtual gratings. These results are consistent with the tangential force profiles recorded for both real and virtual gratings. In particular, the rate of change in tangential force ( dF_t/dt) as a function of spatial period and normal force followed trends similar to those obtained for the roughness estimates of real and virtual gratings, suggesting that it is a better indicator of the perceived roughness than the tangential force magnitude.

Tactile Perception of Virtual Edges and Gratings Displayed by Friction Modulation via Ultrasonic Actuation

Tactile discrimination and roughness perception of real textures are extensively studied and underlying perceptual mechanisms are relatively well-established. However, tactile perception of virtual textures rendered by friction modulation techniques on touch surfaces has not been investigated in detail yet. In this article, we investigated our ability to discriminate two consecutive step changes in friction (called edges), followed by discrimination and roughness perception of multiple edges (called periodic gratings). The results showed that discrimination of two consecutive edges was significantly influenced by edge sequence: a step fall in friction ( FF) followed by a step rise in friction ( RF) was discriminated more easily than the reverse order. On the other hand, periodic gratings displayed by consecutive sequences of FF followed by RF were perceived with the same acuity as compared to vice versa. Independent of the edge sequence, we found that a relative difference of 14% in spatial period was required to discriminate two periodic gratings. Moreover, the roughness perception of periodic gratings decreased with increasing spatial period for the range that we have investigated (spatial period 2 mm), despite the lack of spatial cues on grating height. We also observed that rate of change in friction coefficient was better correlated with the roughness perception than the friction coefficient itself. These results will further help to understand and design virtual textures for touch surfaces.

Step-Change in Friction Under Electrovibration

Rendering tactile effects on a touch screen via electrovibration has many potential applications. However, our knowledge on tactile perception of change in friction and the underlying contact mechanics are both very limited. In this article, we investigate the tactile perception and the contact mechanics for a step change in friction under electrovibration during a relative sliding between a finger and the surface of a capacitive touch screen. First, we conduct magnitude estimation experiments to investigate the role of normal force and sliding velocity on the perceived tactile intensity for a step increase and decrease in friction, called rising friction (RF) and falling friction (FF). To investigate the contact mechanics involved in RF and FF, we then measure the frictional force, the apparent contact area, and the strains acting on the fingerpad during sliding at a constant velocity under three different normal loads using a custom-made experimental set-up. The results show that the participants perceived RF stronger than FF, and both the normal force and sliding velocity significantly influenced their perception. These results are supported by our mechanical measurements; the relative change in friction, the apparent contact area, and the strain in the sliding direction were all higher for RF than those for FF, especially for low normal forces. Taken together, our results suggest that different contact mechanics take place during RF and FF due to the viscoelastic behavior of fingerpad skin, and those differences influence our tactile perception of a step change in friction.

Fingerpad contact evolution under electrovibration

Displaying tactile feedback through a touchscreen via electrovibration has many potential applications in mobile devices, consumer electronics, home appliances and automotive industry though our knowledge and understanding of the underlying contact mechanics are very limited. An experimental study was conducted to investigate the contact evolution between the human finger and a touch screen under electrovibration using a robotic set-up and an imaging system. The results show that the effect of electrovibration is only present during full slip but not before slip. Hence, the coefficient of friction increases under electrovibration as expected during full slip, but the apparent contact area is significantly smaller during full slip when compared to that of no electrovibration condition. It is suggested that the main cause of the increase in friction during full slip is due to an increase in the real contact area and the reduction in apparent area is due to stiffening of the finger skin in the tangential direction.

The Perception of Ultrasonic Square Reductions of Friction With Variable Sharpness and Duration

The human perception of square ultrasonic modulation of the finger-surface friction was investigated during active tactile exploration by using short frictional cues of varying duration and sharpness. In a first experiment, we asked participants to discriminate the transition time and duration of short square ultrasonic reductions of friction. They proved very sensitive to discriminate millisecond differences in these two parameters with the average psychophysical thresholds being 2.3-2.4 ms for both parameters. A second experiment focused on the perception of square friction reductions with variable transition times and durations. We found that for durations of the stimulation larger than 90 ms, participants often perceived three or four edges when only two stimulations were presented while they consistently felt two edges for signals shorter than 50 ms. A subsequent analysis of the contact forces induced by these ultrasonic stimulations during slow and fast active exploration showed that two identical consecutive ultrasonic pulses can induce significantly different frictional dynamics especially during fast motion of the finger. These results confirm the human sensitivity to transient frictional cues and suggest that the human perception of square reductions of friction can depend on their sharpness and duration as well as on the speed of exploration.

Electroadhesion with application to touchscreens

There is growing interest in touchscreens displaying tactile feedback due to their tremendous potential in consumer electronics. In these systems, the friction between the user's fingerpad and the surface of the touchscreen is modulated to display tactile effects. One of the promising techniques used in this regard is electrostatic actuation. If, for example, an alternating voltage is applied to the conductive layer of a surface capacitive touchscreen, an attractive electrostatic force is generated between the finger and the surface, which results in an increase in frictional forces acting on the finger moving on the surface. By altering the amplitude, frequency, and waveform of this signal, a rich set of tactile effects can be generated on the touchscreen. Despite the ease of implementation and its powerful effect on our tactile sensation, the contact mechanics leading to an increase in friction due to electroadhesion has not been fully understood yet. In this paper, we present experimental results for how the friction between a finger and a touchscreen depends on the electrostatic attraction and the applied normal pressure. The dependency of the finger-touchscreen interaction on the applied voltage and on several other parameters is also investigated using a mean field theory based on multiscale contact mechanics. We present detailed theoretical analysis of how the area of real contact and the friction force depend on contact parameters, and show that it is possible to further augment the friction force, and hence the tactile feedback displayed to the user by carefully choosing those parameters.