The Application of Tactile, Audible, and Ultrasonic Forces to Human Fingertips Using Broadband Electroadhesion

High-performance electroadhesive clutches with multilayered architecture

Electroadhesive (EA) clutches are promising for advanced motion and force control in robotics, haptics, and rehabilitation, owing to their compactness and light weight. However, their practical use is limited by the inability to deliver high forces at low voltages, primarily due to a lack of understanding of their mechanics. We introduce a novel deformable body fracture mechanics approach and high-resolution strain field imaging to reveal that nonuniform stress distributions cause EA clutches to fail through delamination and crack propagation. Using this insight, we developed EA clutches sustaining 22 newtons over 1 square centimeter at 100 volts, achieving the highest stress per voltage among similar clutches. This was achieved by incorporating a soft interlayer and peeling stopper for uniform stress distribution and mitigating the failure modes. These EA clutches were integrated into a lightweight ring-based wearable system for finger rehabilitation and haptics. Our findings lay the groundwork for designing low-voltage, high-performance EA clutches for next-generation motion and force control applications.

Local Area Tactile Stimulation Using Interference of Multi-Frequency Airborne Ultrasound

In spatiotemporal modulation (STM) and lateral modulation (LM) used in conventional mid-air ultrasound tactile stimulation, single or multiple focuses are moved by switching the ultrasound transducer phases. A problem with the phase switching method is the limitation of the focus motion speed due to rapid phase switching that causes sound pressure fluctuations. This paper proposes an LM method using multiple-frequency ultrasound to shift the ultrasound focal point without switching the phase. This method can demonstrate a continuous and stable moving stimulus with high-frequency components, without producing unnecessary audible noise. Using the proposed broad-band LM covering up to 400 Hz, we found that a high-frequency 400 Hz LM applied at a finger pad can display a stimulation area with the diameters comparable to or less than the half wavelength of 40 kHz ultrasound, where the perceptual size was evaluated as 4.2 mm for the long axis diameter and 3.4 mm for the short axis diameter.

Effect of Finger Moisture on Tactile Perception of Electroadhesion

We investigate the effect of finger moisture on the tactile perception of electroadhesion with 10 participants. Participants with moist fingers exhibited markedly higher threshold levels. Our electrical impedance measurements show a substantial reduction in impedance magnitude when sweat is present at the finger-touchscreen interface, indicating increased conductivity. Supporting this, our mechanical friction measurements show that the relative increase in electrostatic force due to electroadhesion is lower for a moist finger.

Closed-Loop Control of Electroadhesion Using Current Regulation

Electroadhesion displays provide controllable friction between the fingertip and screen. However, the change of contact condition causes variability in the produced friction. In this paper, we demonstrate a novel method for closed-loop control using current regulation to improve the precision of the electroadhesion force regardless of contact conditions. The current sensor obtains static current (when the finger is stationary) and dynamic current (when the finger is sliding). The static current is used to estimate the apparent contact area. The estimated contact area modulates the driving voltage along with the dynamic current. To verify the proposed method, we measured electroadhesion forces under open-loop control and closed-loop control. The benefit of using this closed-loop control is shown by comparing the relative static error of open-loop control and closed-loop control. The relative error reductions achieved over 34 % (max 112 %) for four changing contact conditions.

Data-Driven Playback of Natural Tactile Texture Via Broadband Friction Modulation

We used broadband electroadhesion to reproduce the friction force profile measured as a finger slid across a textured surface. In doing so, we were also able to reproduce with high fidelity the skin vibrations characteristic of that texture; however, we found that this did not reproduce the original perception. To begin, the reproduction felt weak. In order to maximize perceptual similarity between a real texture and its friction force playback, the vibratory magnitude of the latter must be scaled up on average ≈ 3X for fine texture and ≈ 5X for coarse texture samples. This additional gain appears to correlate with perceived texture roughness. Additionally, even with optimal scaling and high fidelity playback, subjects could identify which of two reproductions corresponds to a real texture with only 71 % accuracy, as compared to 95 % accuracy when using real texture alternatives. We conclude that while tribometry and vibrometry data can be useful for texture classification, they appear to contribute only partially to texture perception. We propose that spatially distributed excitation of skin within the fingerpad may play an additional key role, and may thus be able to contribute to high fidelity texture reproduction.

Surface haptic rendering of virtual shapes through change in surface temperature

Compared to relatively mature audio and video human-machine interfaces, providing accurate and immersive touch sensation remains a challenge owing to the substantial mechanical and neurophysical complexity of touch. Touch sensations during relative lateral motion between a skin-screen interface are largely dictated by interfacial friction, so controlling interfacial friction has the potential for realistic mimicry of surface texture, shape, and material composition. In this work, we show a large modulation of finger friction by locally changing surface temperature. Experiments showed that finger friction can be increased by ~50% with a surface temperature increase from 23° to 42°C, which was attributed to the temperature dependence of the viscoelasticity and the moisture level of human skin. Rendering virtual features, including zoning and bump(s), without thermal perception was further demonstrated with surface temperature modulation. This method of modulating finger friction has potential applications in gaming, virtual and augmented reality, and touchscreen human-machine interaction.

Building a Navigable Fine Texture Design Space

Friction modulation technology enables the creation of textural effects on flat haptic displays. However, an intuitive and manageably small design space for construction of such haptic textures remains an unfulfilled goal for user interface designers. In this paper, we explore perceptually relevant features of fine texture for use in texture construction and modification. Beginning with simple sinusoidal patterns of friction force that vary in frequency and amplitude, we define irregularity, essentially a variable amount of introduced noise, as a third building block of a texture pattern. We demonstrate using multidimensional scaling that all three parameters are scalable features perceptually distinct from each other. Additionally, participants' verbal descriptions of this 3-dimensional design space provide insight into their intuitive interpretation of the physical parameter changes.

Comparison of Wide-Band Vibrotactile and Friction Modulation Surface Gratings

This article seeks to understand conditions under which virtual gratings produced via vibrotaction and friction modulation are perceived as similar and to find physical origins in the results. To accomplish this, we developed two single-axis devices, one based on electroadhesion and one based on out-of-plane vibration. The two devices had identical touch surfaces, and the vibrotactile device used a novel closed-loop controller to achieve precise control of out-of-plane plate displacement under varying load conditions across a wide ranget of frequencies. A first study measured the perceptual intensity equivalence curve of gratings generated under electroadhesion and vibrotaction across the 20-400 Hz frequency range. A second study assessed the perceptual similarity between two forms of skin excitation given the same driving frequency and same perceived intensity. Our results indicate that it is largely the out-of-plane velocity that predicts vibrotactile intensity relative to shear forces generated by friction modulation. A high degree of perceptual similarity between gratings generated through friction modulation and through vibrotaction is apparent and tends to scale with actuation frequency suggesting perceptual indifference to the manner of fingerpad actuation in the upper frequency range.

Finger motion and contact by a second finger influence the tactile perception of electrovibration

Electrovibration holds great potential for creating vivid and realistic haptic sensations on touchscreens. Ideally, a designer should be able to control what users feel independent of the number of fingers they use, the movements they make, and how hard they press. We sought to understand the perception and physics of such interactions by determining the smallest 125 Hz electrovibration voltage that 15 participants could reliably feel when performing four different touch interactions at two normal forces. The results proved for the first time that both finger motion and contact by a second finger significantly affect what the user feels. At a given voltage, a single moving finger experiences much larger fluctuating electrovibration forces than a single stationary finger, making electrovibration much easier to feel during interactions involving finger movement. Indeed, only about 30% of participants could detect the stimulus without motion. Part of this difference comes from the fact that relative motion greatly increases the electrical impedance between a finger and the screen, as shown via detailed measurements from one individual. By contrast, threshold-level electrovibration did not significantly affect the coefficient of kinetic friction in any conditions. These findings help lay the groundwork for delivering consistent haptic feedback via electrovibration.

Tri-Modal Tactile Display and Its Application Into Tactile Perception of Visualized Surfaces

Tactile representation on touchscreens plays an important role in improving realism and richness of users' interaction experience. The dynamic lateral force range and the efficient feedback dimensions are very critical in determining the fidelity of tactile displays. This article develops a tri-modal Electrovibration, Ultrasonic Vibration, and Mechanical Vibration (EUMV) tactile display integrating three types of representative principles, which enhances the dynamic lateral force range by leveraging electrostatic and ultrasonic vibrations stimuli, and induces the normal feedback dimension by utilizing mechanical vibration stimulus. Then, a tactile perception scheme with the EUMV display is proposed for simultaneously rendering contour and texture roughness features of visualized surfaces, in which the contour gradient-lateral force model and the texture gradient-perceived roughness model are determined respectively. Objective and subjective evaluations with 20 participants show that the novel scheme establishes significant improvements in both correct recognition ratios of geometric shapes and tactile perception realism of visualized images than the previous studies.

Organic Haptics: Intersection of Materials Chemistry and Tactile Perception

The goal of the field of haptics is to create technologies that manipulate the sense of touch. In virtual and augmented reality, haptic devices are for touch what loudspeakers and RGB displays are for hearing and vision. Haptic systems that utilize micromotors or other miniaturized mechanical devices (e.g., for vibration and pneumatic actuation) produce interesting effects, but are quite far from reproducing the feeling of real materials. They are especially deficient in recapitulating surface properties: fine texture, friction, viscoelasticity, tack, and softness. The central argument of this Progress Report is that to reproduce the feel of everyday objects requires chemistry: molecular control over the properties of materials and ultimately design of materials which can change these properties in real time. Stimuli-responsive organic materials, such as polymers and composites, are a class of materials which can change their oxidation state, conductivity, shape, and rheological properties, and thus might be useful in future haptic technologies. Moreover, the use of such materials in research on tactile perception could help elucidate the limits of human tactile sensitivity. The work described represents the beginnings of this new area of inquiry, in which the defining approach is the marriage of materials science and psychology.

Electrowetting: A Consideration in Electroadhesion

With the commercialization of haptic devices, understanding behavior under various environmental conditions is crucial for product optimization and cost reduction. Specifically, for surface haptic devices, the dependence of the friction force and the electroadhesion effect on the environmental relative humidity and the finger hydration level can directly impact their design and performance. This article presents the influence of relative humidity on the finger-surface friction force and the electroadhesion performance. Mechanisms including changes to Young's modulus of skin, contact angle change and capillary force were analyzed separately with experimental and numerical methods. Through comparison of the calculated capillary force in this paper and the electroadhesion force calculated in published papers, it was found that electrowetting at high voltage could contribute up to 60% of the total friction force increase in electroadhesion. Therefore, in future design of surface haptic devices, the effect of electrowetting should be considered carefully.

Effect of Temperature on the Absolute and Discrimination Thresholds of Voltage on Electrovibration Tactile Display

Multi-dimensions tactile displays, such as thermal and texture display, are desirable for enhancing perception while users experience virtual shopping such as touching a garment in virtual reality. Understanding the effect of one dimension on the other is fundamental for design of multi-dimensions tactile display. In this article, we report the effect of temperature on thresholds of voltage applied on an electrovibration tactile display. Three temperatures of the electrovibration tactile display at 18°C (cold), 30°C (neutral) and 38°C (warm) were considered in two experiments. In Experiment I, we measured the absolute thresholds of square wave voltage with 25 Hz, 140 Hz and 485 Hz. In Experiment II, we measured the amplitude discrimination thresholds of same voltage signals as in Experiment I. The results show that the absolute thresholds differed significantly between 18°C and 38°C for all the three frequencies. No significant difference in the absolute threshold was found between 18°C and 30°C, except for the 485 Hz voltage. The amplitude discrimination thresholds were essentially constant except for that of the 485 Hz voltage at 18°C, which were 17.11 Vpp and 16.86 Vpp larger than those at 30°C and 38°C, respectively.

Localizable Button Click Rendering via Active Lateral Force Feedback

In this article, we have developed a novel button click rendering mechanism based on active lateral force feedback. The effect can be localized because electroadhesion between a finger and a surface can be localized. Psychophysical experiments were conducted to evaluate the quality of a rendered button click, which subjects judged to be acceptable. Both the experiment results and the subjects' comments confirm that this button click rendering mechanism has the ability to generate a range of realistic button click sensations that could match subjects' different preferences. We can, thus, generate a button click on a flat surface without macroscopic motion of the surface in the lateral or normal direction, and can localize this haptic effect to an individual finger.

Detection of Friction-Modulated Textures is Limited by Vibrotactile Sensitivity

Modulation of the frictional force of a fingertip sliding over a surface-haptic device can produce compelling sensations of texture and relief. The virtual sensation is particularly apparent and feel as fixed in space if the stimulus is rigorously correlated with the displacement of the finger. While frictional textures tactually resemble their real counterparts, some exploratory conditions under which the sharpness of the texture declines exist. We postulate that this decline in sharpness is caused by the perceptual limitation of the attempt to interpret the variation in friction as an out-of-plane sinusoidal topography. To investigate these questions, we measured the detection thresholds of sinusoidal friction-modulated gratings for a wide range of spatial periods explored at two different speeds. We compared the results with the detection thresholds, reported in the literature, of real gratings and vibrotactile stimuli. We found that the detection of spatial friction-modulated textures does not follow the same trend as that of real textures but is more similar to the vibrotactile rendering, which is strongly influenced by the exploratory speed. This article provides a better understanding of the perception of friction-modulated textures and provides insight into how to design impactful stimuli on surface-haptic devices.

Tactile Roughness Perception of Virtual Gratings by Electrovibration

Realistic display of tactile textures on touch screens is a big step forward for haptic technology to reach a wide range of consumers utilizing electronic devices on a daily basis. Since the texture topography cannot be rendered explicitly by electrovibration on touch screens, it is important to understand how we perceive the virtual textures displayed by friction modulation via electrovibration. We investigated the roughness perception of real gratings made of plexiglass and virtual gratings displayed by electrovibration through a touch screen for comparison. In particular, we conducted two psychophysical experiments with ten participants to investigate the effect of spatial period and the normal force applied by finger on roughness perception of real and virtual gratings in macro size. We also recorded the contact forces acting on the participants' finger during the experiments. The results showed that the roughness perception of real and virtual gratings are different. We argue that this difference can be explained by the amount of fingerpad penetration into the gratings. For real gratings, penetration increased tangential forces acting on the finger, whereas for virtual ones where skin penetration is absent, tangential forces decreased with spatial period. Supporting our claim, we also found that increasing normal force increases the perceived roughness of real gratings while it causes an opposite effect for the virtual gratings. These results are consistent with the tangential force profiles recorded for both real and virtual gratings. In particular, the rate of change in tangential force ( dF_t/dt) as a function of spatial period and normal force followed trends similar to those obtained for the roughness estimates of real and virtual gratings, suggesting that it is a better indicator of the perceived roughness than the tangential force magnitude.

Effect of scanning speed on texture-elicited vibrations

To sense the texture of a surface, we run our fingers across it, which leads to the elicitation of skin vibrations that depend both on the surface and on exploratory parameters, particularly scanning speed. The transduction and processing of these vibrations mediate the ability to discern fine surface features. The objective of the present study is to characterize the effect of changes in scanning speed on texture-elicited vibrations to better understand how the exploratory movements shape the neuronal representation of texture. To this end, we scanned a variety of textures across the fingertip of human participants at a variety of speeds (10-160 mm s^-1) while measuring the resulting vibrations using a laser Doppler vibrometer. First, we found that the intensity of the vibrations-as indexed by root-mean-square velocity-increases with speed but that the skin displacement remains constant. Second, we found that the frequency composition of the vibrations shifts systematically to higher frequencies with increases in scanning speed. Finally, we show that the speed-dependent shift in frequency composition accounts for the speed-dependent change in intensity.

Virtual Texture Generated using Elastomeric Conductive Block Copolymer in Wireless Multimodal Haptic Glove

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. This paper describes a haptic glove capable of producing sensations reminiscent of three types of near-surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators were combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in our laboratory. This polymer consisted of a stretchable polyanion which served as a scaffold for the polymerization of poly(3,4-ethylenedioxythiophene) (PEDOT). The scaffold was synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity (0.1 S cm-1), and low impedance relative to metals. The glove was equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants were able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature had an overall accuracy of 98%, while untrained participants had an accuracy of 85%. Sensations could similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.

Effect of Auditory Feedback on Tactile Intensity Perception in a Touchscreen Application

This article presents the effect of auditory feedback on tactile intensity perception, which may be of interest to haptic or audiotactile interaction engineers. An experimental setup consisted of a touchscreen, an electrodynamic shaker, and a closed-back headphone for a subject to interact with the touchscreen and to feel audiotactile feedback. In the experiment, participants were asked to judge perceived tactile intensity, using the magnitude estimation method, in the absence and presence of simultaneous auditory feedback. All data collected from the subjects were analyzed statistically, and then the effect of auditory feedback was investigated focusing on the following aspects: whether the presence of auditory feedback changes perceived tactile intensity, whether the frequency component of auditory feedback affects tactile intensity perception, and whether the coincidence of tactile and auditory frequencies influences on tactile intensity perception. Besides, changes in Stevens's exponent were analyzed to discuss how tactile intensity perception varies due to the auditory feedback. Finally, an equal intensity contour, in the domain of sensation level and frequency of tactile stimulation, was drawn. It can be applied to adjust the level of tactile stimuli for haptic feedback designers to provide a constant perceived tactile intensity considering the presence of auditory feedback.

Data-Driven Texture Modeling and Rendering on Electrovibration Display

With the introduction of variable friction displays, either based on ultrasonic or electrovibration technology, new possibilities have emerged in haptic texture rendering on flat surfaces. In this work, we propose a data-driven method for realistic texture rendering on an electrovibration display. We first describe a motorized linear tribometer designed to collect lateral frictional forces from textured surfaces under various scanning velocities and normal forces. We then propose an inverse dynamics model of the display to describe its output-input relationship using nonlinear autoregressive neural networks with external input. Forces resulting from applying a pseudo-random binary signal to the display are used to train each network under the given experimental condition. In addition, we propose a two-step interpolation scheme to estimate actuation signals for arbitrary conditions under which no prior data have been collected. A comparison between real and virtual forces in the frequency domain shows promising results for recreating virtual textures similar to the real ones, also revealing the capabilities and limitations of the proposed method. We also conducted a human user study to compare the performance of our neural-network-based method with that of a record-and-playback method. The results showed that the similarity between the real and virtual textures generated by our approach was significantly higher.

Closed Loop Application of Electroadhesion for Increased Precision in Texture Rendering

Tactile displays based on friction modulation offer wide-bandwidth forces rendered directly on the fingertip. However, due to a number of touch conditions (e.g., normal force, skin hydration) that result in variations in the friction force and the strength of modulation effect, the precision of the force rendering remains limited. In this paper we demonstrate a closed-loop electroadhesion method for precise playback of friction force profiles on a human finger and we apply this method to the tactile rendering of several textiles encountered in everyday life.

The Effect of Applied Normal Force on the Electrovibration

Electrovibration has become one of the promising approaches for adding tactile feedback on touchscreen. Previous studies revealed that the normal force applied on the touchscreen by the finger affects significantly the electrostatic force. It is obvious that the normal force affects the electrostatic force if it changes the contact area between the finger and the touchscreen. However, it is unclear whether the normal force affects the electrostatic force when the apparent contact area is constant. In this paper, we estimated the electrostatic force via measuring the tangential force of the finger sliding on a 3M touchscreen at different normal forces under the constant apparent contact area. We found that the electrostatic force increases significantly as the normal force increases from 0.5 to 4.5N. We explained the experimental results using the most recently proposed electrostatic force model, which considers the effect of air gap. We estimated the averaged air gap thickness using the electrostatic force model. The results showed that the relationship between the air gap thickness and the normal force follows a power function. Our experiment suggests that the normal force has a significant effect on the air gap thickness, thus require consideration in the design of tactile feedback.

Detection and Identification of Pattern Information on an Electrostatic Friction Display

An electrostatic friction modulation device based on a tablet computer was used to present pattern stimuli to the fingertip for two tasks: detecting patches of friction and matching a frictional pattern to the visual image that produced it. In the detection task, friction patterns were displayed on zero, one two or three cells in a matrix. Errors, whether misses or false alarms, were few. Duration of target-present trials was a linear function of the number of patterns in the display. The intercept indicated an average of under 1 sec to test a location for the presence of a friction patch. The slope was 1.0 sec per item, representing the time to confirm friction change, verify the location, and report. In contrast to fast and accurate detection of friction modulation, identification of patterns by matching to a visual display was at chance, although the patterns were differentiated by form and scale. Given that the patterns fall within the normal acuity of the fingertip, along with previous evidence that fingertip motion per se does not preclude pattern recognition, it appears that the failure to match tactual patterns to visual images resides in processes inherent in information pickup from friction-modulation displays.

UltraShiver: Lateral Force Feedback on a Bare Fingertip via Ultrasonic Oscillation and Electroadhesion

We propose a new lateral force feedback device, the UltraShiver, which employs a combination of in-plane ultrasonic oscillation (around 30 kHz) and out-of-plane electroadhesion. It can achieve a strong active lateral force (400 mN) on the bare fingertip while operating silently. The lateral force is a function of pressing force, lateral vibration velocity, and electroadhesive voltage, as well as the relative phase between the velocity and voltage. In this paper, we perform experiments to investigate characteristics of the UltraShiver and their influence on lateral force. A lumped-parameter model is developed to understand the physical underpinnings of these influences. The model with frequency-weighted electroadhesion forces shows good agreement with experimental results. In addition, a Gaussian-like potential well is rendered as an application of the UltraShiver.

Electroadhesion with application to touchscreens

There is growing interest in touchscreens displaying tactile feedback due to their tremendous potential in consumer electronics. In these systems, the friction between the user's fingerpad and the surface of the touchscreen is modulated to display tactile effects. One of the promising techniques used in this regard is electrostatic actuation. If, for example, an alternating voltage is applied to the conductive layer of a surface capacitive touchscreen, an attractive electrostatic force is generated between the finger and the surface, which results in an increase in frictional forces acting on the finger moving on the surface. By altering the amplitude, frequency, and waveform of this signal, a rich set of tactile effects can be generated on the touchscreen. Despite the ease of implementation and its powerful effect on our tactile sensation, the contact mechanics leading to an increase in friction due to electroadhesion has not been fully understood yet. In this paper, we present experimental results for how the friction between a finger and a touchscreen depends on the electrostatic attraction and the applied normal pressure. The dependency of the finger-touchscreen interaction on the applied voltage and on several other parameters is also investigated using a mean field theory based on multiscale contact mechanics. We present detailed theoretical analysis of how the area of real contact and the friction force depend on contact parameters, and show that it is possible to further augment the friction force, and hence the tactile feedback displayed to the user by carefully choosing those parameters.