MH-Pen: A Pen-Type Multi-Mode Haptic Interface for Touch Screens Interaction

Comparative Experimental Research on Haptic Display Methods of Virtual Surface Shape Based on Touch Screen

With the wide application of touch screen technology in daily life, more and more studies have tried to use touch screen as a carrier to convey the surface shape features of virtual objects to people. Current studies have proposed methods such as direct height mapping and haptic illusion based on lateral force to display the geometric shape of virtual surfaces on the touch screen. However, whether there are differences in the performance of these methods in displaying the shape features, and whether the lateral force-based haptic illusion can be used to display the shape features of complex surfaces, has not been fully explored. Aiming at this situation, in this article, we carry out user experiments on two regular shapes and eight natural images under displacement field and two types of lateral force fields with the help of three haptic devices previously designed. We also evaluate the effect of image smoothing on users' perception of natural image shape features. Experimental results show that the current shape display devices and methods that can simulate the lateral force field proposed by (Minsky et al., 1990) cannot achieve the display effect of the displacement field on complex shapes, and the shape display effect is the worst when only resistive lateral force feedback is employed. Nevertheless, since methods based on lateral force field, such as electrovibration, allow people to directly perceive the shape features of the virtual surface on the touch screen with their bare fingers, continuous research on this shape expression method still has a very attractive prospect.

Research on the Method of Displaying the Contour Features of Image to the Visually Impaired on the Touch Screen

Conveying image information to the blind or visually impaired (BVI) is an important means to improve their quality of life. The touch screen devices used daily are the potential carriers for BVI to perceive image information through touch. However, touch screen devices also have the disadvantages of limited computing power and lack of rich tactile experience. In order to help BVI to access images conveniently through the touch screen, we built an image contour display system based on vibrotactile feedback. In this paper, an image smoothing algorithm based on convolutional neural network that can run quickly on the touch screen device is first used to preprocess the image to improve the effect of contour extraction. Then, based on the haptic physiological characteristics of human beings, this paper proposes a method of using the improved MH-Pen to guide the BVI to perceive image contour on the touch screen. This paper introduces the extraction and expression methods of image contours in detail, and compares and analyzes the effects of the subjects' perception of image contours in two haptic display modes through two types of user experiments. The experimental results show that the image smoothing algorithm is useful and necessary to help obtain the main contour of the image and to ensure the real-time display of the contour, and the contour expression method based on the motion direction guidance helps the subjects recognize the contour of the image more effectively.

Overview and Exploitation of Haptic Tele-Weight Device in Virtual Shopping Stores

In view of the problem of e-commerce scams and the absence of haptic interaction, this research aims to introduce and create a tele-weight device for e-commerce shopping in smart cities. The objective is to use the proposed prototype to provide a brief overview of the possible technological advancements. When the tele-weight device is affixed over the head-mounted display, it allows the user to feel the item’s weight while shopping in the virtual store. Addressing the problem of having no physical interaction between the user (player) and a series game scene in virtual reality (VR) headsets, this research approach focuses on creating a prototype device that has two parts, a sending part and a receiving part. The sending part measures the weight of the object and transmits it over the cellular network to the receiver side. The virtual store user at the receiving side can thus realize the weight of the ordered object. The findings from this work include a visual display of the item’s weight to the virtual store e-commerce user. By introducing sustainability, this haptic technology-assisted technique can help the customer realize the weight of an object and thus have a better immersive experience. In the device, the load cell measures the weight of the object and amplifies it using the HX711 amplifier. However, some delay in the demonstration of the weight was observed during experimentation, and this indirectly altered the performance of the system. One set of the device is sited at the virtual store user premises while the sending end of the device is positioned at the warehouse. The sending end hardware includes an Arduino Uno device, an HX711 amplifier chip to amplify the weight from the load cell, and a cellular module (Sim900A chip-based) to transmit the weight in the form of an encoded message. The receiving end hardware includes a cellular module and an actuator involving a motor gear arrangement to demonstrate the weight of the object. Combining the fields of e-commerce, embedded systems, VR, and haptic sensing, this research can help create a more secure marketplace to attain a higher level of customer satisfaction.

Investigation of a Haptic Actuator Made with Magneto-Rheological Fluids for Haptic Shoes Applications

This paper presents a magneto-rheological (MR) actuator that can be easily inserted into haptic shoes and can haptically simulate the material properties of the ground. To increase the resistive force of the proposed actuator, we designed a movable piston having multiple operation modes of MR fluids. Further, the design of a solenoid coil was optimized to maximize the resistive force in a limited-sized MR actuator. Simulations were conducted to predict the actuation performance and to show that the magnetic flux flows well by forming a closed loop in the proposed actuator. The quantitative evaluation of the proposed actuator was investigated by measuring the resistive force as a function of the input current and its pressed depth. From the result, we found that the proposed actuator can create over 600 N by adjusting the input current.

VnStylus: A Haptic Stylus With Variable Tip Compliance

Variable stiffness tools have been shown to be advantageous for ensuring safety and improving stability, dynamic performance and energy efficiency of interaction tasks. In this article, we present the design, mathematical modeling, implementation, characterization and user evaluations of VnStylus, a stylus with hardware-based tip compliance modulation. The stiffness modulation of the stylus tip is achieved through transverse stiffness variations of axially loaded beams. This approach of mechanically-controlled stiffness variation is advantageous for a variable stiffness stylus, as it can be implemented using a fully compliant mechanism that alleviates the problem of friction forces dominating under miniaturization and allows for high fidelity stiffness display even for very low stiffness values. Thanks to its hardware-based impedance modulation, the device does not suffer from the bandwidth and the stable impedance rendering limitations of software-based impedance control strategies. Featuring a manual adjustment mechanism, the device does not necessitate actuators, sensor or electronics; hence, is lightweight, low cost, and robust. Experimental characterizations verify the large range of stiffness modulation that can be achieved and the accuracy of the equivalent stiffness model of the system. Human subject experiments provide evidence of the efficacy of the modulated stylus stiffness on human performance during several common interactions with styli.

Design and Characterization of an Actuated Drill Mockup for Orthopedic Surgical Training

Haptic feedback in virtual reality-based trainers for surgical bone drilling is mostly provided via impedance-controlled haptic devices. Due to this, the displayable maximum stiffness is limited. In addition, vibration feedback is often only of reduced fidelity. To overcome these shortcomings, we have developed a hand-held, actuated admittance-controlled drill mockup, comprising enhanced kinesthetic and tactile feedback. This article reports on design and characterization of the device, and highlights its use for training. Kinesthetic feedback is provided through haptic augmentation, employing a ball-screw mechanism acting on a retractable drill-bit. Feedback computation relies on admittance control, thus allowing for stable display of very high resistance forces, and thus material stiffness, which cannot be achieved with standard impedance-control approaches. For the tactile mechanism, a modified linear vibration actuator is directly attached to the mockup handle, improving signal transmission. Tactile feedback computation is based on an extension of a previously proposed power spectral density control method. Frequency-specific gains are adjusted in real-time, compensating for differences between desired and measured vibrations. The performance of the device is characterized in several experiments, including comparisons to drilling with a real drill into artificial bone samples. In addition, several user studies have been carried out. We illustrate the capability of the mockup to render bone samples with different material layer stiffness and thickness. Moreover, we show that the mockup system allows for the same training effect as when rehearsing with a real drill.

Development of Haptic Stylus for Manipulating Virtual Objects in Mobile Devices

In mobile devices, the screen size limits conveyance of immersive experiences; haptic feedback coupled with visual feedback is expected to have a better effect to maximize the level of immersion. Therefore, this paper presents a miniature tunable haptic stylus based on magnetorheological (MR) fluids to provide kinesthetic information to users. The designed stylus has a force generation, force transmission, and housing part; moreover, in the stylus, all three operating modes of MR fluids contribute to the haptic actuation to produce a wide range of resistive force generated by MR fluids in a limited size, thereby providing a variety of pressing sensations to users. A universal testing machine was constructed to evaluate haptic performance of the proposed haptic stylus, whose resistive force was measured with the constructed setup as a function of pressed depth and input current, and by varying the pressed depth and pressing speed. Under maximum input voltage, the stylus generates a wide range of resistive force from 2.33 N to 27.47 N, whereas under maximum pressed depth it varied from 1.08 N to 27.47 N with a corresponding change in voltage input from 0 V to 3.3 V. Therefore, the proposed haptic stylus can create varied haptic sensations.

FW-Touch: A Finger Wearable Haptic Interface With an MR Foam Actuator for Displaying Surface Material Properties on a Touch Screen

The haptic interface plays an increasingly important role in enhancing the realism and immersion of the user's interaction with the touch screen. Inspired by the wearable haptic system, this paper proposes a finger wearable device called FW-Touch for touch screen interaction. The device provides normal force, lateral force, and vibrotactile feedback for the interaction of the finger and the touch screen through three internally integrated actuators. By displaying the hardness, friction, and roughness of a virtual surface, the device is capable of simulating the active exploration and sensing process of the finger on a real surface. This paper describes the design and specifications of the FW-Touch, and details the design process of a magnetorheological (MR) foam actuator that uses a Hall sensor to correct the output force. Through physical measurements and psychophysical experiments, we comprehensively evaluated the force feedback performance of the FW-Touch and its ability in displaying the stiffness and friction of the virtual surface. The results show that improving the accuracy of force feedback is necessary for virtual stiffness display, and the accuracy and effectiveness of the FW-Touch in displaying virtual surface features can be confirmed from the measured stiffness and friction Weber fractions.