Three-dimensional tactile display for the blind

Height-renderable morphable tactile display enabled by programmable modulation of local stiffness in photothermally active polymer

Reconfigurable tactile displays are being used to provide refreshable Braille information; however, the delivered information is currently limited to an alternative of Braille because of difficulties in controlling the deformation height. Herein, we present a photothermally activated polymer-bilayer-based morphable tactile display that can programmably generate tangible three-dimensional topologies with varying textures on a thin film surface. The morphable tactile display was composed of a heterogeneous polymer structure that integrated a stiffness-tunable polymer into a light-absorbing elastomer, near-infra-red light-emitting diode (NIR-LED) array, and small pneumatic chamber. Topological expression was enabled by producing localized out-of-plane deformation that was reversible, height-adjustable, and latchable in response to light-triggered stiffness modulation at each target area under switching of stationary pneumatic pressure. Notably, the tactile display could express a spatial softness map of the latched topology upon re-exposing the target areas to modulated light from the NIR-LED array. We expect the developed tactile display to open a pathway for generating high-dimensional tactile information on electronic devices and enable realistic interaction in augmented and virtual environments.

Universal pictures: A lithophane codex helps teenagers with blindness visualize nanoscopic systems

People with blindness have limited access to the high-resolution graphical data and imagery of science. Here, a lithophane codex is reported. Its pages display tactile and optical readouts for universal visualization of data by persons with or without eyesight. Prototype codices illustrated microscopy of butterfly chitin-from N-acetylglucosamine monomer to fibril, scale, and whole insect-and were given to high schoolers from the Texas School for the Blind and Visually Impaired. Lithophane graphics of Fischer-Spier esterification reactions and electron micrographs of biological cells were also 3D-printed, along with x-ray structures of proteins (as millimeter-scale 3D models). Students with blindness could visualize (describe, recall, distinguish) these systems-for the first time-at the same resolution as sighted peers (average accuracy = 88%). Tactile visualization occurred alongside laboratory training, synthesis, and mentoring by chemists with blindness, resulting in increased student interest and sense of belonging in science.

Research on the Method of Displaying the Contour Features of Image to the Visually Impaired on the Touch Screen

Conveying image information to the blind or visually impaired (BVI) is an important means to improve their quality of life. The touch screen devices used daily are the potential carriers for BVI to perceive image information through touch. However, touch screen devices also have the disadvantages of limited computing power and lack of rich tactile experience. In order to help BVI to access images conveniently through the touch screen, we built an image contour display system based on vibrotactile feedback. In this paper, an image smoothing algorithm based on convolutional neural network that can run quickly on the touch screen device is first used to preprocess the image to improve the effect of contour extraction. Then, based on the haptic physiological characteristics of human beings, this paper proposes a method of using the improved MH-Pen to guide the BVI to perceive image contour on the touch screen. This paper introduces the extraction and expression methods of image contours in detail, and compares and analyzes the effects of the subjects' perception of image contours in two haptic display modes through two types of user experiments. The experimental results show that the image smoothing algorithm is useful and necessary to help obtain the main contour of the image and to ensure the real-time display of the contour, and the contour expression method based on the motion direction guidance helps the subjects recognize the contour of the image more effectively.

Design and Analysis of High-Resolution Electrostatic Adhesive Brakes Towards Static Refreshable 2.5D Tactile Shape Display

Tactile displays are haptic devices capable of rendering shape and texture information. Unsolved challenges in building tactile shape displays include their traditionally large form factors, low spatial resolution, and high costs. Using electrostatic adhesion to individually brake each pin and a single platform for global actuation, we developed a prototype static refreshable tactile shape display with high spatial resolution (1.7 mm pitch, 0.8 mm pin width; 4 mm pitch, 1.6 mm pin width), high resistance force (76.3 gf static-loading force per pin for 1.6 mm width) and low cost ($0.11 USD per pin for raw material). We present an analytical model of our electroadhesive brake mechanism and evaluate its maximum contact force and robustness in various conditions. To demonstrate the mechanism's potential, we built a static tactile shape display prototype with a 4×2 array of pins controlled using electroadhesive brakes. To further increase maximsum contact force allowed by our device, we develop and evaluate a global mechanical clutch which can be engaged during user interaction. A user study is carried out to compare our static tactile shape display's performance with printed 2.5D tactile graphics in a shape recognition task, and comparable shape recognition rates and response times are observed.

The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability

Vision loss has severe impacts on physical, social and emotional well-being. The education of blind children poses issues as many scholar disciplines (e.g., geometry, mathematics) are normally taught by heavily relying on vision. Touch-based assistive technologies are potential tools to provide graphical contents to blind users, improving learning possibilities and social inclusion. Raised-lines drawings are still the golden standard, but stimuli cannot be reconfigured or adapted and the blind person constantly requires assistance. Although much research concerns technological development, little work concerned the assessment of programmable tactile graphics, in educative and rehabilitative contexts. Here we designed, on programmable tactile displays, tests aimed at assessing spatial memory skills and shapes recognition abilities. Tests involved a group of blind and a group of low vision children and adolescents in a four-week longitudinal schedule. After establishing subject-specific difficulty levels, we observed a significant enhancement of performance across sessions and for both groups. Learning effects were comparable to raised paper control tests: however, our setup required minimal external assistance. Overall, our results demonstrate that programmable maps are an effective way to display graphical contents in educative/rehabilitative contexts. They can be at least as effective as traditional paper tests yet providing superior flexibility and versatility.

Haptic 3D Surface Representation of Table-Based Data for People With Visual Impairments

The UN Convention on the Rights of Persons with Disabilities Article 24 states that “States Parties shall ensure inclusive education at all levels of education and life long learning.” This article focuses on the inclusion of people with visual impairments in learning processes including complex table-based data. Gaining insight into and understanding of complex data is a highly demanding task for people with visual impairments. Especially in the case of table-based data, the classic approaches of braille-based output devices and printing concepts are limited. Haptic perception requires sequential information processing rather than the parallel processing used by the visual system, which hinders haptic perception to gather a fast overview of and deeper insight into the data. Nevertheless, neuroscientific research has identified great dependencies between haptic perception and the cognitive processing of visual sensing. Based on these findings, we developed a haptic 3D surface representation of classic diagrams and charts, such as bar graphs and pie charts. In a qualitative evaluation study, we identified certain advantages of our relief-type 3D chart approach. Finally, we present an education model for German schools that includes a 3D printing approach to help integrate students with visual impairments.

The role of tactile feedback in grip force during laparoscopic training tasks

BACKGROUND

Laparoscopic minimally invasive surgery has revolutionized surgical care by reducing trauma to the patient, thereby decreasing the need for medication and shortening recovery times. During open procedures, surgeons can directly feel tissue characteristics. However, in laparoscopic surgery, tactile feedback during grip is attenuated and limited to the resistance felt in the tool handle. Excessive grip force during laparoscopic surgery can lead to tissue damage. Providing additional supplementary tactile feedback may allow subjects to have better control of grip force and identification of tissue characteristics, potentially decreasing the learning curve associated with complex minimally invasive techniques.

METHODS

A tactile feedback system has been developed and integrated into a modified laparoscopic grasper that allows forces applied at the grasper tips to be felt by the surgeon's hands. In this study, 15 subjects (11 novices, 4 experts) were asked to perform single-handed peg transfers using these laparoscopic graspers in three trials (feedback OFF, ON, OFF). Peak and average grip forces (newtons) during each grip event were measured and compared using a Wilcoxon ranked test in which each subject served as his or her own control.

RESULTS

After activating the tactile feedback system, the novice subject population showed significant decreases in grip force (p < 0.003). When the system was deactivated for the third trial, there were significant increases in grip force (p < 0.003). Expert subjects showed no significant improvements with the addition of tactile feedback (p > 0.05 in all cases).

CONCLUSION

Supplementary tactile feedback helped novice subjects reduce grip force during the laparoscopic training task but did not offer improvements for the four expert subjects. This indicates that tactile feedback may be beneficial for laparoscopic training but has limited long-term use in the nonrobotic setting.

Web Navigation Tool for Visually Impaired People

Internet access has become a necessity for all sectors in communities. Conventional web navigation means hinder access to the visually impaired. Presented is a low cost tactile web navigator device aimed at enabling blind people to have efficient and economic internet accessibility. Presented in this paper centers around a microcontroller that communicates with a proxy transcoder browser server that runs on a PC to acquire the text from a target web page. An array of solenoid coils is used to output the text in tactile form to the end user. The device has the ability to output the text in Braille language format mode or in plain English character format using the usual dot-matrix representations mode. The navigator can also be configured to output the text in other languages using the Pattern-Generator tool Many navigation functions like loading a webpage address, clicking on links and entering data (E-mails, passwords, etc) have been provided. The navigator was tested by blind volunteer with excellent results. The cost of the implementation is an impressive ~12% of the price of commercial Braille displays.

Touching force response of the piezoelectric Braille cell

PURPOSE

The objective of this research is to investigate dynamic responses of the piezoelectric Braille cell when it is subjected to both electrical signal and touching force.

METHOD

Physical behavior of the piezoelectric actuator inside the piezoelectric Braille cell is analyzed. The mathematical model of the piezoelectric Braille system is presented. Then, data of visually impaired people using a Braille Note is studied as design information and a reference input for calculation of the piezoelectric Braille response under the touching force.

RESULTS

The results show dynamic responses of the piezoelectric Braille cell. The designed piezoelectric bimorph has a settling time of 0.15 second. The relationship between the Braille dot height and applied voltage is linear. The behavior of the piezoelectric Braille dot when it is touched during operation shows that the dot height is decreased as the force increases.

CONCLUSIONS

The result provides understanding of the piezoelectric Braille cell behavior under both touching force and electrical excitation simultaneously. This is the important issue for the design and development of piezoelectric Braille cells in senses of controlling Braille dot displacement or force-feedback in the future.

A haptic feedback system for lower-limb prostheses

A haptic feedback system has been developed to provide sensory information to patients with lower-limb prostheses or peripheral neuropathy. Piezoresistive force sensors were mounted against four critical contact points of the foot to collect and relay force information to a system controller, which in turn drives four corresponding pneumatically controlled balloon actuators. The silicone-based balloon actuators were mounted on a cuff worn on the middle thigh, with skin contacts on the posterior, anterior, medial, and lateral surfaces of the thigh. Actuator characterization and human perceptual testing were performed to determine the effectiveness of the system in providing tactile stimuli. The actuators were determined to have a monotonic input pressure-vertical deflection response. Six normal subjects wearing the actuator cuff were able to differentiate inflation patterns, directional stimuli and discriminate between three force levels with 99.0%, 94.8%, and 94.4% accuracy, respectively. With force sensors attached to a shoe insole worn by an operator, subjects were able to correctly indicate the movements of the operator with 95.8% accuracy. These results suggest that the pneumatic haptic feedback system design is a viable method to provide sensory feedback for the lower limbs.

Psychophysical experiment of vibrotactile pattern perception by human fingertip

In this paper, a preliminary experimentation is carried out to investigate the necessary stimulation conditions for a pattern display that may be used for the blind to display obstacle distribution. To design an efficient vibrotactile display device of array-type and compact size, the effect of some main parameters, such as the amplitude and frequency of vibration and the spacing and quantity of vibrated pins on vibrotactile perception, should be considered. Through experiment and analysis, the minimal amplitude of vibrating stimuli and the spacing effect on the perception of vibrotactile pattern are investigated and discussed. Also an appropriate type of actuator satisfying the requirements of a compact tactile display device for the effective transfer of pattern information is discussed.

Optimizing the use of an artificial tongue-placed tactile biofeedback for improving ankle joint position sense in humans

The performance of an artificial tongue-placed tactile biofeedback device for improving ankle joint position sense was assessed in 12 young healthy adults using an active matching task. The underlying principle of this system consisted of supplying individuals with supplementary information about the position of the matching ankle relative to the reference ankle position through a tongue-placed tactile output device generating electrotactile stimulation on a 36- point (6 x 6) matrix held against the surface of the tongue dorsum. Precisely, (1) no electrodes were activated when both ankles were in a similar angular position within predetermined "angular dead zone" (ADZ); (2) 12 electrodes (2 x 6) of the anterior and posterior zones of the matrix were activated (corresponding to the stimulation of the front and rear portion of the tongue) when the matching ankle was in too plantar and dorsiflexed position relative to the reference ankle, respectively. The effects of two ADZ values of 0.5 degrees and 1.5 degrees were evaluated. Results showed (1) more accurate and more consistent matching performances with than without biofeedback and (2) more accurate and more consistent ankle joint matching performances when using the biofeedback device with the smaller ADZ value. These findings suggest that (1) electrotactile stimulation of the tongue can be used to improve ankle joint proprioception and (2) this improvement can be increased through an appropriate specification of the ADZ parameter provided by the biofeedback system. Further investigations are needed to strengthen the potential clinical value of this device.

Accurate presentation of the target range with a 2-D tactile stimulator array for acoustic vision substitute system

Existing vision substitute systems have insufficient spatial resolution to provide environmental information. To present detailed spatial information, we propose a stimulation method to present the target range accurately using a 2-D tactile stimulator array. A user selects the measurement range and the system presents targets within the range. The user changes the measurement range, and then compares the presented targets in the latter measurement range with that in the former one. The targets, presented in one measurement range and not in the other measurement range, exist between the former and latter ranges. We examine and confirm this method experimentally.

An electrical stimulator for sensory substitution

This work presents an electrical stimulator system for use in sensory substitution (SS), as a mobility aid for visually handicapped people. The whole system passes visual information via cutaneous stimulation, and consists of a webcam, a PC, dedicated hardware to generate stimuli and a 15 x 20 electrode matrix. The same system can also be used in psycophysical and somesthesic research, or even for SS of deaf people, by changing the input signal from a camera to a microphone, and adapting its control software. Circuits for pixel addressing, for signal generation and for switching are described, as well as the software involved in generating a pulse train, which configures the stimuli patterns.

Visual to tactile conversion of vector graphics

Methods to automatically convert graphics into raised-line images have been recently investigated. In this paper, concepts from previous research are extended to the vector graphics case, producing tactile pictures in which important features are emphasized. The proposed algorithm extracts object boundaries and employs a classification process, based on a graphic's hierarchical structure, to determine critical outlines. A single parameter is introduced into the classification process, enabling users to tailor graphics to their own preferences. The resulting outlines are printed using a Braille printer to produce tactile output. Critical outlines are embossed with raised dots of highest height while other lines and details are embossed with a lower height. Psychophysical experiments including discrimination, identification, and comprehension are utilized to evaluate and compare the proposed algorithm. Results indicate that the proposed method outperforms other methods in all three considered tasks. The results also show that emphasizing important features significantly increases comprehension of tactile graphics, validating the proposed method's effectiveness in conveying visual information.

Graphical tactile displays for visually-impaired people

This paper presents an up-to-date survey of graphical tactile displays. These devices provide information through the sense of touch. At best, they should display both text and graphics (text may be considered a type of graphic). Graphs made with shapeable sheets result in bulky items awkward to store and transport; their production is expensive and time-consuming and they deteriorate quickly. Research is ongoing for a refreshable tactile display that acts as an output device for a computer or other information source and can present the information in text and graphics. The work in this field has branched into diverse areas, from physiological studies to technological aspects and challenges. Moreover, interest in these devices is now being shown by other fields such as virtual reality, minimally invasive surgery and teleoperation. It is attracting more and more people, research and money. Many proposals have been put forward, several of them succeeding in the task of presenting tactile information. However, most are research prototypes and very expensive to produce commercially. Thus the goal of an efficient low-cost tactile display for visually-impaired people has not yet been reached.

Figure and texture presentation capabilities of a tactile mouse equipped with a display pad of stimulus pins

To obtain specifications for a tactile display that would be effective in virtual reality and tele-existence systems, we have developed two types of matrix-type experimental tactile displays. One is for virtual figures (display A) and the other is for virtual textures (display B). Display A's pad has a 4 × 6 array of stimulus pins, each 0.8 mm in diameter. Three pad configurations, in which distances between any two adjacent pins (pin pitch) are 1.2, 1.9, or 2.5 mm, were developed to examine the influence of distance on a human operator's determination of virtual figures. Display B has an 8 × 8 array of stimulus pins, each 0.3 mm in diameter and with 1-or 1.8-mm pin pitch, because presentation of virtual textures was presumed to require a higher pin density. To establish a design method for these matrix-type tactile displays, we performed a series of psychophysical experiments using displays A and B. By evaluating variations in the correct answer percentage and threshold caused by different pin arrays and different pin strokes, we determined under what conditions the operator could best feel the virtual figures and textures. The results revealed that the two-point threshold should be adopted as the pitch between pins in the design of the tactile display, that a pin stroke should exceed 0.25 mm, and that the adjustment method is the most appropriate to evaluate the capabilities of tactile displays. Finally, when we compared the virtual texture with the real texture, we found that the threshold for the real texture is almost 1/3rd that of the virtual texture. This result implies that it is effective to present variations in patterns caused by rotation and variation in shearing force, itself produced by relative motion between the finger surface and object surface.

Human interface and transmit frequency control for the through-air acoustic real-time high resolution vision substitute system

Existing vision substitute systems are not useful as navigation system due to the limitation of spatial and time resolution. In this study we propose a transmit control method free from range aliasing for a high resolution acoustic vision substitute systems, which we previously proposed. We also examine a human-machine information transfer method with a vibrotactile stimulator array consisting of 13 &#215; 21 elements. It presents the target area of 30 degree &#215; 60 degree by the sampling interval of 1 degree at the center. The system presents range, direction, and surface topography of targets to the subject.

A dynamic tactile map as a tool for space organization perception: application to the design of an electronic travel aid for visually impaired and blind people

This paper addresses a new concept of visuotactile human-machine interface for a certain representation of the peripersonnal space, especially useful for visually impaired and blind people navigation. The proposed space representation have been successfully implemented on a tactile device and validated via series of experiments involving naive blindfolded sighted people. Results show that is possible to interact with the space via the proposed tactile representation.

System development for multichannel electrotactile stimulation on the lips

Multichannel pulsatile stimulation is essential for electrotactile presentation of spatiotemporal patterns in sensory substitution through the sense of touch. In this paper, a system for multichannel electrotactile stimulation is presented for studying electrotactile sensation on the lips, a site that has been shown to be highly sensitive but never been assessed for electrotactile applications. The system utilized a demultiplexing scheme to deliver voltage-clamped pulse waveforms to 32 channels corresponding to an array of 4x8 stimulators. Initial testing of electrotactile presentation on the lips was performed in human subject experiments on threshold measurement and two-line separation. For an array of hemispherical stimulators each 700 microm in diameter, the threshold values on the lower lip were averaged at only 8.2 V and intensities for well-perceived stimulation were averaged at around 14.2 V. The testing results suggested that lips were promising sites for electrotactile applications due to their high sensitivity to electrotactile stimulation at extremely low intensities.

Space cognitive map as a tool for navigation for visually impaired

Human navigation is based on space mental representations built from several sensory data. This paper investigates the interaction between the visual and tactile modalities during navigation within the experimental platform "perception-movement". A new space representation for navigation and its visual-to-tactile coding are proposed. Four experiments have validated this new coding through some basic navigation tasks. Results obtained show that it is possible to integrate the proposed space representation into a portable navigation tool which could be useful for navigation assistance of the blind/visual impaired.

Transfer information enhancement with a 2-D tactile stimulator array for acoustic vision substitute system

Existing vision substitute systems provide insufficient information as a navigation system. To present spatial information in detail, we propose a stimulation method for transfer information enhancement by using a 2-D tactile stimulator array. Stimulators are divided into several groups. Since each stimulator group is activated alternately, the interval of stimulators can be shortened to less than the two-point discrimination threshold. In the case that stimulators are divided into two and four groups, the number of stimulators increase to twice and four times of that in the case of the two-point discrimination threshold. We examine and confirm this method experimentally.

Phantom-Based Multimodal Interactions for Medical Education and Training: The Munich Knee Joint Simulator

Simulation environments based on virtual reality technologies can support medical education and training. In this paper, the novel approach of an "interactive phantom" is presented that allows a realistic display of haptic contact information typically generated when touching and moving human organs or segments. The key idea of the haptic interface is to attach passive phantom objects to a mechanical actuator. The phantoms look and feel as real anatomical objects. Additional visualization of internal anatomical and physiological information and sound generated during the interaction with the phantom yield a multimodal approach that can increase performance, didactic value, and immersion into the virtual environment. Compared to classical approaches, this multimodal display is convenient to use, provides realistic tactile properties, and can be partly adjusted to different, e.g., pathological properties. The interactive phantom is exemplified by a virtual human knee joint that can support orthopedic education, especially for the training of clinical knee joint evaluation. It is suggested that the technical principle can be transferred to many other fields of medical education and training such as obstetrics and dentistry.

Pattern identification as a function of stimulation on a fingertip-scanned electrotactile display

Two studies were conducted to determine the effect of stimulation current on pattern perception on a 49-point fingertip-scanned electrotactile (electrocutaneous) display. Performance increased monotonically from near chance levels at the lowest sub-threshold current levels tested to approximately 90% at the highest comfortable current levels. This suggests the existence of a tradeoff between spatial performance and usable "gray scale" range in electrotactile presentation of graphical information.

Pattern identification and perceived stimulus quality as a function of stimulation waveform on a fingertip-scanned electrotactile display

The effect of stimulation waveform on pattern perception was investigated on a 49-point fingertip-scanned electrotactile (electrocutaneous) display. Waveform variables burst frequency (F), number of pulses per burst (NPB), and pulse repetition rate (PRR) were varied in a factorial design. Contrast reduction was used to limit performance of perceiving a 1-tactor gap defined within a 3 x 3 tactor outline square. All three variables accounted for significant variations in performance with higher levels of F and NPB and lower levels of PRR, leading to better performance. In addition, we collected qualitative data on each waveform, and the qualitative differences were related to performance (e.g., waveforms perceived as having a more localized sensation were correlated with better pattern identification performance than those waveforms perceived as more broad). We also investigated the effect of stimulation contrast on pattern perception.