Haptic Pictures: Fit Judgments Predict Identification, Recognition Memory, and Confidence

Self-assembled thin films as alternative surface textures in assistive aids with users who are blind

Current tactile graphics primarily render tactile information for blind users through physical features, such as raised bumps or lines. However, the variety of distinctive physical features that can be created is effectively saturated, and alternatives to these physical features are not currently available for static tactile aids. Here, we explored the use of chemical modification through self-assembled thin films to generate distinctive textures in tactile aids. We used two silane precursors, n-butylaminopropyltrimethoxysilane and n-pentyltrichlorosilane, to coat playing card surfaces and investigated their efficacy as a tactile coating. We verified the surface coating process and examined their durability to repeated use by traditional materials characterization and custom mesoscale friction testing. Finally, we asked participants who were both congenitally blind and braille-literate to sort the cards based on touch. We found that participants were able to identify the correct coated card with 82% accuracy, which was significantly above chance, and two participants achieved 100% accuracy. This success with study participants demonstrates that surface coatings and surface modifications might augment or complement physical textures in next-generation tactile aids.

Drawing as a tool for investigating the nature of imagery representations of blind people: The case of the canonical size phenomenon

Several studies have shown that blind people, including those with congenital blindness, can use raised-line drawings, both for "reading" tactile graphics and for drawing unassisted. However, research on drawings produced by blind people has mainly been qualitative. The current experimental study was designed to investigate the under-researched issue of the size of drawings created by people with blindness. Participants (N = 59) varied in their visual status. Adventitiously blind people had previous visual experience and might use visual representations (e.g., when visualising objects in imagery/working memory). Congenitally blind people did not have any visual experience. The participant's task was to draw from memory common objects that vary in size in the real world. The findings revealed that both groups of participants produced larger drawings of objects that have larger actual sizes. This means that the size of familiar objects is a property of blind people's mental representations, regardless of their visual status. Our research also sheds light on the nature of the phenomenon of canonical size. Since we have found the canonical size effect in a group of people who are blind from birth, the assumption of the visual nature of this phenomenon - caused by the ocular-centric biases present in studies on drawing performance - should be revised.

Constituent Elements Affecting the Recognition of Tactile Graphics

Introduction: Many tactile graphics designed for individuals with visual impairments consider single factors. According to the results of our previous study, there may be interactions among scale, representation, and complexity factors. We conducted this integrative study with these three factors. Additionally, for the representation factor, we introduced a new level that mixed the two common levels (line drawing, (LD), and texture picture, (TP)), into a textured-line drawing, (TLD). Methods: We included 18 participants with congenital blindness. They were asked to identify and name tactile graphics. The design of tactile graphics involved three factors, each at different levels, including scale (large, medium, and small), representation (TP, LD, and TLD), and complexity (easy and complex). We recorded the identification time and accuracy and conducted three-way analysis of variance to investigate interactions. Results: The identification time for small-scale graphics was shorter than that for large-scale graphics. The accuracy for small-scale graphics was higher than medium-scale graphics. Under the TLD mode, the accuracy for small and medium-scale graphics was higher than that for large-scale graphics. For medium-scale graphics, TLD performed better than LD. Discussion: Because the sizes of small-scale graphics were similar to those of actual objects, they were easy to identify. If the TLD mode is used for medium-scale graphics, the components in the operation area can be completely presented, which is helpful for identification. However, if large-scale graphics are used under the TLD mode, the operation area is relatively small and difficult to identify. Implications for Practitioners: It is recommended to present objects at 1:1 under the small-scale. Under the medium-scale, the operation area can be presented by closed plains, while non-operation areas can be presented using lines. Under the large-scale, it is recommended to reduce the size of the graphics to an extent where both hands can be used to explore.

Analyzing Abstraction and Hierarchical Decision-Making in Absolute Identification by Information-Theoretic Bounded Rationality

In the face of limited computational resources, bounded rational decision theory predicts that information-processing should be concentrated on actions that make a significant contribution in terms of the utility achieved. Accordingly, information-processing can be simplified by choosing stereotypic actions that lead to satisfactory performance over a range of different inputs rather than choosing a specific action for each input. Such a set of similar inputs with similar action responses would then correspond to an abstraction that can be harnessed with possibly negligible loss in utility, but with potentially considerable savings in information-processing effort. Here we test this prediction in an identification task, where human subjects were asked to estimate the roundness of ellipses varying from a straight line to a perfect circle. Crucially, when reporting their estimates, subjects could choose between three different levels of precision corresponding to three levels of abstraction in a decision-making hierarchy. To induce changes in level selection, we manipulated the information-processing resources available at the perceptual and action stages by varying the difficulty of identifying the stimulus and by enforcing different response times in the action stage. In line with theoretical predictions, we find that subjects adapt their abstraction level depending on the available resources. We compare subjects' behavior to the maximum efficiency predicated by the bounded rational decision-making model and investigate possible sources of inefficiency.

Improved access to museum collections without vision: How museum visitors with very low or no vision perceive and process tactile–auditory pictures

This study investigated how museum visitors with very low or no vision perceived and processed tactile pictures and/or audio-descriptions of visual paintings. Two visual paintings were selected and a focus group was established ( N = 8). Qualitative interview and observation data were collected. This study found two types of museum visitors: those who explored the tactile picture first and those who rather listened to the audio-description. When exploring each element in the tactile picture, they all started by exploring the element’s global (shape) outline and, when struggling to recognise it, turned to the audio-description. They preferred the audio-description to start describing where their fingers were. Tactile texture attracted their attention, sparked their curiosity, and enabled them to create a mental image of the tactile picture, but also confused them. They preferred the global (element shape) outline to be straightened out, so that curves become angular, and texture only for targeting certain elements.

The Relevance of Object Size to the Recognizability of Drawings by Individuals with Congenital Blindness

Introduction:

The aim of this study was to determine whether individuals with congenital blindness make more recognizable drawings of known objects that are furniture sized (table, man, tree) rather than hand sized (egg, coconut, banana; Hypothesis 1). We also investigated whether knowledge that the tactile drawings had been produced by people who are blind increased judges’ perceptions of their recognizability (Hypothesis 2).

Methods:

The raised-line drawings were made by children and teenagers who are blind and had no prior experience in tactile graphics. After a minimal initial training in line drawing, the subjects were asked to draw six objects from memory. The judges used a 7-point Likert-type scale to assess recognizability. All objects were identified for the judges prior to their assessment. One group of judges was told that the drawings they were assessing were made by persons who are blind, while the second group was informed that the drawings were made without the use of sight.

Results:

The real-life size of the objects depicted in the drawings affected judges’ perception of their recognizability. Depictions of hand-sized objects were found to be less recognizable than were depictions of furniture-sized objects. Knowledge of the artists’ blindness had no effect on the judgments of recognizability.

Discussion:

Hypothesis 1 was confirmed, which suggests that individuals with congenital blindness have more difficulty in creating drawings of hand-sized objects than they did creating drawings of furniture-sized objects. Hypothesis 2 was not confirmed, which is inconsistent with the results of previous research in which the awareness of the artists’ disability status influenced the assessment of the artworks. Although the present research focused on recognizability, the issue in earlier research was aesthetic judgment.

Implications for practitioners:

Although few people would question the importance of illustrations in books for sighted children, the value of tactile graphics and the benefits of engaging children who have visual impairments in making and exploring raised-line drawings are not always understood. Full participation in subjects that rely on visual information such as geometry, art, and geography by learners who are born blind requires access to tactile graphics. Basic training in raised-line drawing may be sufficient for some children, particularly those who are older (aged 10 years or more), with congenital blindness who have never drawn before to create from memory recognizable drawings of known objects, especially larger objects.

Haptics and Projection: Drawings by Tracy, a Blind Adult

Outline drawings in a raised form were made by a blind woman, Tracy, who has been blind from very early in life. Highly practiced in drawing, she reports she is largely self-taught. To invoke matters of projection, she was asked to represent an object with faces slanting away from the observer, a fixed array from different vantage points, and sets of objects in depth. In particular, she drew a cube balanced on a vertex, three objects from different vantage points, receding rows of glasses, and a house. Her drawings included features of parallel and polar projection. Her use of these features may reflect an appreciation of direction from a vantage point, which observers deal with via haptics in everyday tasks. Tracy may have advanced drawing-development skills common to the blind and the sighted.

Beyond visual conventions: Rethinking the design of tactile diagrams

This article reviews the debate over the comprehension of two-dimensional microcapsule tactile pictures portraying three-dimensional information through the use of perspective and visual drawing conventions. It is proposed that three-dimensional haptic object recognition and its relationship to visual object recognition can inform the design of two-dimensional tactual displays. Experimental work is discussed which validates the use of such information in informing the design of 2-D tactile pictures.

Simultaneous Sketching Aids the Haptic Identification of Raised Line Drawings

Haptically identifying raised line drawings is difficult. We investigated whether a major component of this difficulty lies in acquiring, integrating, and maintaining shape information from touch. Wijntjes, van Lienen, Verstijnen, and Kappers reported that drawings which participants had failed to identify by touch alone could often subsequently be named if they were sketched. Thus, people sometimes needed to externalize haptically acquired information by making a sketch in order to be able to use it. We extended Wijntjes et al.'s task and found that sketching while touching improved drawing identification even more than sketching after touching, but only if people could see their sketches. Our results suggest that the slow, serial nature of information acquisition seriously hampers the haptic identification of raised line drawings relative to visually identifying line drawings. Simultaneous sketching may aid identification by reducing the burden on working memory and by helping to guide haptic exploration. This conclusion is consistent with the finding reported by Lawson and Bracken that 3-D objects are much easier to identify haptically than raised line drawings since, unlike for vision, simultaneously extracting global shape information is much easier haptically for 3-D stimuli than for line drawings.

The potential for developing a tactile communication system based on Blissymbolics

OBJECTIVE

To be useful for tactile communication, tactile stimuli need to be discriminable from each other. The objective of this study was to determine whether raised-line renderings of Blissymbols have the capacity for being developed into a tactile communication system as measured by their tactile discriminability.

METHODS

Tactile discrimination of Blissymbols was measured by performance on a task in which participants were asked to feel a target raised-line Blissymbol and then to find the target within an array containing the target and raised-line Blissymbol foils.

RESULTS

The vast majority of tactile Blissymbols had tactile discrimination scores of 90% accuracy or better.

CONCLUSION

Most raised-line Blissymbols can be tactilely discriminated from each other, indicating that they have the potential for being developed into a tactile communication system.

Recognizing familiar objects by hand and foot: Haptic shape perception generalizes to inputs from unusual locations and untrained body parts

The limits of generalization of our 3-D shape recognition system to identifying objects by touch was investigated by testing exploration at unusual locations and using untrained effectors. In Experiments 1 and 2, people found identification by hand of real objects, plastic 3-D models of objects, and raised line drawings placed in front of themselves no easier than when exploration was behind their back. Experiment 3 compared one-handed, two-handed, one-footed, and two-footed haptic object recognition of familiar objects. Recognition by foot was slower (7 vs. 13 s) and much less accurate (9 % vs. 47 % errors) than recognition by either one or both hands. Nevertheless, item difficulty was similar across hand and foot exploration, and there was a strong correlation between an individual's hand and foot performance. Furthermore, foot recognition was better with the largest 20 of the 80 items (32 % errors), suggesting that physical limitations hampered exploration by foot. Thus, object recognition by hand generalized efficiently across the spatial location of stimuli, while object recognition by foot seemed surprisingly good given that no prior training was provided. Active touch (haptics) thus efficiently extracts 3-D shape information and accesses stored representations of familiar objects from novel modes of input.

FingerSight: Fingertip Haptic Sensing of the Visual Environment

We present a novel device mounted on the fingertip for acquiring and transmitting visual information through haptic channels. In contrast to previous systems in which the user interrogates an intermediate representation of visual information, such as a tactile display representing a camera generated image, our device uses a fingertip-mounted camera and haptic stimulator to allow the user to feel visual features directly from the environment. Visual features ranging from simple intensity or oriented edges to more complex information identified automatically about objects in the environment may be translated in this manner into haptic stimulation of the finger. Experiments using an initial prototype to trace a continuous straight edge have quantified the user's ability to discriminate the angle of the edge, a potentially useful feature for higher levels analysis of the visual scene.

Intuitive tactile zooming for graphics accessed by individuals who are blind and visually impaired

One possibility of providing access to visual graphics for those who are visually impaired is to present them tactually: unfortunately, details easily available to vision need to be magnified to be accessible through touch. For this, we propose an "intuitive" zooming algorithm to solve potential problems with directly applying visual zooming techniques to haptic displays that sense the current location of a user on a virtual diagram with a position sensor and, then, provide the appropriate local information either through force or tactile feedback. Our technique works by determining and then traversing the levels of an object tree hierarchy of a diagram. In this manner, the zoom steps adjust to the content to be viewed, avoid clipping and do not zoom when no object is present. The algorithm was tested using a small, "mouse-like" display with tactile feedback on pictures representing houses in a community and boats on a lake. We asked the users to answer questions related to details in the pictures. Comparing our technique to linear and logarithmic step zooming, we found a significant increase in the correctness of the responses (odds ratios of 2.64:1 and 2.31:1, respectively) and usability (differences of 36% and 19%, respectively) using our "intuitive" zooming technique.

Exploratory procedures of tactile images in visually impaired and blindfolded sighted children: how they relate to their consequent performance in drawing

The aim of the present study was to compare the types of exploratory procedures employed by children when exploring bidimensional tactile patterns and correlate the use of these procedures with the children's shape drawing performance. 18 early blind children, 20 children with low vision and 24 age-matched blindfolded sighted children aged approximately 7 or 11 years were included in the study. The children with a visual handicap outperformed the sighted children in terms of haptic exploration and did not produce less recognizable drawings than their sighted counterparts. Close relationships were identified between the types of exploratory procedures employed by the children and their subsequent drawing performance, regardless of visual status. This close link between action and perception in the haptic modality indicates the importance of training blind children in exploratory procedures at an early age.

The haptic recognition of geometrical shapes in congenitally blind and blindfolded adolescents: is there a haptic prototype effect?

BACKGROUND

It has been shown that visual geometrical shape categories (rectangle and triangle) are graded structures organized around a prototype as demonstrated by perception and production tasks in adults as well as in children. The visual prototypical shapes are better recognized than other exemplars of the categories. Their existence could emerge from early exposure to these prototypical shapes that are present in our visual environment. The present study examined the role of visual experience in the existence of prototypical shapes by comparing the haptic recognition of geometrical shapes in congenitally blind and blindfolded adolescents.

METHODOLOGY/PRINCIPAL FINDINGS

To determine whether the existence of a prototype effect (higher recognition of prototypical shapes than non prototypical shapes) depended on visual experience, congenitally blind and blindfolded sighted adolescents were asked to recognize in the haptic modality three categories of correct shapes (square, rectangle, triangle) varying in orientation (prototypical/canonical orientation vs. non prototypical/canonical orientation rotated by 45°) among a set of other shapes. A haptic prototype effect was found in the blindfolded sighted whereas no difference between prototypical and non prototypical correct shapes was observed in the congenitally blind. A control experiment using a similar visual recognition task confirmed the existence of a visual prototype effect in a group of sighted adolescents.

CONCLUSION/SIGNIFICANCE

These findings show that the prototype effect is not intrinsic to the haptic modality but depends on visual experience. This suggests that the occurrence of visual and haptic prototypical shapes in the recognition of geometrical shape seems to depend on visual exposure to these prototypical shapes existing in our environment.

Haptic Recognition of Emotions in Raised-Line Drawings by Congenitally Blind and Sighted Adults

15 sighted and 15 congenitally blind adults were to classify raised-line pictures of emotional faces through haptics. Whereas accuracy did not vary significantly between the two groups, the blind adults were faster at the task. These results suggest that raised-line pictures of emotional faces are intelligible to blind adults.

Haptic Object Recognition: How Important are Depth Cues and Plane Orientation?

Raised-line drawings of familiar objects are very difficult to identify with active touch only. In contrast, haptically explored real 3-D objects are usually recognised efficiently, albeit slower and less accurately than with vision. Real 3-D objects have more depth information than outline drawings, but also extra information about identity (eg texture, hardness, temperature). Previous studies have not manipulated the availability of depth information in haptic object recognition whilst controlling for other information sources, so the importance of depth cues has not been assessed. In the present experiments, people named plastic small-scale models of familiar objects. Five versions of bilaterally symmetrical objects were produced. Versions varied only in the amount of depth information: minimal for cookie-cutter and filled-in outlines, partial for squashed and half objects, and full for 3-D models. Recognition was faster and much more accurate when more depth information was available, whether exploration was with both hands or just one finger. Novices found it almost impossible to recognise objects explored with two hand-held probes whereas experts succeeded using probes regardless of the amount of depth information. Surprisingly, plane misorientation did not impair recognition. Unlike with vision, depth information, but not object orientation, is extremely important for haptic object recognition.

Haptic recognition of two-dimensional raised-line patterns by early-blind, late-blind, and blindfolded sighted adults

We investigated the role of visual experience and visual imagery in the processing of two-dimensional (2-D) tactile patterns. The performance of early-blind (EB), late-blind (LB), and blindfolded sighted (S) adults in the recognition of 2-D raised-line patterns was compared. We also examined whether recognition of 2-D tactile patterns depends on the type of memory strategy (eg spatial, visuo-spatial, verbal, and kinesthetic) used by EB, LB, and S participants to perform the task. Significant between-group differences in the recognition performance have not been found despite significant between-group differences in self-reported memory strategies. Recognition performance does not vary significantly with the strategy, but correlates positively with visuo-spatial imagery abilities in the S participants. These findings may be taken to suggest that the difficulties some blind people experience with tactile pictures are not due to difficulties in processing 2-D tactile patterns.

Look what I have felt: unidentified haptic line drawings are identified after sketching

The difficulty that observers experience when trying to identify a raised line drawing by touch is still largely unexplained. In this article, we show that observers who are unable to haptically identify a raised line drawing are suddenly able to do so after they have sketched on paper what they have in their mind. We conducted three experiments: first of all we show that this effect is robust; in the second experiment, we show that identification-after-sketching is caused by visual inspection of the sketch, and not caused by feedback in general; and in the third we show that sketches which were identified by the observers who produced them, were also identified by completely naive viewers. These experiments demonstrate that during raised line drawing identification the mental capacities required to interpret the stimulus seem to be inadequate: although enough pictorial information was present to produce a sketch which could even be identified by naive viewers, the stimulus could not be identified by haptic and mental processing alone. Furthermore, we investigated whether increasing the haptic perceptual field by using two hands instead of one hand had an influence on identification performance. We did indeed find that using two hands significantly increased identification. We use both results to discuss the underlying mechanisms of haptic raised line drawing identification.

The Influence of Picture Size on Recognition and Exploratory Behaviour in Raised-Line Drawings

We demonstrate the influence of picture size on haptic recognition and exploratory behaviour. The stimuli were raised-line drawings of everyday objects. Participants were instructed to think aloud during haptic exploration of the pictures. We measured the delay between initial correct speculation and final correct response. The results indicate that picture size influences accuracy but not response latency: large drawings are recognised more often but not faster. By analysing video recordings of the experiment we found that two-handed exploration increases when picture size increases and that, on average, 83% of the exploration time involves the use of two hands. The thinking-aloud data showed that the average time difference between the initial correct speculation and final correct response amounted to 23% of the total reaction time. We discuss our results with respect to the design of tactile aids and the ecological validity of single-finger exploration.

Looking at perspective pictures from too far, too close, and just right

A central problem for psychology is vision's reaction to perspective. In the present studies, observers looked at perspective pictures projected by square tiles on a ground plane. They judged the tile dimensions while positioned at the correct distance, farther or nearer. In some pictures, many tiles appeared too short to be squares, many too long, and many just right. The judgments were strongly affected by viewing from the wrong distance, eye height, and object orientation. The authors propose a 2-factor angles and ratios together (ART) theory, with the following factors: the ratio of the visual angles of the tile's sides and the angle between (a) the direction to the tile from the observer and (b) the perpendicular, from the picture plane to the observer, that passes through the central vanishing point.

Somatosensory processing and schizophrenia liability: proprioception, exteroceptive sensitivity, and graphesthesia performance in the biological relatives of schizophrenia patients

In the present study, the authors examined somatosensory processing in 30 biological relatives of persons with schizophrenia (hereafter called "schizophrenia relatives"), 30 biological relatives of persons with bipolar affective disorder (psychiatric family control subjects), and 30 healthy control subjects with no family history of psychopathology. All 3 groups completed a weight discrimination task, a 2-point discrimination task, and a complex cognitive somatosensory task (i.e., graphesthesia). The schizophrenia relatives performed significantly worse on all 3 somatosensory tasks compared with both the healthy control subjects and the bipolar relatives. The healthy control subjects and psychiatric family control subjects showed no significant differences on any of the somatosensory tasks. Within the weight discrimination and 2-point discrimination tasks, schizophrenia relatives showed group differences on the d' index, the measure of sensitivity, whereas all 3 groups did not differ on lnbeta, the measure of response bias, suggesting a genuine difference in weight and touch sensitivity. The d' value of the weight discrimination task was significantly associated with both the cognitive-perceptual factor and negative symptom factor of the clinical questionnaire (e.g., Schizotypal Personality Questionnaire; SPQ), whereas the 2-point discrimination d' value and graphesthesia scores were significantly associated only with the cognitive-perceptual factor of the SPQ. Implications for the possible relation between somatosensory task performance and schizophrenia liability are discussed.

Drawings from Gaia, a blind girl

Gaia, a totally blind girl, was asked to make raised-line drawings. Gaia's vision at best was peripheral. She draws out of interest, and has drawn since preschool with encouragement from her mother. She was asked to draw objects and scenes involving depth from a vantage point, eg a table from below, two cars (one behind the other), and two parallel rows of apples (receding from her, on a table top). Gaia represented space in her drawings using T-junctions for overlap, height in the picture plane, parallel projection, and inverse projection. That is, Gaia uses features of systems common in sighted children's drawings. The development of drawing in blind and sighted children may be similar in good measure because haptics provides access to many of the same spatial principles as vision.