Echoic Sensory Substitution Information in a Single Obstacle Circumvention Task

A direct comparison of sound and vibration as sources of stimulation for a sensory substitution glove

Sensory substitution devices (SSDs) facilitate the detection of environmental information through enhancement of touch and/or hearing capabilities. Research has demonstrated that several tasks can be successfully completed using acoustic, vibrotactile, and multimodal devices. The suitability of a substituting modality is also mediated by the type of information required to perform the specific task. The present study tested the adequacy of touch and hearing in a grasping task by utilizing a sensory substitution glove. The substituting modalities inform, through increases in stimulation intensity, about the distance between the fingers and the objects. A psychophysical experiment of magnitude estimation was conducted. Forty blindfolded sighted participants discriminated equivalently the intensity of both vibrotactile and acoustic stimulation, although they experienced some difficulty with the more intense stimuli. Additionally, a grasping task involving cylindrical objects of varying diameters, distances and orientations was performed. Thirty blindfolded sighted participants were divided into vibration, sound, or multimodal groups. High performance was achieved (84% correct grasps) with equivalent success rate between groups. Movement variables showed more precision and confidence in the multimodal condition. Through a questionnaire, the multimodal group indicated their preference for using a multimodal SSD in daily life and identified vibration as their primary source of stimulation. These results demonstrate that there is an improvement in performance with specific-purpose SSDs, when the necessary information for a task is identified and coupled with the delivered stimulation. Furthermore, the results suggest that it is possible to achieve functional equivalence between substituting modalities when these previous steps are met.

Partial visual loss disrupts the relationship between judged room size and sound source distance

Visual spatial information plays an important role in calibrating auditory space. Blindness results in deficits in a number of auditory abilities, which have been explained in terms of the hypothesis that visual information is needed to calibrate audition. When judging the size of a novel room when only auditory cues are available, normally sighted participants may use the location of the farthest sound source to infer the nearest possible distance of the far wall. However, for people with partial visual loss (distinct from blindness in that some vision is present), such a strategy may not be reliable if vision is needed to calibrate auditory cues for distance. In the current study, participants were presented with sounds at different distances (ranging from 1.2 to 13.8 m) in a simulated reverberant (T₆₀ = 700 ms) or anechoic room. Farthest distance judgments and room size judgments (volume and area) were obtained from blindfolded participants (18 normally sighted, 38 partially sighted) for speech, music, and noise stimuli. With sighted participants, the judged room volume and farthest sound source distance estimates were positively correlated (p < 0.05) for all conditions. Participants with visual losses showed no significant correlations for any of the conditions tested. A similar pattern of results was observed for the correlations between farthest distance and room floor area estimates. Results demonstrate that partial visual loss disrupts the relationship between judged room size and sound source distance that is shown by sighted participants.

Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution

Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution.

The accuracy of auditory spatial judgments in the visually impaired is dependent on sound source distance

Blindness leads to substantial enhancements in many auditory abilities, and deficits in others. It is unknown how severe visual losses need to be before changes in auditory abilities occur, or whether the relationship between severity of visual loss and changes in auditory abilities is proportional and systematic. Here we show that greater severity of visual loss is associated with increased auditory judgments of distance and room size. On average participants with severe visual losses perceived sounds to be twice as far away, and rooms to be three times larger, than sighted controls. Distance estimates for sighted controls were most accurate for closer sounds and least accurate for farther sounds. As the severity of visual impairment increased, accuracy decreased for closer sounds and increased for farther sounds. However, it is for closer sounds that accurate judgments are needed to guide rapid motor responses to auditory events, e.g. planning a safe path through a busy street to avoid collisions with other people, and falls. Interestingly, greater visual impairment severity was associated with more accurate room size estimates. The results support a new hypothesis that crossmodal calibration of audition by vision depends on the severity of visual loss.

A low-cost 2-D video system can accurately and reliably assess adaptive gait kinematics in healthy and low vision subjects

3-D gait analysis is the gold standard but many healthcare clinics and research institutes would benefit from a system that is inexpensive and simple but just as accurate. The present study examines whether a low-cost 2-D motion capture system can accurately and reliably assess adaptive gait kinematics in subjects with central vision loss, older controls, and younger controls. Subjects were requested to walk up and step over a 10 cm high obstacle that was positioned in the middle of a 4.5 m walkway. Four trials were simultaneously recorded with the Vicon motion capture system (3-D system) and a video camera that was positioned perpendicular to the obstacle (2-D system). The kinematic parameters (crossing height, crossing velocity, foot placement, single support time) were calculated offline. Strong Pearson’s correlations were found between the two systems for all parameters (average r = 0.944, all p < 0.001). Bland-Altman analysis showed that the agreement between the two systems was good in all three groups after correcting for systematic biases related to the 2-D marker positions. The test-retest reliability for both systems was high (average ICC = 0.959). These results show that a low-cost 2-D video system can reliably and accurately assess adaptive gait kinematics in healthy and low vision subjects.

Distance Estimation by Blindfolded Sighted Participants Using Echolocation

Auditory perceived distance can be distorted in one’s internal representation. Thus, the present study examined whether blindfolded sighted participants could reduce the bias and preserve estimated distance for 5 to 15 s using echolocation. The participants performed a delayed reproduction task that consisted of testing sessions on 2 separate days in which the target distance was manipulated from 20 to 50 cm. Participants were blindfolded and asked to reproduce the distance of a target after a temporal delay of several seconds using click bursts produced by a loudspeaker. The testing session was preceded by a practice session that included training and feedback. The relationship between estimated and actual distances was approximated based on a power function and the over- and underestimation of the target distance on each test day. Although participants showed systematic bias in distance estimation on both days, participants changed their bias in the second session by shifting reproduced locations closer to their bodies. The accuracy and consistency of their responses improved across the 2 days. Neither accuracy nor consistency was affected by the retention intervals. These enhancements of performance might be due to improved hearing ability or calibration of internal spatial references through a practice session.

Quantifying the impact on navigation performance in visually impaired: Auditory information loss versus information gain enabled through electronic travel aids

This study’s purpose was to analyze and quantify the impact of auditory information loss versus information gain provided by electronic travel aids (ETAs) on navigation performance in people with low vision. Navigation performance of ten subjects (age: 54.9±11.2 years) with visual acuities >1.0 LogMAR was assessed via the Graz Mobility Test (GMT). Subjects passed through a maze in three different modalities: ‘Normal’ with visual and auditory information available, ‘Auditory Information Loss’ with artificially reduced hearing (leaving only visual information), and ‘ETA’ with a vibrating ETA based on ultrasonic waves, thereby facilitating visual, auditory, and tactile information. Main performance measures comprised passage time and number of contacts. Additionally, head tracking was used to relate head movements to motion direction. When comparing ‘Auditory Information Loss’ to ‘Normal’, subjects needed significantly more time (p<0.001), made more contacts (p<0.001), had higher relative viewing angles (p = 0.002), and a higher percentage of orientation losses (p = 0.011). The only significant difference when comparing ‘ETA’ to ‘Normal’ was a reduced number of contacts (p<0.001). Our study provides objective, quantifiable measures of the impact of reduced hearing on the navigation performance in low vision subjects. Significant effects of ‘Auditory Information Loss’ were found for all measures; for example, passage time increased by 17.4%. These findings show that low vision subjects rely on auditory information for navigation. In contrast, the impact of the ETA was not significant but further analysis of head movements revealed two different coping strategies: half of the subjects used the ETA to increase speed, whereas the other half aimed at avoiding contacts.

Strategies for obstacle avoidance during walking in the cat

Avoiding obstacles is essential for successful navigation through complex environments. This study aimed to clarify what strategies are used by a typical quadruped, the cat, to avoid obstacles during walking. Four cats walked along a corridor 2.5 m long and 25 or 15 cm wide. Obstacles, small round objects 2.5 cm in diameter and 1 cm in height, were placed on the floor in various locations. Movements of the paw were recorded with a motion capture and analysis system (Visualeyez, PTI). During walking in the wide corridor, cats' preferred strategy for avoiding a single obstacle was circumvention, during which the stride direction changed while stride duration and swing-to-stride duration ratio were preserved. Another strategy, stepping over the obstacle, was used during walking in the narrow corridor, when lateral deviations of walking trajectory were restricted. Stepping over the obstacle involved changes in two consecutive strides. The stride preceding the obstacle was shortened, and swing-to-stride ratio was reduced. The obstacle was negotiated in the next stride of increased height and normal duration and swing-to-stride ratio. During walking on a surface with multiple obstacles, both strategies were used. To avoid contact with the obstacle, cats placed the paw away from the object at a distance roughly equal to the diameter of the paw. During obstacle avoidance cats prefer to alter muscle activities without altering the locomotor rhythm. We hypothesize that a choice of the strategy for obstacle avoidance is determined by minimizing the complexity of neuro-motor processes required to achieve the behavioral goal.NEW & NOTEWORTHY In a study of feline locomotor behavior we found that the preferred strategy to avoid a small obstacle is circumvention. During circumvention, stride direction changes but length and temporal structure are preserved. Another strategy, stepping over the obstacle, is used in narrow walkways. During overstepping, two strides adjust. A stride preceding the obstacle decreases in length and duration. The following stride negotiating the obstacle increases in height while retaining normal temporal structure and nearly normal length.

Blindness enhances auditory obstacle circumvention: Assessing echolocation, sensory substitution, and visual-based navigation

Performance for an obstacle circumvention task was assessed under conditions of visual, auditory only (using echolocation) and tactile (using a sensory substitution device, SSD) guidance. A Vicon motion capture system was used to measure human movement kinematics objectively. Ten normally sighted participants, 8 blind non-echolocators, and 1 blind expert echolocator navigated around a 0.6 x 2 m obstacle that was varied in position across trials, at the midline of the participant or 25 cm to the right or left. Although visual guidance was the most effective, participants successfully circumvented the obstacle in the majority of trials under auditory or SSD guidance. Using audition, blind non-echolocators navigated more effectively than blindfolded sighted individuals with fewer collisions, lower movement times, fewer velocity corrections and greater obstacle detection ranges. The blind expert echolocator displayed performance similar to or better than that for the other groups using audition, but was comparable to that for the other groups using the SSD. The generally better performance of blind than of sighted participants is consistent with the perceptual enhancement hypothesis that individuals with severe visual deficits develop improved auditory abilities to compensate for visual loss, here shown by faster, more fluid, and more accurate navigation around obstacles using sound.