Multi-Section Sensing and Vibrotactile Perception for Walking Guide of Visually Impaired Person

Auditory and tactile frequency mapping for visual distance perception: A step forward in sensory substitution and augmentation

Vision is crucial for daily tasks and interacting with the environment, but visual impairment can hinder these activities. Many sensory substitution products and studies prioritize providing abundant and accurate information, yet often overlook the inherent relationship between different modalities, potentially preventing users from receiving information intuitively. This study investigated the representation of visual distance using auditory and vibrotactile frequency through a series of psychological cross-modal matching experiments. By establishing mapping functions between auditory/vibrotactile frequency and visual distance, we aim to facilitate the design of sensory substitution devices that take visual distance information (ranging from 1 m to 12 m) and convert it into non-visual information (auditory frequency within the range 47-2764 Hz or vibrotactile frequency within the range 10-99 Hz). Results show distinct patterns regarding the correlation between visual distance and frequency in both auditory (auditory frequency-to-visual distance) and vibrotactile (vibrotactile frequency-to-visual distance) domains. The prevailing trend (59%) was a monotonic negative correlation (i.e. higher frequencies are associated with shorter distances), while 24% of participants demonstrated a consistently positive correlation. Additionally, we compare this study with our previous investigations into the reverse cross-modal mapping of visual distance-to-auditory frequency and visual distance-to-vibrotactile frequency. We reveal common patterns between these two studies (negative and positive correlations), suggesting a bidirectional mapping between visual distance and frequency in both auditory and vibrotactile domains, and the potential for new sensory substitution devices for those with visual impairment by integrating underlying cross-modal mechanisms to enhance intuitive and natural human-machine interaction.

Survey and analysis of the current status of research in the field of outdoor navigation for the blind

PURPOSE

In this article, we comprehensively review the current situation and research on technology related to outdoor travel for blind and visually impaired people (BVIP), given the diverse types and incomplete functionality of navigation aids for the blind. This aims to provide a reference for related research in the fields of outdoor travel for BVIP and blind navigation.

MATERIALS AND METHODS

We compiled articles related to blind navigation, of which a total of 227 of them are included in the search criteria. One hundred and seventy-nine articles are selected from the initial set, from a technical point of view, to elaborate on five aspects of blind navigation: system equipment, data sources, guidance algorithms, optimization of related methods, and navigation maps.

RESULTS

The wearable form of assistive devices for the blind has the most research, followed by the handheld type of aids. The RGB data class based on vision sensor is the most common source of navigation environment information data. Object detection based on picture data is also particularly rich among navigation algorithms and associated methods, indicating that computer vision technology has become an important study content in the field of blind navigation. However, research on navigation maps is relatively less.

CONCLUSIONS

In the study and development of assistive equipment for BVIP, there will be an emphasis on prioritizing attributes, such as lightness, portability, and efficiency. In light of the upcoming driverless era, the research focus will be on the development of visual sensors and computer vision technologies that can aid in navigation for the blind.

Towards sensory substitution and augmentation: Mapping visual distance to audio and tactile frequency

Multimodal perception is the predominant means by which individuals experience and interact with the world. However, sensory dysfunction or loss can significantly impede this process. In such cases, cross-modality research offers valuable insight into how we can compensate for these sensory deficits through sensory substitution. Although sight and hearing are both used to estimate the distance to an object (e.g., by visual size and sound volume) and the perception of distance is an important element in navigation and guidance, it is not widely studied in cross-modal research. We investigate the relationship between audio and vibrotactile frequencies (in the ranges 47-2,764 Hz and 10-99 Hz, respectively) and distances uniformly distributed in the range 1-12 m. In our experiments participants mapped the distance (represented by an image of a model at that distance) to a frequency via adjusting a virtual tuning knob. The results revealed that the majority (more than 76%) of participants demonstrated a strong negative monotonic relationship between frequency and distance, across both vibrotactile (represented by a natural log function) and auditory domains (represented by an exponential function). However, a subgroup of participants showed the opposite positive linear relationship between frequency and distance. The strong cross-modal sensory correlation could contribute to the development of assistive robotic technologies and devices to augment human perception. This work provides the fundamental foundation for future assisted HRI applications where a mapping between distance and frequency is needed, for example for people with vision or hearing loss, drivers with loss of focus or response delay, doctors undertaking teleoperation surgery, and users in augmented reality (AR) or virtual reality (VR) environments.

Wearable Drone Controller: Machine Learning-Based Hand Gesture Recognition and Vibrotactile Feedback

We proposed a wearable drone controller with hand gesture recognition and vibrotactile feedback. The intended hand motions of the user are sensed by an inertial measurement unit (IMU) placed on the back of the hand, and the signals are analyzed and classified using machine learning models. The recognized hand gestures control the drone, and the obstacle information in the heading direction of the drone is fed back to the user by activating the vibration motor attached to the wrist. Simulation experiments for drone operation were performed, and the participants' subjective evaluations regarding the controller's convenience and effectiveness were investigated. Finally, experiments with a real drone were conducted and discussed to validate the proposed controller.

Tom Pouce III, an Electronic White Cane for Blind People: Ability to Detect Obstacles and Mobility Performances

We present a protocol for evaluating the efficiency of an electronic white cane for improving the mobility of blind people. The electronic cane used during the test is the Tom Pouce III, made of LIDAR sensors (light detection and ranging) with tactile feedback. The protocol comprises two parts. The first part, the "detection test", evaluates the efficiency of the sensors in the Tom Pouce III for detecting the obstacles found in everyday life (thin and large poles, apertures) under different environmental conditions (darkness, sun light, rain). The second part of the test, the "mobility test", compares the ability of blind participants to cross a 25 m path by avoiding obstacles with the simple white cane and the electronic cane. The 12 blind participants had between 2 and 20 years of experience of everyday usage of Tom Pouce devices. The results show a significant improvement in the capacity to avoid obstacles with the electronic cane relative to the simple white cane, and there was no speed difference. There was no correlation between the results and the years of experience of the users.

A Systematic Review of Urban Navigation Systems for Visually Impaired People

Blind and Visually impaired people (BVIP) face a range of practical difficulties when undertaking outdoor journeys as pedestrians. Over the past decade, a variety of assistive devices have been researched and developed to help BVIP navigate more safely and independently. In addition, research in overlapping domains are addressing the problem of automatic environment interpretation using computer vision and machine learning, particularly deep learning, approaches. Our aim in this article is to present a comprehensive review of research directly in, or relevant to, assistive outdoor navigation for BVIP. We breakdown the navigation area into a series of navigation phases and tasks. We then use this structure for our systematic review of research, analysing articles, methods, datasets and current limitations by task. We also provide an overview of commercial and non-commercial navigation applications targeted at BVIP. Our review contributes to the body of knowledge by providing a comprehensive, structured analysis of work in the domain, including the state of the art, and guidance on future directions. It will support both researchers and other stakeholders in the domain to establish an informed view of research progress.

When Ultrasonic Sensors and Computer Vision Join Forces for Efficient Obstacle Detection and Recognition

In the most recent report published by the World Health Organization concerning people with visual disabilities it is highlighted that by the year 2020, worldwide, the number of completely blind people will reach 75 million, while the number of visually impaired (VI) people will rise to 250 million. Within this context, the development of dedicated electronic travel aid (ETA) systems, able to increase the safe displacement of VI people in indoor/outdoor spaces, while providing additional cognition of the environment becomes of outmost importance. This paper introduces a novel wearable assistive device designed to facilitate the autonomous navigation of blind and VI people in highly dynamic urban scenes. The system exploits two independent sources of information: ultrasonic sensors and the video camera embedded in a regular smartphone. The underlying methodology exploits computer vision and machine learning techniques and makes it possible to identify accurately both static and highly dynamic objects existent in a scene, regardless on their location, size or shape. In addition, the proposed system is able to acquire information about the environment, semantically interpret it and alert users about possible dangerous situations through acoustic feedback. To determine the performance of the proposed methodology we have performed an extensive objective and subjective experimental evaluation with the help of 21 VI subjects from two blind associations. The users pointed out that our prototype is highly helpful in increasing the mobility, while being friendly and easy to learn.