Improving Computer Vision-Based Indoor Wayfinding for Blind Persons with Context Information

User Interface Preferences in the Design of a Camera-Based Navigation and Wayfinding Aid

Introduction

Development of a sensing device that can provide a sufficient perceptual substrate for persons with visual impairments to orient themselves and travel confidently has been a persistent rehabilitation technology goal, with the user interface posing a significant challenge. In the study presented here, we enlist the advice and ideas of individuals who are blind with respect to this challenge, for an envisioned camera-based aid to navigation and wayfinding.

Methods

We administered a short questionnaire about user preferences and needs for such a device to a sample of 10 well-educated, employed (or retired) visually impaired participants with light perception or less, who were familiar and comfortable with assistive technology. Generally, the items were rankings of relative priority.

Results

Participants preferred speech as a communications medium for navigating the environment; preferred controlling the auditory display by querying the system rather than interacting via a menu or receiving a stream of continuous speech; and preferred providing input to the system through a keypad rather than through a voice recognition system. Architectural features such as doors and stairs were ranked the most important environmental objects to be located with such a device (over furniture, persons, personal items, and even text signs).

Discussion

Our sample reported a desire for devices that can guide them to architectural features of their environment. They appear to prefer device interfaces that give them control, and would rather query a system than interact with a menu. They prefer unobtrusive input on a device via keypad rather than through voice recognition.

Implications for practitioners

Designers of camera-based navigation devices may wish to consider the preferences of our sample by incorporating a query-based interface with simple keypad input and speech output, and to include in their object recognition efforts the goal of identifying architectural features that are significant to users who are blind in navigation.

Toward a Computer Vision-based Wayfinding Aid for Blind Persons to Access Unfamiliar Indoor Environments

Independent travel is a well known challenge for blind and visually impaired persons. In this paper, we propose a proof-of-concept computer vision-based wayfinding aid for blind people to independently access unfamiliar indoor environments. In order to find different rooms (e.g. an office, a lab, or a bathroom) and other building amenities (e.g. an exit or an elevator), we incorporate object detection with text recognition. First we develop a robust and efficient algorithm to detect doors, elevators, and cabinets based on their general geometric shape, by combining edges and corners. The algorithm is general enough to handle large intra-class variations of objects with different appearances among different indoor environments, as well as small inter-class differences between different objects such as doors and door-like cabinets. Next, in order to distinguish intra-class objects (e.g. an office door from a bathroom door), we extract and recognize text information associated with the detected objects. For text recognition, we first extract text regions from signs with multiple colors and possibly complex backgrounds, and then apply character localization and topological analysis to filter out background interference. The extracted text is recognized using off-the-shelf optical character recognition (OCR) software products. The object type, orientation, location, and text information are presented to the blind traveler as speech.

Robust and Effective Component-based Banknote Recognition for the Blind

We develop a novel camera-based computer vision technology to automatically recognize banknotes for assisting visually impaired people. Our banknote recognition system is robust and effective with the following features: 1) high accuracy: high true recognition rate and low false recognition rate, 2) robustness: handles a variety of currency designs and bills in various conditions, 3) high efficiency: recognizes banknotes quickly, and 4) ease of use: helps blind users to aim the target for image capture. To make the system robust to a variety of conditions including occlusion, rotation, scaling, cluttered background, illumination change, viewpoint variation, and worn or wrinkled bills, we propose a component-based framework by using Speeded Up Robust Features (SURF). Furthermore, we employ the spatial relationship of matched SURF features to detect if there is a bill in the camera view. This process largely alleviates false recognition and can guide the user to correctly aim at the bill to be recognized. The robustness and generalizability of the proposed system is evaluated on a dataset including both positive images (with U.S. banknotes) and negative images (no U.S. banknotes) collected under a variety of conditions. The proposed algorithm, achieves 100% true recognition rate and 0% false recognition rate. Our banknote recognition system is also tested by blind users.