Comparison of computer interface devices for persons with severe physical disabilities

Development of inductive sensors for a robotic interface based on noninvasive tongue control

Tongue based robotic interfaces have shown the potential to control assistive robotic devices developed for individuals with severe disabilities due to spinal cord injury. However, current tongue-robotic interfaces require invasive methods such as piercing to attach an activation unit (AU) to the tongue. A noninvasive tongue interface concept, which used a frame integrated AU instead of a tongue attached AU, was previously proposed. However, there is a need for the development of compact one-piece sensor printed circuit boards (PCBs) to enable activation of all inductive sensors. In this study, we developed and tested four designs of compact one-piece sensor PCBs incorporating inductive sensors for the design of a noninvasive tongue-robotic interface. We measured electrical parameters of the developed sensors to detect activation and compared them with a sensor of the current version of the inductive tongue-computer interface (ITCI) by moving AUs with different contact surfaces at the surface of the sensors. Results showed that, the newly developed inductive sensors had higher and wider activation than the sensor of ITCI and the AU with a flat contact surface had 3.5 - 4 times higher activation than the AU with a spherical contact surface. A higher sensor activation can result in a higher signal to noise ratio and thus a higher AU tracking resolution.

Design and Evaluation of a Hands-Free Video Game Controller for Individuals With Motor Impairments

Over the past few decades, video gaming has evolved at a tremendous rate although game input methods have been slower to change. Game input methods continue to rely on two-handed control of the joystick and D-pad or the keyboard and mouse for simultaneously controlling player movement and camera actions. Bi-manual input poses a significant play impediment to those with severe motor impairments. In this work, we propose and evaluate a hands-free game input control method that uses real-time facial expression recognition. Through our novel input method, our goal is to enable and empower individuals with neurological and neuromuscular diseases, who may lack hand muscle control, to be able to independently play video games. To evaluate the usability and acceptance of our system, we conducted a remote user study with eight severely motor-impaired individuals. Our results indicate high user satisfaction and greater preference for our input system with participants rating the input system as easy to learn. With this work, we aim to highlight that facial expression recognition can be a valuable input method.

Early classification of motor tasks using dynamic functional connectivity graphs from EEG

OBJECTIVE

Classification of electroencephalography (EEG) signals with high accuracy using short recording intervals has been a challenging problem in developing brain computer interfaces (BCIs). This paper presents a novel feature extraction method for EEG recordings to tackle this problem.

APPROACH

The proposed approach is based on the concept that the brain functions in a dynamic manner, and utilizes dynamic functional connectivity graphs. The EEG data is first segmented into intervals during which functional networks sustain their connectivity. Functional connectivity networks for each identified segment are then localized, and graphs are constructed, which will be used as features. To take advantage of the dynamic nature of the generated graphs, a Long Short Term Memory (LSTM) classifier is employed for classification.

MAIN RESULTS

Features extracted from various durations of post-stimulus EEG data associated with motor execution and imagery tasks are used to test the performance of the classifier. Results show an average accuracy of 85.32% using features extracted from only 500 ms of the post-stimulus data.

SIGNIFICANCE

Our results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms). This duration is considerably shorter than what has been reported before. These results will have significant implications for improving the effectiveness and the speed of BCIs, particularly for those used in assistive technologies.

Cursor Click Modality in an Accelerometer-Based Computer Access Device

The purpose of this study is to investigate the effects of different cursor click modalities in an alternative computer access device using accelerometry from head tilt to control cursor movement. Eighteen healthy adults performed a target acquisition task using the device with five different cursor click modalities, while maintaining cursor movement control via accelerometry. Three dwell-based click modalities with dwell times of 0.5 s, 1.0 s, and 1.5 s were tested. Two surface electromyography-based click modalities - with the sensor placed next to the eye for a blink and above the eyebrow for a brow raise - were tested. Performance was evaluated using metrics of target selection accuracy, path efficiency, target selection time, and user effort. Surface electromyography-based click modalities were as fast as the shortest dwell time and as accurate as the longest dwell time, and also minimized user effort. Three of the four performance metrics were not affected by sensor location. Future studies will investigate if these results are similar in individuals with neuromuscular disorders.

A Stand-Alone Intraoral Tongue-Controlled Computer Interface for People With Tetraplegia

The intraoral Tongue Drive System (iTDS) is an embedded wireless tongue-operated assistive technology developed for people with tetraplegia to provide them a higher level of independence in performing daily living tasks, such as accessing computers, smartphones, and driving wheelchairs. The iTDS was built as an arch-shaped dental retainer hermetically sealed and placed in the buccal shelf area of the mouth, completely hidden from sight. To provide high level of comfort, the iTDS is customized based on the users' oral anatomy to stably fix onto the lower teeth. We have presented a standalone version of the iTDS, capable of recognizing tongue gestures/commands by processing raw magnetic sensor data with a built-in pattern recognition algorithm in real time. The iTDS then sends the commands out in 10-b packets through a custom-designed high-gain intraoral antenna at 2.4 GHz to an external receiver. To evaluate the standalone iTDS performance, four subjects performed a computer access task by issuing random tongue commands over five sessions. Subjects completed 99.2% of the commands, and achieved an information transfer rate of 150.1 b/min. Moreover, a new typing method, designed specifically for the iTDS, resulted in typing at a rate of 3.76 words/min and error rate of 2.23%.

Speaking Ability while Using an Inductive Tongue-Computer Interface for Individuals with Tetraplegia: Talking and Driving a Powered Wheelchair - a Case Study

This paper assesses the ability of speaking while using an inductive tongue-computer interface. Lately, tongue- computer interfaces have been proposed for computer/robotic interfacing for individuals with tetraplegia. To be useful in home settings these interfaces should be aesthetic and interfere as little as possible with the limited preserved functionality of individuals with tetraplegia. As tongue interfaces from an aesthetical point of view are preferred to be entirely intra-oral it is relevant to address their effect on speech. Here we show that reading more than 566 words while using an inductive tongue-computer interface results in a maximum sensor activation time of less than 0.6 s, which means that false activations can be avoided by a sensor dwell time of 0.6 s. Furthermore, we show that it is possible to speak while controlling a powered wheelchair with the inductive tongue computer interface.

A Gaze-Based Virtual Keyboard Using a Mouth Switch for Command Selection

Portable eye-trackers provide an efficient way to access the point of gaze from a user on a computer screen. Thanks to eyetracking, gaze-based virtual keyboard can be developed by taking into account constraints related to the gaze detection accuracy. In this paper, we propose a new gaze-based virtual keyboard where all the letters can be accessed directly through a single command. In addition, we propose a USB mouth switch that is directly connected through a computer mouse, with the mouse switch replacing the left click button. This approach is considered to tackle the Midas touch problem with eye-tracking for people who are severely disabled. The performance is evaluated on 10 participants by comparing the following three conditions: gaze detection with mouth switch, gaze detection with dwell time by considering the distance to the closest command, and the gaze detection within the surface of the command box. Finally, a workload using NASA-TLX test was conducted on the different conditions. The results revealed that the proposed approach with the mouth switch provides a better performance in terms of typing speed (36.6 ± 8.4 letters/minute) compared to the other conditions, and a high acceptance as an input device.

A Low-Power Wearable Stand-Alone Tongue Drive System for People With Severe Disabilities

This paper presents a low-power stand-alone tongue drive system (sTDS) used for individuals with severe disabilities to potentially control their environment such as computer, smartphone, and wheelchair using their voluntary tongue movements. A low-power local processor is proposed, which can perform signal processing to convert raw magnetic sensor signals to user-defined commands, on the sTDS wearable headset, rather than sending all raw data out to a PC or smartphone. The proposed sTDS significantly reduces the transmitter power consumption and subsequently increases the battery life. Assuming the sTDS user issues one command every 20 ms, the proposed local processor reduces the data volume that needs to be wirelessly transmitted by a factor of 64, from 9.6 to 0.15 kb/s. The proposed processor consists of three main blocks: serial peripheral interface bus for receiving raw data from magnetic sensors, external magnetic interference attenuation to attenuate external magnetic field from the raw magnetic signal, and a machine learning classifier for command detection. A proof-of-concept prototype sTDS has been implemented with a low-power IGLOO-nano field programmable gate array (FPGA), bluetooth low energy, battery and magnetic sensors on a headset, and tested. At clock frequency of 20 MHz, the processor takes 6.6 s and consumes 27 nJ for detecting a command with a detection accuracy of 96.9%. To further reduce power consumption, an application-specified integrated circuit processor for the sTDS is implemented at the postlayout level in 65-nm CMOS technology with 1-V power supply, and it consumes 0.43 mW, which is 10 lower than FPGA power consumption and occupies an area of only 0.016 mm.

Simultaneous Multimodal PC Access for People With Disabilities by Integrating Head Tracking, Speech Recognition, and Tongue Motion

Multimodal Tongue Drive System (mTDS) is a highly integrated wireless assistive technology (AT) in the form of a lightweight wearable headset that utilizes three remaining key control and communication abilities in people with severe physical disabilities, such as tetraplegia, to provide them with effective access to computers: 1) tongue motion for discrete/switch-based control (e.g., clicking), 2) head tracking for proportional control (e.g., mouse pointer movements), and 3) speech recognition for typing, all available simultaneously. The mTDS architecture is presented here with new sensor signal processing algorithm for head tracking. To evaluate the device performance, it was compared against keyboard-and-mouse (KnM) combination, the gold standard in computer input methods, by 15 able-bodied participants, who used both mTDS and KnM to generate and sent an email with randomly selected content, under a 5-minute time constraint. In four repetitions, in the last trial, it took participants only 1.8 times longer to complete the email task, on average, using the mTDS versus KnM at 82.4% typing accuracy. Mean task completion time and typing accuracy improved 24.6% and 18.8% from first to fourth trial using mTDS. Multimodal simultaneous discrete and proportional control input options of mTDS, plus rapid typing, is expected to provide more effective computer access to people with severe physical disabilities.

Changes in motor performance and mental workload during practice of reaching movements: a team dynamics perspective

Few investigations have examined mental workload during motor practice or learning in a context of team dynamics. This study examines the underlying cognitive-motor processes of motor practice by assessing the changes in motor performance and mental workload during practice of reaching movements. Individuals moved a robotic arm to reach targets as fast and as straight as possible while satisfying the task requirement of avoiding a collision between the end-effector and the workspace limits. Individuals practiced the task either alone (HA group) or with a synthetic teammate (HRT group), which regulated the effector velocity to help satisfy the task requirements. The findings revealed that the performance of both groups improved similarly throughout practice. However, when compared to the individuals of the HA group, those in the HRT group (1) had a lower risk of collisions, (2) exhibited higher performance consistency, and (3) revealed a higher level of mental workload while generally perceiving the robotic teammate as interfering with their performance. As the synthetic teammate changed the effector velocity in specific regions near the workspace boundaries, individuals may have been constrained to learn a piecewise visuomotor map. This piecewise map made the task more challenging, which increased mental workload and perception of the synthetic teammate as a burden. The examination of both motor performance and mental workload revealed a combination of both adaptive and maladaptive team dynamics. This work is a first step to examine the human cognitive-motor processes underlying motor practice in a context of team dynamics and contributes to inform human-robot applications.

Effect of diagnosis, body site and experience on text entry rate of individuals with physical disabilities: a systematic review

OBJECTIVE

This study systematically reviewed the research on computer text entry by people with physical disabilities, and conducted a quantitative synthesis of text entry rates associated with individuals' diagnosis, body site used with the interface and their level of experience.

METHOD

We searched 10 databases and included studies in which: typing speed was reported; the access interface was available for public use; and individuals with physical impairments were in the study population. For quantitative synthesis, we used only the text entry rates (TER) reported for individuals with physical impairments; studies also had to report the sample size, and the average and standard deviation for the text entry rates.

RESULTS

Thirty-nine studies involving 248 subjects met the criteria for quantitative synthesis. Cerebral palsy was associated with significantly slower TER, at 5.5 wpm, than muscular dystrophy (12.5 wpm), spina bifida (10.4 wpm), SCI high cervical (10.1 wpm) and SCI low cervical (13.3 wpm). Among the 19 body sites represented, the Fingers bilateral category had the highest average, at 17.72 wpm. Head (2.92 wpm) and Hand (non-typing) (3.95 wpm) were each associated with significantly slower TER than Hands unspecified, Fingers bilateral, Hand with control enhancer, Voice and Mouth. The three levels of experience examined, LowPlus, Medium and High, provided very similar TER.

CONCLUSION

This study contributes external evidence for service providers who provide computer access interventions for individuals with disabilities. The analyzed text entry performances serve as benchmarks to be used as guidelines during interface selection and training. Implications for Rehabilitation The median text entry rate (TER) across the literature for individuals with physical disabilities is 7.0 wpm. TER is affected by a user's diagnosis and the body site used for typing. These TER data can serve as diagnostic norms and benchmarks to be used as guidelines during interface selection and training. We recommend that practitioners measure text entry rate with clients, to place their performance in the context of these results and provide a baseline against which to measure effectiveness of an intervention.

Wearable wireless User Interface Cursor-Controller (UIC-C)

Controlling a computer or a smartphone's cursor allows the user to access a world full of information. For millions of people with limited upper extremities motor function, controlling the cursor becomes profoundly difficult. Our team has developed the User Interface Cursor-Controller (UIC-C) to assist the impaired individuals in regaining control over the cursor. The UIC-C is a hands-free device that utilizes the tongue muscle to control the cursor movements. The entire device is housed inside a subject specific retainer. The user maneuvers the cursor by manipulating a joystick imbedded inside the retainer via their tongue. The joystick movement commands are sent to an electronic device via a Bluetooth connection. The device is readily recognizable as a cursor controller by any Bluetooth enabled electronic device. The device testing results have shown that the time it takes the user to control the cursor accurately via the UIC-C is about three times longer than a standard computer mouse controlled via the hand. The device does not require any permanent modifications to the body; therefore, it could be used during the period of acute rehabilitation of the hands. With the development of modern smart homes, and enhancement electronics controlled by the computer, UIC-C could be integrated into a system that enables individuals with permanent impairment, the ability to control the cursor. In conclusion, the UIC-C device is designed with the goal of allowing the user to accurately control a cursor during the periods of either acute or permanent upper extremities impairment.

Error-Free Text Typing Performance of an Inductive Intra-Oral Tongue Computer Interface for Severely Disabled Individuals

For severely paralyzed individuals, alternative computer interfaces are becoming increasingly essential for everyday life as social and vocational activities are facilitated by information technology and as the environment becomes more automatic and remotely controllable. Tongue computer interfaces have proven to be desirable by the users partly due to their high degree of aesthetic acceptability, but so far the mature systems have shown a relatively low error-free text typing efficiency. This paper evaluated the intra-oral inductive tongue computer interface (ITCI) in its intended use: Error-free text typing in a generally available text editing system, Word. Individuals with tetraplegia and able bodied individuals used the ITCI for typing using a MATLAB interface and for Word typing for 4 to 5 experimental days, and the results showed an average error-free text typing rate in Word of 11.6 correct characters/min across all participants and of 15.5 correct characters/min for participants familiar with tongue piercings. Improvements in typing rates between the sessions suggest that typing ratescan be improved further through long-term use of the ITCI.

Text entry rate of access interfaces used by people with physical disabilities: A systematic review

This study systematically reviewed the research on assistive technology (AT) access interfaces used for text entry, and conducted a quantitative synthesis of text entry rates (TER) associated with common interfaces. We searched 10 databases and included studies in which: typing speed was reported in words per minute (WPM) or equivalent; the access interface was available for public use; and individuals with physical impairments were in the study population. For quantitative synthesis, we used only the TER reported for individuals with physical impairments. Studies also had to report the sample size, and the average and standard deviation for the TER data. Thirty-nine studies met the criteria for quantitative synthesis. Studies involved seven interface types: standard keyboard typing, on-screen keyboard (OSK) with cursor selection, OSK with scanning selection, automatic speech recognition (ASR), Morse code, brain-computer interface (BCI), and other. ASR, standard keyboard, cursor OSK, and scanning OSK had at least four studies and 30 subjects, with TER averaging 15.4, 12.5, 4.2, and 1.7 WPM, respectively. When combined with measurements of a particular client's text entry performance, the TER from this review can be used within an evidence-based decision-making process for selecting control interfaces.

Children's Computer Access: Analysis of the Visual-Motor Demands of Software Designed for Children

As technology access becomes an increasingly important activity of daily living, debate persists as to the manner in which computers are best able to assist children in educational and recreational settings. In particular, information regarding the suitability of commonly used child-computer interfaces is required. Occupational therapists are uniquely qualified to assess and problem-solve computer access issues but, to date, have limited the application of this skill to the areas of rehabilitation and disability.

This paper describes a process where the traditional occupational therapy tool of task analysis was used to identify the visual-motor demands of the children's computer game packages that use the mouse for operation. The first author examined 12 game packages comprising 45 separate games and applications and recorded information on the types of movement, task design features and recommended ages for all games. The results of the analysis are discussed in terms of the degree of visual-motor development that is required of children accessing these games and the implications for game designers producing developmentally sensitive software.

Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons

PURPOSE

Individuals with tetraplegia depend on alternative interfaces in order to control computers and other electronic equipment. Current interfaces are often limited in the number of available control commands, and may compromise the social identity of an individual due to their undesirable appearance. The purpose of this study was to implement an alternative computer interface, which was fully embedded into the oral cavity and which provided multiple control commands.

METHODS

The development of a wireless, intraoral, inductive tongue computer was described. The interface encompassed a 10-key keypad area and a mouse pad area. This system was embedded wirelessly into the oral cavity of the user. The functionality of the system was demonstrated in two tetraplegic individuals and two able-bodied individuals Results: The system was invisible during use and allowed the user to type on a computer using either the keypad area or the mouse pad. The maximal typing rate was 1.8 s for repetitively typing a correct character with the keypad area and 1.4 s for repetitively typing a correct character with the mouse pad area.

CONCLUSION

The results suggest that this inductive tongue computer interface provides an esthetically acceptable and functionally efficient environmental control for a severely disabled user. Implications for Rehabilitation New Design, Implementation and detection methods for intra oral assistive devices. Demonstration of wireless, powering and encapsulation techniques suitable for intra oral embedment of assistive devices. Demonstration of the functionality of a rechargeable and fully embedded intra oral tongue controlled computer input device.

Performance of Able-Bodied Subjects on a Text-Typing Task Using a Head-Operated Device and Expanded Membrane Cursor Keys

Children and adults with disabilities frequently rely on computers to complete written tasks. Those with significant motor limitations typically use alternative computer-input devices since the traditional keyboard and mouse are insufficient to accommodate their abilities. For persons unable to isolate their fingers, input devices controlled by movements of the head or whole hand or arm movements may be among the options considered. This study investigated the performance of a head-operated device and expanded membrane cursor keys for text entry. Data from 24 young adults indicated the head-operated device performed significantly faster given reduced cognitive demands for device operation, increased stimulus–response compatibility, and simplicity of movement. Use of the expanded membrane cursor keys resulted in significantly lower error rates. No significant differences in comfort or ease of use were reported for the two devices. The relative performance of device options for users sharing similar motor challenges provides rehabilitation specialists with important clinical information.

Case study: Head orientation and neck electromyography for cursor control in persons with high cervical tetraplegia

We evaluated the ability of an individual with a high cervical spinal cord injury (SCI) to control a cursor on a computer screen using two different user interfaces: (1) head movements measured via a head-mounted orientation sensor and (2) electromyography (EMG) signals from four head and neck muscles acquired using a 4-channel implanted upper-limb neuroprosthesis that had been deployed in an earlier study. The subject moved the cursor to a set of targets on the screen in a two-dimensional, center-out, target-acquisition task, and his performance was evaluated with a variety of performance measures to assess both position and velocity control accuracy. The subject's performance with both command sources was also compared with the performance of a group of nondisabled subjects. Head orientation provided more accurate performance but was less responsive than EMG. Both command sources showed some directionally dependent performance, with movement to diagonally located targets being performed by a series of sequential motions rather than via straight paths. Overall, the SCI subject's performance with each command source was similar to that reported for a nondisabled population using the same interfaces and performing the same task.

The effect of word prediction settings (frequency of use) on text input speed in persons with cervical spinal cord injury: a prospective study

PURPOSE

To determine whether activation of the frequency of use and automatic learning parameters of word prediction software has an impact on text input speed.

METHODS

Forty-five participants with cervical spinal cord injury between C4 and C8 Asia A or B accepted to participate to this study. Participants were separated in two groups: a high lesion group for participants with lesion level is at or above C5 Asia AIS A or B and a low lesion group for participants with lesion is between C6 and C8 Asia AIS A or B. A single evaluation session was carried out for each participant. Text input speed was evaluated during three copying tasks: • without word prediction software (WITHOUT condition) • with automatic learning of words and frequency of use deactivated (NOT_ACTIV condition) • with automatic learning of words and frequency of use activated (ACTIV condition) Results: Text input speed was significantly higher in the WITHOUT than the NOT_ACTIV (p< 0.001) or ACTIV conditions (p = 0.02) for participants with low lesions. Text input speed was significantly higher in the ACTIV than in the NOT_ACTIV (p = 0.002) or WITHOUT (p < 0.001) conditions for participants with high lesions.

CONCLUSIONS

Use of word prediction software with the activation of frequency of use and automatic learning increased text input speed in participants with high-level tetraplegia. For participants with low-level tetraplegia, the use of word prediction software with frequency of use and automatic learning activated only decreased the number of errors. Implications in rehabilitation   Access to technology can be difficult for persons with disabilities such as cervical spinal cord injury (SCI). Several methods have been developed to increase text input speed such as word prediction software.This study show that parameter of word prediction software (frequency of use) affected text input speed in persons with cervical SCI and differed according to the level of the lesion. • For persons with high-level lesion, our results suggest that this parameter must be activated so that text input speed is increased. • For persons with low lesion group, this parameter must be activated so that the numbers of errors are decreased. • In all cases, the activation of the parameter of frequency of use is essential in order to improve the efficiency of the word prediction software. • Health-related professionals should use these results in their clinical practice for better results and therefore better patients 'satisfaction.

Text input speed in persons with cervical spinal cord injury

STUDY DESIGN

This is a prospective clinical study.

OBJECTIVES

The objectives of this study were to determine text input speed (TIS) in persons with cervical spinal cord injury (SCI) and to study the influence of personal characteristics and type of computer access device on TIS.

SETTING

This study was conducted in the Rehabilitation Department, Garches, France.

METHODS

People with cervical SCI were included if their level of injury was between C4 and C8 Asia A or B, and if they were computer users. In addition, able-bodied people were recruited from the hospital staff. Each participant underwent a single evaluation using their usual computer access devices. TIS was evaluated during a 10- min copying task. The relationship between the characteristics of participants with cervical SCI, type of computer access device and TIS were analyzed using a Scheirer-Ray-Hare test (nonparametric test similar to a two-way analysis of variance).

RESULTS

Thirty-five participants with cervical SCI and 21 able-bodied people were included. Median TIS of participants with cervical SCI was 11 (6; 14) words per minute (w.p.m.) and of able-bodied participants was 19 (14; 24) w.p.m. (P=0.001). Median TIS of participants with lesions at or above C5 was 12 (4; 13) w.p.m. and of those with lesions below C5 was 10 (9; 18) w.p.m. (P=0.38) [corrected]. The Scheirer-Ray-Hare test showed that only the type of computer access device significantly influenced TIS. Surprisingly, none of the person's characteristics, including the level of cervical lesion, affected TIS.

CONCLUSION

This is the first study to analyze TIS in a group of participants with cervical SCI. The results showed that only the type of computer access device influenced TIS.

Improving Target Acquisition for Computer Users With Athetosis

Prior work has highlighted the challenges faced by people with athetosis when trying to acquire on-screen targets using a mouse or trackball. The difficulty of positioning the mouse cursor within a confined area has been identified as a challenging task. We have developed a target acquisition assistance algorithm that features transition assistance via directional gain variation based on target prediction, settling assistance via gain reduction in the vicinity of a predicted target, and expansion of the predicted target as the cursor approaches it. We evaluated the algorithm on improving target acquisition efficiency among seven participants with athetoid cerebral palsy. Our results showed that the algorithm significantly reduced the overall movement time by about 20%. Considering the target acquisition occurs countless times in the course of regular computer use, the accumulative effect of such improvements can be significant for improving the efficiency of computer interaction among people with athetosis.

Facial Position and Expression-Based Human-Computer Interface for Persons With Tetraplegia

A human-computer interface (namely Facial position and expression Mouse system, FM) for the persons with tetraplegia based on a monocular infrared depth camera is presented in this paper. The nose position along with the mouth status (close/open) is detected by the proposed algorithm to control and navigate the cursor as computer user input. The algorithm is based on an improved Randomized Decision Tree, which is capable of detecting the facial information efficiently and accurately. A more comfortable user experience is achieved by mapping the nose motion to the cursor motion via a nonlinear function. The infrared depth camera enables the system to be independent of illumination and color changes both from the background and on human face, which is a critical advantage over RGB camera-based options. Extensive experimental results show that the proposed system outperforms existing assistive technologies in terms of quantitative and qualitative assessments.

Wireless Communication of Intraoral Devices and Its Optimal Frequency Selection

This paper explores communication methods and frequencies for wireless intraoral electronic devices, by using an intraoral tongue drive system (iTDS) as a practical example. Because intraoral devices do not meet the operating conditions of the body channel communication, we chose radio frequency communication. We evaluated and compared three frequencies in industrial, scientific, and medical bands (27 MHz, 433.9 MHz, and 2.48 GHz) in terms of their data link performance based on path loss and radiation patterns over horizontal and vertical planes. To do so, we dynamically minimize the impedance mismatch caused by the varying oral environment by applying the adaptive impedance matching technique to 433.9 MHz and 2.48 GHz bands. Experimental results showed that 27 MHz has the smallest path loss in the near-field up to 39 cm separation between transmitter and receiver antennas. However, 433.9 MHz shows the best performance beyond 39 cm and offers a maximum operating distance of 123 cm with 0 dBm transmitter output power. These distances were obtained by a bit error rate test and verified by a link budget analysis and full functionality test of the iTDS with computer access.

The tongue enables computer and wheelchair control for people with spinal cord injury

The Tongue Drive System (TDS) is a wireless and wearable assistive technology, designed to allow individuals with severe motor impairments such as tetraplegia to access their environment using voluntary tongue motion. Previous TDS trials used a magnetic tracer temporarily attached to the top surface of the tongue with tissue adhesive. We investigated TDS efficacy for controlling a computer and driving a powered wheelchair in two groups of able-bodied subjects and a group of volunteers with spinal cord injury (SCI) at C6 or above. All participants received a magnetic tongue barbell and used the TDS for five to six consecutive sessions. The performance of the group was compared for TDS versus keypad and TDS versus a sip-and-puff device (SnP) using accepted measures of speed and accuracy. All performance measures improved over the course of the trial. The gap between keypad and TDS performance narrowed for able-bodied subjects. Despite participants with SCI already having familiarity with the SnP, their performance measures were up to three times better with the TDS than with the SnP and continued to improve. TDS flexibility and the inherent characteristics of the human tongue enabled individuals with high-level motor impairments to access computers and drive wheelchairs at speeds that were faster than traditional assistive technologies but with comparable accuracy.

Medical tongue piercing - development and evaluation of a surgical protocol and the perception of procedural discomfort of the participants

BACKGROUND

A system providing disabled persons with control of various assistive devices with the tongue has been developed at Aalborg University in Denmark. The system requires an activation unit attached to the tongue with a small piercing. The aim of this study was to establish and evaluate a safe and tolerable procedure for medical tongue piercing and to evaluate the expected and perceived procedural discomfort.

METHODS

Four tetraplegic subjects volunteered for the study. A surgical protocol for a safe insertion of a tongue barbell piercing was presented using sterilized instruments and piercing parts. Moreover, post-procedural observations of participant complications such as bleeding, edema, and infection were recorded. Finally, procedural discomforts were monitored by VAS scores of pain, changes in taste and speech as well as problems related to hitting the teeth.

RESULTS

The piercings were all successfully inserted in less than 5 min and the pain level was moderate compared with oral injections. No bleeding, infection, embedding of the piercing, or tooth/gingival injuries were encountered; a moderate edema was found in one case without affecting the speech. In two cases the piercing rod later had to be replaced by a shorter rod, because participants complained that the rod hit their teeth. The replacements prevented further problems. Moreover, loosening of balls was encountered, which could be prevented with the addition of dental glue. No cases of swallowing or aspiration of the piercing parts were recorded.

CONCLUSIONS

The procedure proved simple, fast, and safe for insertion of tongue piercings for tetraplegic subjects in a clinical setting. The procedure represented several precautions in order to avoid risks in these susceptible participants with possible co-morbidity. No serious complications were encountered, and the procedure was found tolerable to the participants. The procedure may be used in future studies with tongue piercings being a prerequisite for similar systems, and this may include insertion in an out-patient setting.

Tongue-Supported Human-Computer Interaction systems: a review

The tongue can substitute human sensory systems and has been used as a medium of input to help impaired patients communicate with the world. Innovative techniques have been employed to realize tongue movement, sense its position and exploit tongue dexterity, in order to achieve Tongue Supported Human Computer Interaction (TSHCI). This paper examines various approaches of using tongue dexterousness in TSHCI systems and introduces two infrared signal supported minimally-invasive TSHCI systems developed at Curtin University. Methods of sensing tongue movement and position are especially discussed and depending on the employed methods, TSHCI systems are categorized as either invasive or minimally-invasive. A set of system usability criteria is proposed to help build more effective TSHCI systems in future.

On the tip of the tongue: learning typing and pointing with an intra-oral computer interface

PURPOSE

To evaluate typing and pointing performance and improvement over time of four able-bodied participants using an intra-oral tongue-computer interface for computer control.

BACKGROUND

A physically disabled individual may lack the ability to efficiently control standard computer input devices. There have been several efforts to produce and evaluate interfaces that provide individuals with physical disabilities the possibility to control personal computers.

METHOD

Training with the intra-oral tongue-computer interface was performed by playing games over 18 sessions. Skill improvement was measured through typing and pointing exercises at the end of each training session.

RESULTS

Typing throughput improved from averages of 2.36 to 5.43 correct words per minute. Pointing throughput improved from averages of 0.47 to 0.85 bits/s. Target tracking performance, measured as relative time on target, improved from averages of 36% to 47%. Path following throughput improved from averages of 0.31 to 0.83 bits/s and decreased to 0.53 bits/s with more difficult tasks.

CONCLUSIONS

Learning curves support the notion that the tongue can rapidly learn novel motor tasks. Typing and pointing performance of the tongue-computer interface is comparable to performances of other proficient assistive devices, which makes the tongue a feasible input organ for computer control.

IMPLICATIONS FOR REHABILITATION

Intra-oral computer interfaces could provide individuals with severe upper-limb mobility impairments the opportunity to control computers and automatic equipment. Typing and pointing performance of the tongue-computer interface is comparable to performances of other proficient assistive devices, but does not cause fatigue easily and might be invisible to other people, which is highly prioritized by assistive device users. Combination of visual and auditory feedback is vital for a good performance of an intra-oral computer interface and helps to reduce involuntary or erroneous activations.

Effects of sensory feedback in intra-oral target selection tasks with the tongue

PURPOSE

To investigate the effects of visual and tactile intra-oral sensor-position feedback for target selection tasks with the tip of the tongue.

METHOD

Target selection tasks were performed using an inductive tongue-computer interface (ITCI). Visual feedback was established by highlighting the area on a visual display corresponding to the activated intra-oral target. Tactile feedback was established using a sensor-border matrix over the sensor plates of the ITCI, which provided sensor-position tactile queues via the user's tongue. Target selection tasks using an on-screen keyboard by controlling the mouse pointer with the ITCI's was also evaluated.

RESULTS

Mean target selection rates of 23, 5 and 15 activations per minute were obtained using visual, tactile and "none" feedback techniques in the 3rd training session. On-screen keyboard target selection tasks averaged 10 activations per minute in the 3rd training session. Involuntary activations while speaking or drinking were significantly reduced either through a sensor-matrix or dwell time for sensor activation.

CONCLUSIONS

These results provide key design considerations to further increase the typing efficiency of tongue-computer interfaces for individuals with upper-limb mobility impairments.

Quantitative and comparative assessment of learning in a tongue-operated computer input device--part II: navigation tasks

Tongue drive system (TDS) is a novel tongue-operated assistive technology (AT) for the mobility impaired, to empower them to access computers and drive powered wheelchairs (PWC) using their free voluntary tongue motion. We have evaluated the TDS performance in five sessions over 5-8 weeks to study the learning process in different tasks of computer access and PWC navigation on nine able-bodied subjects who already had tongue piercing and used our magnetic tongue studs throughout the trial. Computer access tasks included on-screen maze navigation and issuing random commands to measure the TDS information transfer rate. PWC navigation included driving through a ~50-m obstacle course using three control strategies. Some of the qualitative aspects of using the TDS were also evaluated based on the two Likert scale questionnaires, one of which was short (eight questions) and asked at the end of each session and the other one (46 questions) at the end of the trial. Included in this study was also a task to measure the tongue fatigue as a result of using the TDS continuously for a few hours. All performance measures showed significant improvement from the first to the second session as well as further gradual improvements throughout the rest of the sessions, suggesting a rapid learning process.

Command detection and classification in tongue drive assistive technology

Tongue Drive System (TDS) is a new assistive technology that enables individuals with severe disabilities such as those with spinal cord injury (SCI) to regain environmental control using their tongue motion. We have developed a new sensor signal processing (SSP) algorithm which uses four 3-axial magneto-resistive sensor outputs to accurately detect and classify between seven different user-control commands in stationary as well as mobile conditions. The new algorithm employs a two-stage classification method with a combination of 9 classifiers to discriminate between 4 commands on the left or right side of the oral cavity (one neutral command shared on both sides). Evaluation of the new SSP algorithm on five able-bodied subjects resulted in true positive rates in the range of 70-99% with corresponding false positive rates in the range of 5-7%, showing a notable improvement in the resulted true-false (TF) differences when compared to the previous algorithm.

Dual-task motor performance with a tongue-operated assistive technology compared with hand operations

Background

To provide an alternative motor modality for control, navigation, and communication in individuals suffering from impairment or disability in hand functions, a Tongue Drive System (TDS) has been developed that allows for real time tracking of tongue motion in an unobtrusive, wireless, and wearable device that utilizes the magnetic field generated by a miniature disk shaped magnetic tracer attached to the tip of the tongue. The purpose of the study was to compare the influence of a concurrent motor or cognitive task on various aspects of simple movement control between hand and tongue using the TDS technology.

Methods

Thirteen young able-bodied adults performed rapid and slow goal-directed movements of hand and tongue (with TDS) with and without a concurrent motor (hand or tongue) or cognitive (arithmetic and memory) task. Changes in reaction time, completion time, speed, correctness, accuracy, variability of displacement, and variability of time due to the addition of a concurrent task were compared between hand and tongue.

Results

The influence of an additional concurrent task on motor performance was similar between the hand and tongue for slow movement in controlling their displacement. In rapid movement with a concurrent motor task, most aspects of motor performance were degraded in hand, while tongue speed during rapid continuous task was maintained. With a concurrent cognitive task, most aspects of motor performance were degraded in tongue, while hand accuracy during the rapid discrete task and hand speed during the rapid continuous task were maintained.

Conclusion

Rapid goal-directed hand and tongue movements were more consistently susceptible to interference from concurrent motor and cognitive tasks, respectively, compared with the other movement.

Quantitative and comparative assessment of learning in a tongue-operated computer input device

Tongue drive system (TDS) is a wireless, wearable assistive technology that enables individuals with severe motor impairments to access computers, drive wheelchairs, and control their environments using tongue motion. In this paper, we have evaluated the TDS performance as a computer input device in four tasks, commonly known as horizontal, vertical, center-out, and multidirectional rapid tapping, based on Fitts' law and ISO9241-9 Standard. Nine able-bodied subjects, who already had tongue piercing, participated in this trial over five sessions during 5 weeks, allowing us to study the TDS learning process and its current limiting factors. Subjects wore tongue rings made of titanium in the form of a barbell with a small rare-earth magnetic tracer hermetically sealed inside the upper ball. Participants performed the same tasks with a mouse (only in the first session) as a reference as well as a standard keypad for benchmarking. Six performance measures were considered, including throughput, error rate, and reaction time, all of these improved significantly from the first to the last session, and some of these plateaued over the course of the experiment. The comparison between tongue-TDS versus index-finger-keypad provides valuable insights into tongue human factors, which can lead the way in improving the usability of the TDS and similar tongue-operated assistive technologies.

Use of microswitch technology and a keyboard emulator to support literacy performance of persons with extensive neuro-motor disabilities

OBJECTIVE

To assess the effectiveness and acceptability of microswitch technology and a keyboard emulator to enable three participants with extensive neuro-motor disabilities to write words.

METHOD

In Study I, two participants triggered an automatic scanning keyboard and selected/wrote letters via a small sliding movement of their hand(s) activating a touch/pressure panel (microswitch). In Study II, a third participant used the sliding movement and panel and a vocalization response with a voice-detecting microswitch. The sliding movement allowed her to light up the keyboard and select the letters and the vocalization to perform the scanning.

RESULTS

Participants showed a better performance (shorter writing time) or an equally effective but less tiring performance with the new microswitch technology and response(s). They also preferred using this technology, and social validation ratings favoured such technology over previous solutions.

CONCLUSION

The aforementioned technology may be useful to enable persons with extensive neuro-motor disabilities to write successfully.

Evaluation of the tongue drive system by individuals with high-level spinal cord injury

Tongue Drive System (TDS) is a tongue operated, unobtrusive, minimally invasive, wireless assistive technology (AT), which can enable people with severe disabilities to control different devices using their tongue motion. TDS can translate specific tongue movements into user-defined commands by detecting the position of a small permanent magnetic tracer attached to the users' tongue. We have built an external TDS (eTDS) prototype on a wireless headphone and interfaced it to a laptop and a commercial powered wheelchair (PWC). eTDS performance was evaluated by eight subjects with high level (C3 approximately C5) spinal cord injury (SCI) at the Shepherd Center in Atlanta, GA. Preliminary results show that all the subjects can successfully perform common tasks related to computer access, such as controlling a mouse cursor or playing a computer game, as well as complex wheelchair navigation tasks, such as driving through an obstacle course.

Fully integrated wireless inductive tongue computer interface for disabled people

This work describes a novel fully integrated inductive tongue computer interface for disabled people. The interface consists of an oral unit placed in the mouth, including inductive sensors, related electronics, a system for wireless transmission and a rechargeable battery. The system is activated using an activation unit placed on the tongue, and incorporates 18 inductive sensors, arranged in both a key area and a mouse-pad area. The system's functionality was demonstrated in a pilot experiment, where a typing rate of up to 70 characters/minute was obtained with an error rate of 3%. Future work will include tests with disabled subjects.

Evaluation of a wireless wearable tongue-computer interface by individuals with high-level spinal cord injuries

The tongue drive system (TDS) is an unobtrusive, minimally invasive, wearable and wireless tongue-computer interface (TCI), which can infer its users' intentions, represented in their volitional tongue movements, by detecting the position of a small permanent magnetic tracer attached to the users' tongues. Any specific tongue movements can be translated into user-defined commands and used to access and control various devices in the users' environments. The latest external TDS (eTDS) prototype is built on a wireless headphone and interfaced to a laptop PC and a powered wheelchair. Using customized sensor signal processing algorithms and graphical user interface, the eTDS performance was evaluated by 13 naive subjects with high-level spinal cord injuries (C2-C5) at the Shepherd Center in Atlanta, GA. Results of the human trial show that an average information transfer rate of 95 bits/min was achieved for computer access with 82% accuracy. This information transfer rate is about two times higher than the EEG-based BCIs that are tested on human subjects. It was also demonstrated that the subjects had immediate and full control over the powered wheelchair to the extent that they were able to perform complex wheelchair navigation tasks, such as driving through an obstacle course.

Development and Rasch analysis of the Assessment of Computer-Related Skills

The purpose of this study is to evaluate the internal scale validity, person response validity, and reliability of the newly developed Assessment of Computer-Related Skills (ACRS). Data from 32 healthy adult participants who performed two to three computer tasks were analysed to determine how well the participants fitted the many-faceted Rasch (MFR) model of the ACRS, as well as how well the ACRS skill items and tasks (a) fitted the MFR model of the ACRS, (b) matched the expectations for hierarchical ordering of their difficulties, and (c) differentiated persons into different levels of ability. Results indicated that with three skill items removed, the remaining 34 skill items, 8 computer tasks, and 30 participants demonstrated goodness-of-fit to the MFR model of the ACRS. The skills and tasks appeared to have logical hierarchical ordering and differentiated participants into at least three levels of ability. The findings affirmed the internal scale validity, person response validity and reliability of the ACRS for assessing persons' computer abilities. Future studies using a larger sample that includes individuals with disabilities and with difficulties with computer use are needed to further evaluate the validity and reliability of the ACRS.

Wireless control of powered wheelchairs with tongue motion using tongue drive assistive technology

Tongue Drive system (TDS) is a tongue-operated unobtrusive wireless assistive technology, which can potentially provide people with severe disabilities with effective computer access and environment control. It translates users' intentions into control commands by detecting and classifying their voluntary tongue motion utilizing a small permanent magnet, secured on the tongue, and an array of magnetic sensors mounted on a headset outside the mouth or an orthodontic brace inside. We have developed customized interface circuitry and implemented four control strategies to drive a powered wheelchair (PWC) using an external TDS prototype. The system has been evaluated by five able-bodied human subjects. The results showed that all subjects could easily operate the PWC using their tongue movements, and different control strategies worked better depending on the users' familiarity with the TDS.

Using unconstrained tongue motion as an alternative control mechanism for wheeled mobility

Tongue drive system (TDS) is a tongue-operated, minimally invasive, unobtrusive, noncontact, and wireless assistive technology that infers users' intentions by detecting and classifying their voluntary tongue motions, and translating them to user-defined commands. We have developed customized interface circuitry between an external TDS (eTDS) prototype and a commercial powered wheelchair (PWC) as well as three control strategies to evaluate the tongue motion as an alternative control input for wheeled mobility. We tested the eTDS performance in driving PWCs on 12 able-bodied human subjects, of which 11 were novice. The results showed that all subjects could complete navigation tasks by operating the PWC using their tongue motions. Despite little prior experience, the average time using the eTDS and the tongue was only approximately three times longer than using a joystick and the fingers. Navigation time was strongly dependant on the number of issued commands, which reduced by gaining experience. Particularly, the unintended issued commands (the Midas touch problem) were rare, demonstrating the effectiveness of the tongue tracking and external magnetic field cancellation algorithms as well as the safety of the TDS for wheeled mobility.

A voice-detecting sensor and a scanning keyboard emulator to support word writing by two boys with extensive motor disabilities

The present study assessed the use of a voice-detecting sensor interfaced with a scanning keyboard emulator to allow two boys with extensive motor disabilities to write. Specifically, the study (a) compared the effects of the voice-detecting sensor with those of a familiar pressure sensor on the boys' writing time, (b) checked which of the sensors the boys preferred, and (c) conducted a social validation assessment of the boys' performance with the two sensors, employing psychology students as raters. The difference in the boys' overall mean writing time per letter across sensors was, by the end of the study, about 1.5s. This difference favored the pressure sensor for one of the boys and the voice-detecting sensor for the other boy. Both boys showed preference for the voice-detecting sensor. Moreover, the psychology students involved in the social validation assessment indicated that such sensor was more satisfactory, suitable, and educationally relevant than the pressure sensor, and represented the solution that they as raters supported more.

Evaluation of head orientation and neck muscle EMG signals as command inputs to a human-computer interface for individuals with high tetraplegia

We investigated the performance of three user interfaces for restoration of cursor control in individuals with tetraplegia: head orientation, electromyography (EMG) from face and neck muscles, and a standard computer mouse (for comparison). Subjects engaged in a 2-D, center-out, Fitts' Law style task and performance was evaluated using several measures. Overall, head orientation commanded motion resembled mouse commanded cursor motion (smooth, accurate movements to all targets), although with somewhat lower performance. EMG commanded movements exhibited a higher average speed, but other performance measures were lower, particularly for diagonal targets. Compared to head orientation, EMG as a cursor command source was less accurate, was more affected by target direction and was more prone to overshoot the target. In particular, EMG commands for diagonal targets were more sequential, moving first in one direction and then the other rather than moving simultaneous in the two directions. While the relative performance of each user interface differs, each has specific advantages depending on the application.

A magneto-inductive sensor based wireless tongue-computer interface

We have developed a noninvasive, unobtrusive magnetic wireless tongue-computer interface, called "Tongue Drive," to provide people with severe disabilities with flexible and effective computer access and environment control. A small permanent magnet secured on the tongue by implantation, piercing, or tissue adhesives, is utilized as a tracer to track the tongue movements. The magnetic field variations inside and around the mouth due to the tongue movements are detected by a pair of three-axial linear magneto-inductive sensor modules mounted bilaterally on a headset near the user's cheeks. After being wirelessly transmitted to a portable computer, the sensor output signals are processed by a differential field cancellation algorithm to eliminate the external magnetic field interference, and translated into user control commands, which could then be used to access a desktop computer, maneuver a powered wheelchair, or control other devices in the user's environment. The system has been successfully tested on six able-bodied subjects for computer access by defining six individual commands to resemble mouse functions. Results show that the Tongue Drive system response time for 87% correctly completed commands is 0.8 s, which yields to an information transfer rate of approximately 130 b/min.

Tongue operated assistive technologies

Assistive technologies help improving the quality of life for severely disabled individuals by enabling them to pursue self-care, educational, vocational, and recreational activities. Tongue has a set of unique characteristics that makes it a suitable appendage for manipulating paralyzed individuals' environments through the use of tongue-operated assistive devices. A number of these devices have been reviewed and a new one, called Tongue Drive, has been introduced.

A wireless tongue-computer interface using stereo differential magnetic field measurement

We have developed an enhanced prototype of the new Tongue Drive system (TDS), which is a noninvasive, unobtrusive wireless magnetic tongue-computer interface for people with severe disabilities. A small permanent magnet secured on the tongue using tissue adhesives, implantation, or piercing is utilized as a tracer to track tongue movements. The magnetic field variations due to the tracer movements are detected by a pair of 3-axial linear magnetic sensor modules mounted bilaterally on a headset near the user's cheeks. The sensors stereo outputs are processed and translated into user control commands after being wirelessly transmitted to a portable computer. These commands have been used in human trials to access the computer by substituting mouse functions. Measurement results showed a response time of less than 1.0 s with 99.9% accuracy for 6 direct commands, yielding an information transfer rate (ITR) of 150 bits/min.

An inductive tongue computer interface for control of computers and assistive devices

This letter introduces a new inductive tongue computer interface to be used by disabled people for environmental control. The interface demands little effort from the user, provides a basis for an invisible interface, and has potential to allow a large number of commands to be facilitated.

Accessible microscopy workstation for students and scientists with mobility impairments

An integrated accessible microscopy workstation was designed and developed to allow persons with mobility impairments to control all aspects of light microscopy with minimal human assistance. This system, named AccessScope, is capable of performing brightfield and fluorescence microscopy, image analysis, and tissue morphometry requisite for undergraduate science courses to graduate-level research. An accessible microscope is necessary for students and scientists with mobility impairments to be able to use a microscope independently to better understand microscopical imaging concepts and cell biology. This knowledge is not always apparent by simply viewing a catalog of histological images. The ability to operate a microscope independently eliminates the need to hire an assistant or rely on a classmate and permits one to take practical laboratory examinations by oneself. Independent microscope handling is also crucial for graduate students and scientists with disabilities to perform scientific research. By making a personal computer as the user interface for controlling AccessScope functions, different upper limb mobility impairments could be accommodated by using various computer input devices and assistive technology software. Participants with a range of upper limb mobility impairments evaluated the prototype microscopy workstation. They were able to control all microscopy functions including loading different slides without assistance.

An oral tactile interface for blind navigation

An oral tactile interface was designed and evaluated to provide directional cues through the tactile channel, which may be utilized by a blind traveler to obtain directional guidance in outdoor navigation. The device was implemented as a mouthpiece with a microfabricated electrotactile display on top for tactile presentation onto the roof of the mouth and a tongue touch keypad at the bottom for simultaneous operation by the tongue. An experimental system allowed a user to communicate with a computer tactilely by using the oral interface. Directional cues were presented to the user as line or arrow patterns with four moving directions (leftward, rightward, forward, and backward). Electrotactile presentation on the roof of the mouth was evaluated in experiments of threshold measurement and identification of directional cues. Experimental results from six human subjects showed that the roof of the mouth required stimulation intensities around 15 V for threshold sensation, and around 25-30 V for comfortable and well-perceived stimulation. Furthermore, identification of leftward or rightward movements was highly accurate while performance on forward or backward moving patterns was mixed and varied considerably among subjects.