Haptic force-feedback devices for the office computer: performance and musculoskeletal loading issues

Nondominant hand computer mouse training and the bilateral transfer effect to the dominant hand

This study explored the effects of training computer mouse use in the nondominant hand on clicking performance of the dominant and nondominant hands. Computer mouse use is a daily operation in the workplace and requires minute hand and wrist movements developed and refined through practice and training for many years. Our study had eleven right-handed computer mouse users train their nondominant hand for 15 min a day, five days per week, for six weeks. This study found improved performance with the computer mouse in the dominant hand following nondominant hand training because of the bilateral transfer effect of training. Additionally, our study showed that the nondominant hand is capable of learning the complex movements that our dominant hand has trained for many years. Last, our research showed that nondominant hand performance decreases when the skill is not trained for over a year, but the performance is significantly higher than that prior to the original training and can be rapidly relearned. Overall, training the nondominant hand on the computer mouse will allow for improved performance in industry while allowing safer, sustainable, and more achievable work in a multitude of economies.

A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping

Although robotic telemanipulation has always been a key technology for the nuclear industry, little advancement has been seen over the last decades. Despite complex remote handling requirements, simple mechanically linked master-slave manipulators still dominate the field. Nonetheless, there is a pressing need for more effective robotic solutions able to significantly speed up the decommissioning of legacy radioactive waste. This paper describes a novel haptic shared-control approach for assisting a human operator in the sort and segregation of different objects in a cluttered and unknown environment. A three-dimensional scan of the scene is used to generate a set of potential grasp candidates on the objects at hand. These grasp candidates are then used to generate guiding haptic cues, which assist the operator in approaching and grasping the objects. The haptic feedback is designed to be smooth and continuous as the user switches from a grasp candidate to the next one, or from one object to another one, avoiding any discontinuity or abrupt changes. To validate our approach, we carried out two human-subject studies, enrolling 15 participants. We registered an average improvement of 20.8%, 20.1%, and 32.5% in terms of completion time, linear trajectory, and perceived effectiveness, respectively, between the proposed approach and standard teleoperation.

Three-Dimensional Aerial Image Interface, 3DAII

In this paper, we introduce the three-dimensional aerial image interface, 3DAII. This interface reconstructs and aerially projects a three-dimensional object image, which can be simultaneously observed from various viewpoints or by multiple users with the naked eye. A pyramid reflector is used to reconstruct the object image, and a pair of parabolic mirrors is used to aerially project the image. A user can directly manipulate the three-dimensional object image by superimposing a user’s hand-finger or a rod on the image. A motion capture sensor detects the user’s hand-finger that manipulates the projected image, and the system immediately exhibits some reaction such as deformation, displacement, and discoloration of the object image, including sound effects. A performance test is executed to confirm the functions of 3DAII. The execution time of the end-tip positioning of a robotic arm has been compared among four operating devices: touchscreen, gamepad, joystick, and 3DAII. The results exhibit the advantages of 3DAII; we can directly instruct the movement direction and movement speed of the end-tip of the robotic arm, using the three-dimensional Euclidean vector outputs of 3DAII in which we can intuitively make the end-tip of the robotic arm move in three-dimensional space. Therefore, 3DAII would be one important alternative to an intuitive spatial user interface, e.g., an operation device of aerial robots, a center console of automobiles, and a 3D modelling system. A survey has been conducted to evaluatecomfortandfatiguebased on ISO/TS 9241-411 andease of learningandsatisfactionbased on the USE questionnaire. We have identified several challenges related to visibility, workspace, and sensory feedback to users that we would like to address in the future.

Haptics to improve task performance in people with disabilities: A review of previous studies and a guide to future research with children with disabilities

This review examines the studies most pertinent to the potential of haptics on the functionality of assistive robots in manipulation tasks for use by children with disabilities. Haptics is the fast-emerging science that studies the sense of touch concerning the interaction of a human and his/her environment; this paper particularly studies the human–machine interaction that happens through a haptic interface to enable touch feedback. Haptics-enabled user interfaces for assistive robots can potentially benefit children whose haptic exploration is impaired due to a disability in their infancy and throughout their childhood. A haptic interface can provide touch feedback and potentially contribute to an enhancement in perception of objects and overall ability to perform manipulation tasks. The intention of this paper is to review the research on the applications of haptics, exclusively focusing on attributes affecting task performance. A review of studies will give a retrospective insight into previous research with various disability populations, and inform potential limitations/challenges in research regarding haptic interfaces for assistive robots for use by children with disabilities.

Force-sensitive detents improve user performance for linear selection tasks

Haptic technology, providing force cues and creating a programmable interface, can assist users in more accurately using an interface. This paper investigates haptic assistance in combination with auditory feedback instead of visual feedback. A user test is carried out in which participants select fundamental frequencies from a continuous range to play brief musical melodies. Two control conditions are compared with two detent-based haptic assistance conditions. The detents gently guide the users toward locations of equal tempered fundamental frequencies. Results from the user test confirm improved accuracy brought about by the detents. It is further helpful to provide regulation of the strength of haptic assistance in real time, allowing the user to remain always in control. This concept motivated the force-sensitive detent condition, which enables the user to adjust the strength of the haptic assistance in real time by changing the downward force applied to the haptic device. The work implies that users of graphical user interfaces could similarly benefit from force-sensitive detents and more generally real-time regulation of the strength of haptic assistance.

A Basic Study on Biological Signal of Operator During Master-Slave System Control

In a master-slave system, operators control the master device based on visual information and sensory feedback. This suggests that the position sensors on the slave side are not always necessary. However, position signals are required to estimate the acting force conveyed to the slave using an observer which realize the force sensorless control. In this paper, we focused on the biological signals of the operator to estimate the slave side’s condition which can be used to estimate the external force. In the early stage of the research, we investigate biological signals just with 1-DOF masterslave system. First, we measured electromyographic (EMG) and gripping force of operators when pushing objects. We verified that predictive signals can be used for estimation of touching the objects. Second, we propose a method, which uses wrist joint torque and Total Contraction Level (TCL) calculated by the EMG when pushing the virtual elastic films, of estimating the acting force conveyed to the slave. We verify that the proposed method can estimate external force under specific conditions with a trained subject.

Effects of four types of non-obtrusive feedback on computer behaviour, task performance and comfort

This study investigated the effects of non-obtrusive feedback on continuous lifted hand/finger behaviour, task performance and comfort. In an experiment with 24 participants the effects of two visual and two tactile feedback signals were compared to a no-feedback condition in a computer task. Results from the objective measures showed that all types of feedback were equally effective to reduce lifted hand/finger behaviour (effectiveness) compared to absence of feedback, while task performance was not affected (efficiency). In contrast to objective measures, subjective user experience was significantly different for the four types of feedback signals. Continuous tactile feedback appeared to be the best signal; not only the effectiveness and efficiency were rated reasonable, it also scored best on perceived match between signal and required action. This study shows the importance of including user experiences when investigating usability of feedback signals. Non-obtrusive feedback embedded in products and environments may successfully be used to support office workers to adopt healthy, productive and comfortable working behaviour.

An adaptive dynamic pointing assistance program to help people with multiple disabilities improve their computer pointing efficiency with hand swing through a standard mouse

The latest research adopted software technology to redesign the mouse driver, and turned a mouse into a useful pointing assistive device for people with multiple disabilities who cannot easily or possibly use a standard mouse, to improve their pointing performance through a new operation method, Extended Dynamic Pointing Assistive Program (EDPAP), where the user can swing his hand on the desktop to quickly move the cursor to a target. However, EDPAP has lower pointing efficiency due to the cursor jumping in sequence amongst the targets once there are many targets on the screen. This study evaluated whether two people with multiple disabilities would be able to improve their pointing performance through a new target acquisition technique based on EDPAP, Adaptive Dynamic Pointing Assistive Program (ADPAP), where the cursor movement path is selected adaptively by users' hand swing direction, as opposed to being sequential. This study used multiple probe design across participants. Participants typically received three 30-min ADPAP training sessions per week, for a period of about 6-7 weeks. Initially, both participants had their baseline sessions. Then intervention started with the first participant. When his performance was consolidated, new baseline and intervention occurred with the second participant. Finally, both participants were exposed to the maintenance phase, in which their pointing performance improved significantly. Data indicated that both participants improved their pointing efficiency with the use of ADPAP and remained highly successful through the maintenance phase. Results of this study showed that, with the assistance of ADPAP, participants can acquire targets quickly, easily, and accurately, thus helping the disabled to solve their pointing problems.

Assisting people with multiple disabilities improve their computer pointing efficiency with thumb poke through a standard trackball

This study evaluated whether two people with multiple disabilities who could not easily use a computer through a standard input device (i.e., mouse or trackball) would be able to improve their pointing performance using thumb poke with a standard trackball through a Dynamic Trackball-Pointing Assistive Program (DTPAP) and a newly developed trackball driver (i.e., a new trackball driver replaces the standard trackball driver, and changes a trackball into a precise thumb poke detector, and intercepts trackball action). Initially, both participants were given baseline sessions, then intervention started with the first participant. When his performance was consolidated, new baseline and intervention occurred with the second participant. Finally, both participants were exposed to the maintenance phase. Data indicated that both participants improved their pointing performance significantly with the use of DTPAP and remained highly successful through the maintenance phase. Implications of the findings are discussed.

Evaluation of automatic pointing assistive function effect in cursor-positioning task for people with disabilities

PURPOSE

This study evaluates the cursor-positioning effect with the automatic pointing assistive programme (APAP) and a newly mouse driver for people with disabilities.

METHOD

The APAP is designed to move the cursor to a target instantaneously when the cursor reaches inside this circular activation area. In this study, 20 people with disabilities participated in the experiment. Trial completion time was determined and compared with those in previous studies of cursor-capturing functions (jumping and gravity) and normal condition.

RESULT

Experimental results reveal that the pointing efficiency was improved by 65.29% with the APAP compared to that of the normal condition (p < 0.001) for people with disabilities. Data were subjected to one-way analysis of variance (ANOVA), with the level of significance p ≤ 0.05 and the ANOVA indicates that the capturing functions had a significant effect on the trial completion time (F(2,57) = 199.21; p < 0.001).

CONCLUSIONS

The performance of the APAP is superior to those in previous studies in positioning time for people with disabilities. The APAP function is expected to help aged users and novices in operating a mouse efficiently and easily.

Development of a Pneumatic Surgical Manipulator IBIS IV

In teleoperated, minimally invasive surgery systems, the measurement and conveyance of a sense of force to the operator is problematic. In order to carry out safer and more precise operations using robotic manipulators, force measurement and operator feedback are very important factors. We previously proposed a pneumatic surgical manipulator that is capable of estimating external force without the use of force sensors. However, the force estimation had a sensitivity of only 3 N because of inertia and friction effects. In this paper, we develop a new and improved model of the pneumatic surgical manipulator, IBIS IV. We evaluate its performance in terms of force estimation. The experimental results indicate that IBIS IV estimates external forces with a sensitivity of 1.0 N. We also conduct an in-vivo experiment and confirm the effectiveness and improvement of the manipulator.

Assisting people with multiple disabilities and minimal motor behavior to improve computer pointing efficiency through a mouse wheel

This study evaluated whether two people with multiple disabilities and minimal motor behavior would be able to improve their pointing performance using finger poke ability with a mouse wheel through a Dynamic Pointing Assistive Program (DPAP) and a newly developed mouse driver (i.e., a new mouse driver replaces standard mouse driver, changes a mouse wheel into a thumb/finger poke detector, and intercepts mouse action). Initially, both participants had their baseline sessions. Then intervention started with the first participant. When his performance was consolidated, new baseline and intervention occurred with the second participant. Finally, both participants were exposed to maintenance phase, in which their pointing performance improved significantly. Both participants improved their pointing efficiency with the use of DPAP and remained highly successful through maintenance phase. Implications of the findings are discussed.

Alternative computer mouse design and testing to reduce finger extensor muscle activity during mouse use

OBJECTIVE

The purpose of this study was to design and test alternative computer mouse designs that attempted to reduce extensor muscle loading of the index and middle fingers by altering the orientation of the button switch direction and the force of the switch.

BACKGROUND

Computer users of two-button mouse designs exhibit sustained lifted finger behaviors above the buttons, which may contribute to hand and forearm musculoskeletal pain associated with intensive mouse use.

METHODS

In a repeated-measures laboratory experiment, 20 participants completed point-and-click, steering, and drag tasks with four alternative mouse designs and a reference mouse. Intramuscular and surface electromyography (EMG) measured muscle loading, and movement times recorded by software provided a measure of performance.

RESULTS

Changing the direction of the switch from a conventional downward to a forward design reduced (up to 2.5% maximum voluntary contraction [MVC]) sustained muscle activity (10th percentile EMG amplitude distribution) in the finger extensors but increased (up to 0.6% MVC) flexor EMG and increased movement times (up to 31%) compared with the reference mouse (p < .001). Implementing a high switch force design also increased flexor EMG but did not differ in movement times compared with the reference mouse (p < .001).

CONCLUSION

The alternative mouse designs with altered switch direction reduced sustained extensor muscle loading; however, trade-offs with higher flexor muscle loading and lower performance existed.

APPLICATION

Potential applications of this study include ergonomic and human computer interface design strategies in reducing the exposure to risk factors that may lead to upper extremity musculoskeletal disorders.

Forearm electromyographic changes with the use of a haptic force-feedback computer mouse

OBJECTIVE

To examine changes in biomechanical and motor control associated with a force-feedback computer mouse.

BACKGROUND

Haptic computer mice can improve the movement times for point-and-click tasks; however, changes in upper extremity biomechanics and motor control are unknown.

METHOD

Eighteen people (ages 22-37 years) performed a point-and-click task 80 times using a force-feedback computer mouse across three different conditions: (a) no force feedback, emulating a conventional mouse; (b) a single attractive force field at the desired target that pulls the mouse to the center of the target; and (c) an attractive force field at the desired target as well as others between the two possible targets, distracting the user from the intended target. Cursor kinematics, wrist posture, and electromyographic (EMG) forearm muscle activity were recorded.

RESULTS

The point-and-click movements were 30% faster with the addition of a single force field and 3% faster with the addition of multiple force fields. The Fitts' law index of performance metrics improved from 2.9 bits/response to 4.1 bits/response for multiple attractive fields and to 6.0 bits/response for a single force field. For the distracting force fields, the cursor maximum velocities were over 50% faster. EMG amplitude values were largest for the distracting force fields. These data suggest that the operator uses increased muscle activity to accelerate the mouse through the distracting force fields.

CONCLUSION

When implementing attractive haptic force fields, one needs to consider how to reduce these observed effects of potential distracting force fields.

APPLICATION

Applications include human-computer interface design for pointing devices extensively used for the graphical user interface.

Multimodal feedback for the acquisition of small targets

This paper examines how multimodal feedback assists small-target acquisition in graphical user interfaces. All combinations of three feedback modes are analysed: non-speech audio; tactile; and pseudo-haptic "sticky" feedback. The tactile conditions used stimulation through vibration (rather than force-feedback), and the sticky conditions were implemented by dynamically reconfiguring mouse control-display gain as the cursor entered the target. Results show that for small, discretely located targets all feedback modes reduce targeting times, with stickiness providing substantial improvements. Furthermore, stickiness and tactile appear to combine well. However, the results of a more ecologically oriented menu-selection task show the need for caution, revealing that excessive feedback can damage interaction though "noise" that interferes with the acquisition of neighbouring targets.

Effect of sensory substitution on suture-manipulation forces for robotic surgical systems

OBJECTIVES

Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators.

METHODS

Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback.

RESULTS

The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces.

CONCLUSIONS

Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.

Sharing control between humans and automation using haptic interface: primary and secondary task performance benefits

This paper describes a paradigm for human/automation control sharing in which the automation acts through a motor coupled to a machine's manual control interface. The manual interface becomes a haptic display, continually informing the human about automation actions. While monitoring by feel, users may choose either to conform to the automation or override it and express their own control intentions. This paper's objective is to demonstrate that adding automation through haptic display can be used not only to improve performance on a primary task but also to reduce perceptual demands or free attention for a secondary task. Results are presented from three experiments in which 11 participants completed a lane-following task using a motorized steering wheel on a fixed-base driving simulator. The automation behaved like a copilot, assisting with lane following by applying torques to the steering wheel. Results indicate that haptic assist improves lane following by least 30%, p < .0001, while reducing visual demand by 29%, p < .0001, or improving reaction time in a secondary tone localization task by 18 ms, p = .0009. Potential applications of this research include the design of automation interfaces based on haptics that support human/automation control sharing better than traditional push-button automation interfaces.

Effect of tactile feedback on movement speed and precision during work-related tasks using a computer mouse

Effects of tactile feedback on movement accuracy and speed were studied. Younger and older participants performed three tasks (1, select and drag word; 2, menu navigation; 3, select and drag cell) using commercial software and a mouse with or without tactile feedback. Task time and error number were recorded. Tasks were divided according to presence or absence of tactile feedback, and participants were divided into subgroups (high, average, low) based on Task 1 performance. Overall, older participants took longer (p < .0001) and made more errors (p < .001) than younger participants. There was an effect of feedback by task in younger participants for all six outcomes (p < 0.02). At the task level, with feedback, younger participants reduced performance time (13%) and errors (24%) on Task 1. Low- and average-performance younger participants benefited most from feedback for Task 1. Older low-performance participants also benefited from feedback for Task 1. For Task 3, older participants tended to take more time and make more errors with feedback. Tactile feedback may enhance performance when feedback is event related. Older people may not integrate sensation as well as younger individuals to enhance performance. Potential applications of this research include the development of tactile feedback interfaces to facilitate computer use.

The form of augmented force-feedback fields and the efficiency and satisfaction in computer-aided pointing tasks

This study investigates operation efficiency and user satisfaction for spatial and temporal shapes of augmented force-feedback fields to be used with computer pointing devices in target acquisition tasks. In an experiment, three different force-field shapes at two different mean force levels were compared, with an electromechanical force-feedback trackball as control device and with efficiency and user satisfaction as dependent variables. Efficiency was measured by the time required to perform a certain task, and user satisfaction was measured through a subjective rating procedure. Satisfaction results indicate that to a rough approximation, participants can be grouped into those who prefer high and those who prefer low force levels. Members of the former group were most satisfied with force fields with a gradual start and an abrupt ending. This force-field shape also proved to be the most efficient for both groups. When all participants were considered as a single group, none of the effects was found to be statistically significant. A gender effect was also found; in both preference groups women achieved significantly shorter task completion times than men. Actual or potential applications of this research include enabling computer interaction for people prone to repetitive strain injuries and the increment of efficiency and satisfaction in human-computer interaction in general.