Tactile cueing effects on performance in simulated aerial combat with high acceleration

When to use vibrotactile displays? A meta-analysis for the role of vibrotactile displays in human-computer interaction

OBJECTIVE

This study aims to investigate the benefits of unimodal tactile displays relative to other modal displays and the performance gains of adding redundant tactile displays by integrating empirical studies.

BACKGROUND

Tactile displays have attracted increasing attention in recent years due to their unique advantages. Synthesizing experimental data is necessary to analyze the performance benefits of tactile displays for participants and better help practitioners in utilizing them.

METHOD

Five meta-analyses were conducted. Two meta-analyses compared the participants' performance between tactile and other modal displays (visual vs. tactile and auditory vs. tactile). Three meta-analyses examined the performance gains of adding redundant tactile displays based on other modal displays (visual vs. visual + tactile, auditory vs. auditory + tactile, and visual + auditory vs. visual + auditory + tactile). The related moderator variables, the types of presented information and concurrent tasks, were analyzed.

RESULTS

Little evidence shows the performance difference between tactile and auditory displays. Tactile displays are more beneficial than visual displays for presenting alert information or in the situation with a visual concurrent task. The performance gains of adding redundant tactile displays to other modal displays also depend on the specific type of presented information and the concurrent task.

CONCLUSION

When using tactile displays to convey information, interface designers should consider the specific type of presented information and the concurrent tasks.

APPLICATIONS

The present study's findings can provide some implications for designers to utilize tactile displays when they construct and implement information displays.

Gaze-independent ERP-BCIs: augmenting performance through location-congruent bimodal stimuli

Gaze-independent event-related potential (ERP) based brain-computer interfaces (BCIs) yield relatively low BCI performance and traditionally employ unimodal stimuli. Bimodal ERP-BCIs may increase BCI performance due to multisensory integration or summation in the brain. An additional advantage of bimodal BCIs may be that the user can choose which modality or modalities to attend to. We studied bimodal, visual-tactile, gaze-independent BCIs and investigated whether or not ERP components’ tAUCs and subsequent classification accuracies are increased for (1) bimodal vs. unimodal stimuli; (2) location-congruent vs. location-incongruent bimodal stimuli; and (3) attending to both modalities vs. to either one modality. We observed an enhanced bimodal (compared to unimodal) P300 tAUC, which appeared to be positively affected by location-congruency (p = 0.056) and resulted in higher classification accuracies. Attending either to one or to both modalities of the bimodal location-congruent stimuli resulted in differences between ERP components, but not in classification performance. We conclude that location-congruent bimodal stimuli improve ERP-BCIs, and offer the user the possibility to switch the attended modality without losing performance.

Visual, Tactile, and Bimodal Presentation of Lateral Drift in Simulated Helicopter

Helicopter landing and take-off in degraded visibility caused by blowing sand or dust (brown-out) may distort the pilot’s comprehension of the helicopter’s position. This is a serious problem that may lead to unattended lateral drift or descending rate. We have previously shown advantages of redundant tactile and multimodal information a simulated combat vehicle. In order to investigate if lateral drift in a helicopter can be reduced by use of a drift display an experiment with a simulated helicopter was performed. Three types of drift displays were tested: visual, tactile, and bimodal display and compared with the primary display that did not present lateral drift. Compared with the primary display lateral drift was reduced with all three drift display configurations. This indicates the value of a drift display in the helicopter and the possibilities of disengaging the pilot’s vision for parallel tasks by the use of tactile or bimodal drift displays.

Improving target detection in visual search through the augmenting multi-sensory cues

The present experiment tested 60 individuals on a multiple screen, visual target detection task. Using a within-participant design, individuals received no-cue augmentation, an augmenting tactile cue alone, an augmenting auditory cue alone or both of the latter augmentations in combination. Results showed significant and substantive improvements in performance such that successful search speed was facilitated by more than 43%, errors of omission were reduced by 86% and errors of commission were reduced by more than 77% in the combinatorial cueing condition compared with the non-cued control. These outcomes were not a trade of performance efficiency for associated mental effort because recorded levels of cognitive workload were also reduced by more than 30% in the multi-cued circumstance compared with the control condition. When the tactile modality was incorporated it led to the highest gain in performance speed, when the auditory modality was incorporated, it led to the best levels of performance accuracy. The combined condition rendered the best of each from of performance increment. Reasons for this outcome pattern are discussed alongside their manifest practical benefits.

PRACTITIONER SUMMARY

This experiment tested 60 individuals on a multiple screen, visual target detection task. Individuals received no-cue augmentation, tactile cue alone, an augmenting auditory cue alone or both of the latter augmentations in combination. Results showed significant and substantive improvements in the combinatorial cueing condition compared with the non-cued control.

Control-display mapping in brain-computer interfaces

Event-related potential (ERP) based brain-computer interfaces (BCIs) employ differences in brain responses to attended and ignored stimuli. When using a tactile ERP-BCI for navigation, mapping is required between navigation directions on a visual display and unambiguously corresponding tactile stimuli (tactors) from a tactile control device: control-display mapping (CDM). We investigated the effect of congruent (both display and control horizontal or both vertical) and incongruent (vertical display, horizontal control) CDMs on task performance, the ERP and potential BCI performance. Ten participants attended to a target (determined via CDM), in a stream of sequentially vibrating tactors. We show that congruent CDM yields best task performance, enhanced the P300 and results in increased estimated BCI performance. This suggests a reduced availability of attentional resources when operating an ERP-BCI with incongruent CDM. Additionally, we found an enhanced N2 for incongruent CDM, which indicates a conflict between visual display and tactile control orientations.

PRACTITIONER SUMMARY

Incongruency in control-display mapping reduces task performance. In this study, brain responses, task and system performance are related to (in)congruent mapping of command options and the corresponding stimuli in a brain-computer interface (BCI). Directional congruency reduces task errors, increases available attentional resources, improves BCI performance and thus facilitates human-computer interaction.

Enhanced perception and performance by multimodal threat cueing in simulated combat vehicle

OBJECTIVE

In a simulated combat vehicle, uni-, bi-, and trimodal cueing of direction to threat were compared with the purpose to investigate whether multisensory redundant information may enhance dynamic perception and performance.

BACKGROUND

Previous research has shown that multimodal display presentation can enhance perception of information and task performance.

METHOD

Two experiments in a simulated combat vehicle were performed under the instructions to turn the vehicle toward the threat as fast and accurately as possible after threat cue onset. In Experiment 1, direction to threat was presented by four display types: visual head-down display, tactile belt, 3-D audio, and trimodal with the three displays combined. In Experiment 2, direction to threat was presented by three display types: visual head-up display (HUD)-3-D audio, tactile belt-3-D audio, and trimodal with HUD, tactile belt, and 3-D audio combined.

RESULTS

In Experiment I,the trimodal display provided overall best performance and perception of threat direction. In Experiment 2, both the trimodal and HUD--3-D audio displays led to overall best performance, and the trimodal display provided overall the best perception of threat direction. None of the trimodal displays induced higher mental workload or secondary task interference.

CONCLUSION

The trimodal displays provided overall enhanced perception and performance in the dynamically framed threat scenario and did not entail higher mental workload or decreased spare capacity.

APPLICATION

Trimodal displays with redundant information may contribute to safer and more reliable peak performance in time-critical dynamic tasks and especially in more extreme and stressful situations with high perceptual or mental workload.

A tactile P300 brain-computer interface

In this study, we investigated a Brain-Computer Interface (BCI) based on EEG responses to vibro-tactile stimuli around the waist. P300 BCIs based on tactile stimuli have the advantage of not taxing the visual or auditory system and of being potentially unnoticeable to other people. A tactile BCI could be especially suitable for patients whose vision or eye movements are impaired. In Experiment 1, we investigated its feasibility and the effect of the number of equally spaced tactors. Whereas a large number of tactors is expected to enhance the P300 amplitude since the target will be less frequent, it could also negatively affect the P300 since it will be difficult to identify the target when tactor density increases. Participants were asked to attend to the vibrations of a target tactor, embedded within a stream of distracters. The number of tactors was two, four or six. We demonstrated the feasibility of a tactile P300 BCI. We did not find a difference in SWLDA classification performance between the different numbers of tactors. In a second set of experiments we reduced the stimulus onset asynchrony (SOA) by shortening the on- and/or off-time of the tactors. The SOA for an optimum performance as measured in our experiments turned out to be close to conventional SOAs of visual P300 BCIs.

Field-Based Validation of a Tactile Navigation Device

In this paper, we present three field-based evaluations of a tactile land navigation system. In Experiment 1, we transition from a laboratory setting to rugged terrain used to train US Army soldier land navigation. Navigation in this challenging terrain requires careful attention to one's surroundings. Participants navigated 3 waypoints along 600 meters through heavily wooded terrain, using 1) map and compass, 2) standard alpha-numeric handheld GPS device, and 3) the tactile GPS system, while also responding to radio requests for information. Experiment 2 used the same challenging terrain during night operations, where participants must also search for live and silhouette targets, using 1) handheld GPS device, 2) head-mounted map-based GPS, and 3) the tactile GPS system. In addition to navigating, participants searched for silhouette and live (human) targets. Experiment 3 had participants navigate with 1) a commercial GPS arrow display, 2) the tactile GPS system, and 3) both together. We conclude that tactile navigation displays can be used in strenuous outdoor environments and can outperform visual displays under conditions of high cognitive and visual workload.

Direction coding using a tactile chair

This laboratory study examined the possibility of using a car seat instrumented with a tactile display to communicate directional information to a driver. A car seat fitted with an 8 x 8 matrix of vibrators embedded in the seat pan was used to code eight different directions. Localization response time and angular accuracy were examined as a function of stimulus direction, presence of a tactile attention cue, temporal pattern, stimulus layout, age, and gender. The mean absolute angular error was 23 degrees, and both localization accuracy and response times were superior for the back left, backward, and back right directions. Of the various temporal pattern/attention cue combinations examined, results favored the relatively fast patterns consisting of vibration bursts of 125 or 250 ms without a centrally located attention cue over 500 ms bursts that were preceded by an attention cue. Observed age and gender effects were relatively modest, suggesting that using tactile cueing to communicate direction is effective across a wide range of users. In addition, the tactile stimulus was detected by more than 90% of the participants under surprise trial conditions. Overall, these results indicate that the tactile chair provides a promising and robust method of providing directional information.

Multimodal Threat Cueing in Simulated Combat Vehicle

We investigated three types of display combinations for threat cueing in a simulated combat vehicle. The display combinations consisted of two bimodal combinations, a visual head-up display (HUD) combined with 3D audio; a tactile torso belt combined with 3D audio; and a multimodal combination, the HUD, tactile belt, and 3D audio combined. The participant's main task was to as fast as possible align the heading of the combat vehicle with the displayed direction to a threat. To increase general task difficulty and provide a secondary measure of mental workload, the participant also was required to identify radio calls. Threat localization accuracy was highest and reaction time shortest with the use of both the HUD combined with 3D audio and with the multimodal display. Subjective ratings of perception of initial threat direction were most positive for both the tactile belt combined with 3D audio and for the multimodal display. The ratings of perceived threat direction at the final phase of threat alignment, however, were most positive for the HUD combined with 3D audio and for the multimodal display. Thus, the multimodal display with HUD, tactile belt, and 3D audio combined proved to be beneficial for all measures.

Tactile and Multisensory Spatial Warning Signals for Drivers

The last few years have seen many exciting developments in the area of tactile and multisensory interface design. One of the most rapidly-moving practical application areas for these findings is in the development of warning signals and information displays for drivers. For instance, tactile displays can be used to awaken sleepy drivers, to capture the attention of distracted drivers, and even to present more complex information to drivers who may be visually-overloaded. This review highlights the most important potential costs and benefits associated with the use of tactile and multisensory information displays in a vehicular setting. Multisensory displays that are based on the latest cognitive neuroscience research findings can capture driver attention significantly more effective than their unimodal (i.e., tactile) counterparts. Multisensory displays can also be used to transmit information more efficiently, as well as to reduce driver workload. Finally, we highlight the key research questions currently awaiting further research, including questions such as: Are tactile warning signals really intuitive? Are there certain regions of the body (or in the space surrounding the body) where tactile/multisensory warning signals are particularly effective? To what extent is the spatial coincidence and temporal synchrony of the individual sensory signals critical to determining the effectiveness of multisensory displays? And, finally, how does the issue of compliance vs. reliance (or the 'cry wolf' phenomenon associated with the presentation of signals that are perceived as false alarms) influence the effectiveness of tactile and/or multisensory warning signals?