Quantitative effects of overlay clutter and information access effort: Examining the scan-clutter trade-off in displays with geospatial maps

Overlaying images from multiple geospatial databases increases clutter and imposes attentional costs by disrupting focusing attention on each database and dividing attention when comparing databases. Costs of overlay clutter may offset the benefits of reduced scanning between two images displayed separately. In two experiments, we examine these attention issues using computational metrics to quantify clutter. We also examine how the scan-clutter trade-off is modified by different levels of clutter, display separation, and task attentional requirements. Participants viewed information from a geographical terrain database and a schematic map database and made judgments that required focusing attention on either database or integrating information across both. In Experiment 1, databases were presented as either overlaid or adjacent displays, and in Experiment 2, as either overlay, adjacent, or more separated displays. Results showed that response time was modulated by the magnitude of clutter, spatial separation, and task type. Results also revealed that clutter costs dominated those of spatial separation, particularly in tasks requiring focused attention. A computational feature congestion metric of clutter effectively predicted performance but could be improved by incorporating an overlay component, which amplified the costs of clutter. The results provide design guidelines for overlay displays (e.g., head-mounted displays) that will minimize the scan-clutter trade-off. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Beyond the Wizard of Oz: Negative Effects of Imperfect Machine Learning to Examine the Impact of Reliability of Augmented Reality Cues on Visual Search Performance

Despite knowing exactly what an object looks like, searching for it in a person's visual field is a time-consuming and error-prone experience. In Augmented Reality systems, new algorithms are proposed to speed up search time and reduce human errors. However, these algorithms might not always provide 100% accurate visual cues, which might affect users' perceived reliability of the algorithm and, thus, search performance. Here, we examined the detrimental effects of automation bias caused by imperfect cues presented in the Augmented Reality head-mounted display using the YOLOv5 machine learning model. 53 participants in the two groups received either 100% accurate visual cues or 88.9% accurate visual cues. Their performance was compared with the control condition, which did not include any additional cues. The results show how cueing may increase performance and shorten search times. The results also showed that performance with imperfect automation was much worse than perfect automation and that, consistent with automation bias, participants were frequently enticed by incorrect cues.

Supporting detection of hostile intentions: automated assistance in a dynamic decision-making context

In a dynamic decision-making task simulating basic ship movements, participants attempted, through a series of actions, to elicit and identify which one of six other ships was exhibiting either of two hostile behaviors. A high-performing, although imperfect, automated attention aid was introduced. It visually highlighted the ship categorized by an algorithm as the most likely to be hostile. Half of participants also received automation transparency in the form of a statement about why the hostile ship was highlighted. Results indicated that while the aid's advice was often complied with and hence led to higher accuracy with a shorter response time, detection was still suboptimal. Additionally, transparency had limited impacts on all aspects of performance. Implications for detection of hostile intentions and the challenges of supporting dynamic decision making are discussed.

Information Access Effort: The Role of Head Movements for Information Presented at Increasing Eccentricity on Flat Panel and Head-Mounted Displays

OBJECTIVE

This experiment examined performance costs when processing two sources of information positioned at increasing distances using a flat panel display and an augmented reality head-mounted display (AR-HMD).

BACKGROUND

The AR-HMD enables positioning virtual information at various distances in space. However, the proximity compatibility principle suggests that closer separation when two sources of information require mental integration assists performance, whereas increased separation between two sources hurts integration performance more than when a single source requires focused attention. Previous studies have provided inconsistent findings regarding costs associated with increased separation. Few of these experiments have examined separation for both focused and integration tasks, compared vertical and lateral separation, or measured head movements.

METHOD

Three experiments collectively examined these issues using a flat panel display and a virtual display presented with an HMD, where the separation of information varied laterally or vertically during a focused attention (digit reading) task and an information integration (mental subtraction) task.

RESULTS

There was no performance cost for either display when information was increasingly separated. However, head movements mitigated performance costs by preserving accuracy at larger separations without increasing response time.

CONCLUSION

Head movements appear to mitigate performance costs associated with presenting information increasingly far apart on flat panel displays and HMDs. Both eye scanning and head movements appear to be less effortful than expected.

APPLICATION

These findings have important implications for design guidelines regarding the placement of information presented on flat panel displays and, more specifically, HMDs, which can present information 360° around the user.

Rendezvous Under Temporal Uncertainty

OBJECTIVE

Three experiments sought to understand performance limitations in controlling a ship attempting to meet another moving ship that approached from various trajectories. The influence of uncertainty, resulting from occasional unpredictable delays in one's own movement, was examined.

BACKGROUND

Cognitive elements of rendezvous have been little studied. Related work such as the planning fallacy and bias toward underestimating time-to-contact imply a tendency toward late arrival at a rendezvous.

METHODS

In a simplified simulation, participants controlled the speed and/or heading of their own ship once per scenario to try to rendezvous with another ship. Forty-five scenarios of approximately 30 s were conducted with different starting geometries and, in two of three experiments, with different frequencies and lengths of the unexpected delays.

RESULTS

Perfect rendezvous were hard to obtain, with a general tendency to arrive late and pass behind the target vessel, although this was dependent on the angle of approach and relative speed. When occasional delays were introduced, less frequent but longer delays disrupted performance more than shorter but more frequent delays. Where delays were possible, but no delay occurred, there was no longer evidence of a general tendency to more frequently pass behind the target ship. Additionally, people did not wait to see if the unpredictable delays would occur before executing a course of action. Different control strategies were deployed and dual axis control was preferred.

CONCLUSIONS

The tendency to arrive late and the influence of the possibility of uncertain delays are discussed in relationship to control strategies.

Mitigating the Costs of Spatial Transformations With a Situation Awareness Augmented Reality Display: Assistance for the Joint Terminal Attack Controller 3-17

OBJECTIVE

Evaluate and model the advantage of a situation awareness (SA) supported by an augmented reality (AR) display for the ground-based joint terminal attack Controller (JTAC), in judging and describing the spatial relations between objects in a hostile zone.

BACKGROUND

The accurate world-referenced description of relative locations of surface objects, when viewed from an oblique slant angle (aircraft, observation post) is hindered by (1) the compression of the visual scene, amplified at a lower slang angle, (2) the need for mental rotation, when viewed from a non-northerly orientation.

APPROACH

Participants viewed a virtual reality (VR)-simulated four-object scene from either of two slant angles, at each of four compass orientations, either unaided, or aided by an AR head-mounted display (AR-HMD), depicting the scene from a top-down (avoiding compression) and north-up (avoiding mental rotation) perspective. They described the geographical layout of four objects within the display.

RESULTS

Compared with the control condition, that condition supported by the north-up SA display shortened the description time, particularly on non-northerly orientations (9 s, 30% benefit), and improved the accuracy of description, particularly for the more compressed scene (lower slant angle), as fit by a simple computational model.

CONCLUSION

The SA display provides large, significant benefits to this critical phase of ground-air communications in managing an attack-as predicted by the task analysis of the JTAC.

APPLICATION

Results impact the design of the AR-HMD to support combat ground-air communications and illustrate the magnitude by which basic cognitive principles "scale up" to realistically simulated real-world tasks such as search and rescue.

Detection of Hostile Intent by Spatial Movements

OBJECTIVE

The ability of people to infer intentions from movement of other vessels was investigated. Across three levels of variability in movements in the path of computer-controlled ships, participants attempted to determine which entity was hostile.

BACKGROUND

Detection of hostile intentions through spatial movements of vessels is important in an array of real-world scenarios. This experiment sought to determine baseline abilities of humans to do so.

METHODS

Participants selected a discrete movement direction of their ship. Six other ships' locations then updated. A single entity displayed one of two hostile behaviors: shadowing, which involved mirroring the participant's vessel's movements; and hunting, which involved closing in on the participant's vessel. Trials allowed up to 35 moves before identifying the hostile ship and its behavior. Uncertainty was introduced through adding variability to ships' movements such that their path was 0%, 25%, or 50% random.

RESULTS

Even with no variability in the ships' movements, accurate detection was low, identifying the hostile entity about 60% of the time. Variability in the paths decreased detection. Detection of hunting was strongly degraded by distance between ownship and the hostile ship, but shadowing was not. Strategies employing different directions of movement across the trial, but also featuring some runs of consecutive movements, facilitated detection.

CONCLUSIONS

Early identification of threats based on movement characteristics alone is likely to be difficult, but particularly so when adversaries employ some level of uncertainty to mask their intentions. These findings highlight the need to develop decision aids to support human performance.

How history trails and set size influence detection of hostile intentions

Previous research suggests people struggle to detect a series of movements that might imply hostile intentions of a vessel, yet this ability is crucial in many real world Naval scenarios. To investigate possible mechanisms for improving performance, participants engaged in a simple, simulated ship movement task. One of two hostile behaviors were present in one of the vessels: Shadowing—mirroring the participant’s vessel’s movements; and Hunting—closing in on the participant’s vessel. In the first experiment, history trails, showing the previous nine positions of each ship connected by a line, were introduced as a potential diagnostic aid. In a second experiment, the number of computer-controlled ships on the screen also varied. Smaller set size improved detection performance. History trails also consistently improved detection performance for both behaviors, although still falling well short of optimal, even with the smaller set size. These findings suggest that working memory plays a critical role in performance on this dynamic decision making task, and the constraints of working memory capacity can be decreased through a simple visual aid and an overall reduction in the number of objects being tracked. The implications for the detection of hostile intentions are discussed.

Modeling Task Scheduling Decisions of Emergency Department Physicians

OBJECTIVE

This study evaluated task-scheduling decisions in the context of emergency departments by comparing patterns of emergency physicians' task-scheduling models across levels of experience.

BACKGROUND

Task attributes (priority, difficulty, salience, and engagement) influence task-scheduling decisions. However, it is unclear how attributes interact to affect decisions, especially in complex contexts. An existing model of task scheduling, strategic task overload management-no priority (STOM-NP), found that an equal weighting of attributes can predict task-scheduling behavior. Alternatively, mathematical modeling estimated that priority alone could make similar predictions as STOM-NP in a parsimonious manner. Experience level may also influence scheduling decisions.

METHOD

An experimental design methodology shortened a judgment analysis approach to compare a priori task-scheduling decision strategies. Emergency physicians with two levels of experience rank-ordered 10 sets of 3 tasks varying on 4 task attributes in this complex environment.

RESULTS

Bayesian statistics were used to identify best-fit decision strategies. STOM-NP and priority-only provided the best model fits. STOM-NP fit the lower-experienced physicians best, whereas priority-only-using only one cue-fit the higher-experienced physicians best.

CONCLUSION

Models of decision strategies for task-scheduling decisions were extended to complex environments. Experts' level of experience influenced task-scheduling decisions, where the scheduling decisions of more-experienced experts was consistent with a more frugal decision process. Findings have implications for training and evaluation.

APPLICATION

We assessed models of cues that influence task-scheduling decisions, including a parsimonious model for task priority only. We provided a sample approach for shortening methods for understanding decisions.

The Planning Bias in the Spatial Rendezvous: Partial Compensation for Temporal Uncertainty

Rendezvous behaviors involve meeting at a time and place, therefore requiring accurate prediction and planning of both your own and the target trajectory. Previous work suggests that the planning bias occurs in spatial situations, with control decisions in a basic ship simulation producing a general tendency towards late rendezvous arrivals. The current experiment introduced temporal uncertainty, operationalized through intermittent delays in the trajectory of the participant’s ship on only some trials. Most trends found under certainty, such as preference for dual axis control, were replicated. Compensation for the possible delays resulted in improved accuracy overall, although not enough to overcome the delays themselves; late arrivals were still prevalent on those trials. Experiencing uncertainty therefore influences the planning bias, and the inclusion of some unpredictable events may counterintuitively improve overall performance. The implications for real-world planning and decision making are discussed.

Nautical Collision Avoidance : The Cognitive Challenges of Balancing Safety, Efficiency, and Procedures

OBJECTIVE

Experimentally investigate maneuver decision preferences in navigating ships to avoid a collision. How is safety (collision avoidance) balanced against efficiency (deviation from path and delay) and rules of the road under conditions of both trajectory certainty and uncertainty.

BACKGROUND

Human decision error is a prominent factor in nautical collisions, but the multiple factors of geometry of collisions and role of uncertainty have been little studied in empirical human factors literature.

APPROACH

Eighty-seven Mechanical Turk participants performed in a lower fidelity ship control simulation, depicting ownship and a cargo ship hazard on collision or near-collision trajectories of various conflict geometries, while controlling heading and speed with the sluggish relative dynamics. Experiment 1 involved the hazard on a straight trajectory. In Experiment 2, the hazard could turn on unpredictable trials. Participants were rewarded for efficiency and penalized for collisions or close passes.

RESULTS

Participants made few collisions, but did so more often when on a collision path. They sometimes violated the instructed rules of the road by maneuvering in front of the hazard ship's path. They preferred speed control to heading control. Performance degraded in conditions of uncertainty.

CONCLUSION

Data reveal an understanding of maneuver decisions and conditions that affect the balance between safety and efficiency.

APPLICATION

The simulation and data highlight the degrading role of uncertainty and provide a foundation upon which more complex questions can be asked, asked of more trained navigators, and decision support tools examined.

Transitions Between Low and High Levels of Mental Workload can Improve Multitasking Performance

OCCUPATIONAL APPLICATIONS Complex and dynamic environments including military operations, healthcare, aviation, and driving require operators to transition seamlessly between levels of mental workload. However, little is known about how the rate of an increase in workload impacts multitasking performance, especially in the context of real-world tasks. We evaluated both gradual and sudden workload increases in the dynamic multitasking environment of an Unmanned Aerial Vehicle (UAV) command and control testbed and compared them to constant workload. Workload transitions were found to improve response time and accuracy compared to when workload was held constant at low or high. These results suggest that workload transitions may allow operators to better regulate mental resources. These findings can also inform the design of operations and technology to assist operators' management of cognitive resources, which include negating the adverse effects of vigilance decrements during low workload periods and data overload during high workload periods.

Absence of DOA Effect but No Proper Test of the Lumberjack Effect: A Reply to Jamieson and Skraaning (2019)

OBJECTIVE

The aim was to evaluate the relevance of the critique offered by Jamieson and Skraaning (2019) regarding the applicability of the lumberjack effect of human-automation interaction to complex real-world settings.

BACKGROUND

The lumberjack effect, based upon a meta-analysis, identifies the consequences of a higher degree of automation-to improve performance and reduce workload-when automation functions as intended, but to degrade performance more, as mediated by a loss of situation awareness (SA) when automation fails. Jamieson and Skraaning provide data from a process control scenario that they assert contradicts the effect.

APPROACH

We analyzed key aspects of their simulation, measures, and results which we argue limit the strength of their conclusion that the lumberjack effect is not applicable to complex real-world systems.

RESULTS

Our analysis revealed limits in their inappropriate choice of automation, the lack of a routine performance measure, support for the lumberjack effect that was actually provided by subjective measures of the operators, an inappropriate assessment of SA, and a possible limitation of statistical power.

CONCLUSION

We regard these limitations as reasons to temper the strong conclusions drawn by the authors, of no applicability of the lumberjack effect to complex environments. Their findings should be used as an impetus for conducting further research on human-automation interaction in these domains.

APPLICATIONS

The collective findings of both Jamieson and Skraaning and our study are applicable to system designers and users in deciding upon the appropriate level of automation to deploy.

Foundational Elements of a Rendezvous: When (and Where) Planning & Prediction Meet

For one entity to initiate a meeting, interception, or rendezvous with another, such situations necessitate both accurate perception of the target’s trajectory and also planning of one’s own trajectory. To examine these cognitive processes and resulting performance trends, the current study presented participants with simplified simulated ship rendezvous scenarios in which they had to predict a target ship’s future course while planning their own actions to facilitate a meeting. Rendezvous error increased under high time pressure, either when the ships started by moving towards each other, or when the target ship moved faster than the user’s ship. When participants missed a rendezvous, they often did so by passing behind the target ship as if maneuvering too late. Participants also preferred to adjust both the heading and the speed of their ship when maneuvering. Finally, users took longer to assess situations characterized by high time pressure.

Anchoring and Adjustment in Uncertain Spatial Trajectory Prediction

OBJECTIVE

The aim of this study was to explore the impact of prior information on spatial prediction and understanding of variability.

BACKGROUND

In uncertain spatial prediction tasks, such as hurricane forecasting or planning search-and-rescue operations, decision makers must consider the most likely case and the distribution of possible outcomes. Base performance on these tasks is varied (and in the case of understanding the distribution, often poor). Humans must update mental models and predictions with new information, sometimes under cognitive workload.

METHOD

In a spatial-trajectory prediction task, participants were anchored on accurate or inaccurate information, or not anchored, regarding the future behavior of an object (both average behavior and the variability). Subsequently, they predicted an object's future location and estimated its likelihood at multiple locations. In a second experiment, participants repeated the process under varying levels of external cognitive workload.

RESULTS

Anchoring influenced understanding of most likely predicted location, with fairly rapid adjustment following inaccurate anchors. Increasing workload resulted in decreased overall performance and an impact on the adjustment component of the task. Overconfidence was present in all conditions.

CONCLUSION

Prior information exerted short-term influence on spatial predictions. Cognitive load impaired users' ability to effectively adjust to new information. Accurate graphical anchors did not improve user understanding of variability.

APPLICATION

Prior briefings or forecasts about spatiotemporal trajectories affect decisions even in the face of initial contradictory information. To best support spatial prediction tasks, efforts also need to be made to separate extraneous load-causing tasks from the process of integrating new information. Implications are discussed.

Task Switching in Rock Climbing: Validation of a Computational Model for Different Skill Levels

Thirty-two rock climbers, all self-identifying as capable of lead climbing (place protective gear in the rock, to mitigate the risks of falling), climbed an outdoor route while placing what they believed was the necessary amount of protection. Cameras recorded the percentage of time they spent climbing upward (productivity) relative to placing protection (safety). We then applied STOM (strategic task overload model) to predict percent time-on-task, using the differences in their ratings of task interest, task priority, and task difficulty as predictors. The model significantly predicted time on task for the participants categorized as experts, but not for those categorized as non-experts. Time on the climbing (versus protection) task for the expert group, but not the non-expert group, was also predicted by a derived measure inferred to assess risk tolerance in climbing.

Calibrating Uncertainty: Commonalities in the Estimation of Numeric Variability Versus Spatial Prediction

To assess whether there may be a common ability related to the understanding and calibration of instance variability and mean behavior, participants performed spatial prediction and numeric estimation tasks. In the first task, participants experienced variability in a set of spatial trajectories whose endpoints they predicted along with a central mean. In the second task, they experienced variability in a set of random numbers whose mean and variability they estimated. For both tasks, estimated variability was compared with the true variability of instances to derive measures of bias (e.g., over-or under-estimation) and precision. Correlations between these estimates across the two experiments revealed mixed evidence for a common ability to estimate variability, but suggested similar performance when estimating mean behavior. Implications for individual differences and interventions are discussed.

Assessing Dynamic Value for Safety Gear During a Rock Climbing Task

Value is a component of the SEEV model of attention combined with measures of salience, effort, and expectancy to characterize areas of interest in a visual workspace. In the current project, an assessment of dynamic value was created by prompting rock climbers to assess their post-hoc priority for placing safety gear at various points during a climb when the risk of injury from a fall varied. Analyses determined that, for expert and non-expert rock climbers, gear placement priority rose as time since the most recent gear placement increased, after controlling for climbing speed. Prediction of the attentional value of a target via dynamic estimates may be applied in other highly dynamic environments.

Attention Does Not Improve Impaired Understanding Of Variability In Spatial Prediction

Understanding variability of uncertain systems is often vital for decision makers, yet is habitually disregarded in favor of developing superior capability to predict most likely outcomes. One potential path to improving appreciation of variability is simply to attend more carefully to it. The present study explores a trade-off in the ability to predict average trajectories and estimate the variability in a spatial prediction task. Through instructional and task manipulations, some participants were encouraged to emphasize variability in a task that had previously shown good performance of mean prediction but poor estimation of variability. Overall estimation of variability was poor in both the prediction emphasis condition, and crucially in the variability emphasis condition. The results suggest overlooking variability is not just the result of neglecting that dimension, but rather represents a more systematic limitation in human performance. The implications of these findings are discussed.

Effect of Visualization Training on Uncertain Spatial Trajectory Predictions

OBJECTIVE

The goal of this study was to explore the ways in which visualizations influence the prediction of uncertain spatial trajectories (e.g., the unknown path of a downed aircraft or future path of a hurricane) and participant overconfidence in such prediction.

BACKGROUND

Previous research indicated that spatial predictions of uncertain trajectories are challenging and are often associated with overconfidence. Introducing a visualization aid during training may improve the understanding of uncertainty and reduce overconfidence.

METHOD

Two experiments asked participants to predict the location of various trajectories at a future time. Mean and variance estimates were compared for participants who were provided with a visualization and those who were not.

RESULTS

In Experiment 1, participants exhibited less error in mean estimations when a linear visualization was present but performed worse than controls once the visualization was removed. Similar results were shown in Experiment 2, with a nonlinear visualization. However, in both experiments, participants who were provided with a visualization did not retain any advantage in their variance estimations once the visualization was removed.

CONCLUSIONS

Visualizations may support spatial predictions under uncertainty, but they are associated with benefits and costs for the underlying knowledge being developed.

APPLICATION

Visualizations have the potential to influence how people make spatial predictions in the presence of uncertainty. Properly designed and implemented visualizations may help mitigate the cognitive biases related to such predictions.

Experimental Evaluation of STOM in a Business Setting

This study investigated the effect of autonomous prioritization on attention in a high-workload environment. It explored the “priority mystery,” or neglect of priority in switching behavior, found in previous Strategic Task Overload Model (STOM) experiments (Wickens Gutzwiller & Santamaria, 2015; Raby & Wickens 1994). We hypothesized that participants who could choose their own prioritization of tasks would view their prioritized task as more attractive and allocate more time and attention than participants who were not able to autonomously prioritize the tasks. Ninety-three participants were told to choose their own priority, given a priority, or not given any direction on priority of tasks. Participants had thirty minutes to complete two computer-based tasks and then were asked to rate the priority, interest, difficulty, and salience of each task. Priority influenced time on task most for the self-chosen priority group and not at all for the group given no priority direction. The more difficult task was also chosen less frequently. The implications and limitations of this study are discussed.

The Status of the Strategic Task Overload Model (STOM) for Predicting Multi-Task Management

A model for task switching which focuses on the decision making of operators in overloaded multitask conditions is reviewed and new research presented. The STOM model is an ongoing effort and as such, work is now accumulating, which serves to validate the model as a useful predictive method, but also is uncovering uncertainties that require further investigation. Here we summarize the origins of the model, which was informed by past modeling efforts, a literature review and a meta-analysis. We then describe in detail the basic parameters of STOM and the current status of each, before discussing future directions and six uncertainties uncovered when building our understanding of task switching choice.

Effect of Visualization on Spatial Trajectory Prediction under Uncertainty

Spatial predictions of uncertain trajectories are challenging, but are often associated with overconfidence. This study explored how a visualization influenced prediction of uncertain spatial trajectories (e.g., unknown path of a downed aircraft or future path of a hurricane). Mean and variance estimates were compared for participants provided with a gradient-shaded “cone of uncertainty” visualization and those who were not provided with a visualization. Participants exhibited less error in mean estimations when a visualization was present, but performed worse than controls once the visualization was removed. For variance estimations, participants provided with a visualization did not retain any advantage in their estimations once the visualization was removed. Combined these findings suggest that visualizations may support some aspects of spatial predictions under uncertainty, but they can be associated with costs for the underlying knowledge being developed.

Implementing Lumberjacks and Black Swans Into Model-Based Tools to Support Human-Automation Interaction

OBJECTIVE

The objectives were to (a) implement theoretical perspectives regarding human-automation interaction (HAI) into model-based tools to assist designers in developing systems that support effective performance and (b) conduct validations to assess the ability of the models to predict operator performance.

BACKGROUND

Two key concepts in HAI, the lumberjack analogy and black swan events, have been studied extensively. The lumberjack analogy describes the effects of imperfect automation on operator performance. In routine operations, an increased degree of automation supports performance, but in failure conditions, increased automation results in more significantly impaired performance. Black swans are the rare and unexpected failures of imperfect automation.

METHOD

The lumberjack analogy and black swan concepts have been implemented into three model-based tools that predict operator performance in different systems. These tools include a flight management system, a remotely controlled robotic arm, and an environmental process control system.

RESULTS

Each modeling effort included a corresponding validation. In one validation, the software tool was used to compare three flight management system designs, which were ranked in the same order as predicted by subject matter experts. The second validation compared model-predicted operator complacency with empirical performance in the same conditions. The third validation compared model-predicted and empirically determined time to detect and repair faults in four automation conditions.

CONCLUSION

The three model-based tools offer useful ways to predict operator performance in complex systems.

APPLICATION

The three tools offer ways to predict the effects of different automation designs on operator performance.

Circadian Effects on Simple Components of Complex Task Performance

The goal of this study was to advance understanding and prediction of the impact of circadian rhythm on aspects of complex task performance during unexpected automation failures, and subsequent fault management. Participants trained on two tasks: a process control simulation, featuring automated support; and a multi-tasking platform. Participants then completed one task in a very early morning (circadian night) session, and the other during a late afternoon (circadian day) session. Small effects of time of day were seen on simple components of task performance, but impacts on more demanding components, such as those that occur following an automation failure, were muted relative to previous studies where circadian rhythm was compounded with sleep deprivation and fatigue. Circadian low participants engaged in compensatory strategies, rather than passively monitoring the automation. The findings and implications are discussed in the context of a model that includes the effects of sleep and fatigue factors.

The Role of Individual Differences in Executive Attentional Networks and Switching Choices in Multi-Task Management

Individual differences in cognitive processing relate to critical performance differences in real-world environments. Task switching is required for many of them and especially for task management during overload. Research exploring individual differences related to switching behavior (both frequency, and adherence to “optimal” switch times) is, however, sparse. We examined these relationships here, using the attentional network task to index executive control, and an ongoing tracking task (within a larger suite of concurrent task demands) to examine switching behavior. The results failed to support a general relationship between executive control and frequency in a complex, heterogeneous multi-task environment. However, higher executive control participants more successfully exploited “optimal” switching times, highlighting the varying role of individual differences in task management, when choice is unconstrained.

Mapping Spatial Uncertainty in Prediction

Past research in a range of domains has suggested that individuals tend to underestimate the contribution of random factors in predicting the future and therefore have difficulties understanding variance. This study explored understanding of central tendency versus understanding of underlying variance in a novel spatial task. Participants were required to predict the most likely outcome of an object’s somewhat uncertain movement, as well as estimate its probability of it being present at a number of potential locations. Our findings indicate that individuals became adept at predicting the target location, but they were relatively insensitive to the change variability in the underlying distribution and vastly overestimated the likelihood of samples around the mean. Additionally, there appeared to be no strong relationship between performance on prediction of the mean and performance on estimation of the likelihood. These findings highlight the need for interventions to improve human understanding of variability in order to facilitate prediction in real-world settings.

The Event-Related Brain Potential as an Index of Attention Allocation in Complex Displays

The advantages of employing the event-related brain potential (ERP) in the assessment of allocation of attention in dynamic environments are discussed. Three experiments are presented in which the P300 component of the ERP is demonstrated to be a useful index of subjects' locus of attention. The first two experiments were concerned with the allocation of atttention during discrete and continuous visual monitoring tasks. The results indicated that a P300 was elicited only by stimuli to which the subject had to attend in order to perform successfully the task. The third experiment was conducted to assess the sensitivity of P300 to the manner in which attention is allocated to different aspects of a display during the performance of a 3-dimensional target acquistion task. The amplitude of the P300 was found to reflect differences between two levels of workload, as well as the task relevance of the stimuli. The results of the experiments are discussed in terms of their utility in the evaluation of the design of man-machine systems as well as in the study of the allocation of attention in operational environments.

The Event-Related Brain Potential as a Selective Index of Display Load

As an index of task workload, the possible advantages of the event-related brain potential (ERP) over traditional secondary task techniques and psychophysiological measures are described and previous efforts to validate the use of ERPs in this context are discussed. A series of six experiments is then reported in which the effects of tracking difficulty and display-monitoring load manipulations on secondary-task reaction time and ERP measures are compared. The results are interpreted within the framework of a “multiple reservoir” conception of processing resources and task workload.

Eye-tracking and Individual Differences in off-Normal Event Detection when Flying with a Synthetic Vision System Display

Eye-tracking data were collected on eight pilots who flew a high fidelity flight simulator with a synthetic vision system (SVS) display, which included a tunnel-in-the-sky in half of the experimental conditions. In this paper, we examined the individual pilots' detection and response to two unexpected events, where the true information was inaccurately (or not) represented on the SVS display. The results indicate that the pilots who failed to detect these unexpected events spent most of their time scanning the SVS display, which included the tunnel, and rarely scanned the outside world. The pilot who detected the unexpected events showed scan patterns with a more even distribution of scans to the SVS, the outside world, and other displays even when the tunnel was present. This suggests that detection of unexpected in-the-world events depends on the individual pilot's vigilance in maintaining a regular scan pattern and resistance to “cognitively tunneling” into only one display.

Does Workload Modulate the Difference between Cockpit Traffic Display Formats?

Eighteen certified flight instructors from the University of Illinois Institute of Aviation participated in an experiment exploring the format of the Cockpit Display of Traffic Information for free flight traffic avoidance maneuvers. Pilots flew a sequence of flight scenarios to compare the effects of traffic load, display dimensionality (3D vs. 2D coplanar), and a vertical profile orientation on maneuver choice, conflict avoidance performance, and maneuver efficiency. The highest levels of workload induced more combined lateral/vertical maneuvers, degraded safety on the 2D coplanar displays, and degraded efficiency regardless of display type. In the context of an overwhelming preference for vertical maneuvers, the 3D display increased the frequency of the less-safe descent maneuvers (relative to climbs) and increased subjective workload, while the 2D rear-view display decreased the vertical efficiency of all maneuvers, relative to its side-view counterpart.

Measuring Traffic Awareness in an Integrated Hazard Display

The ability for pilots to estimate traffic location, and how such estimations should be measured, was examined within an Integrated Hazard Display context. Twelve pilots viewed static images of traffic scenarios and then estimated the outside world locations of queried traffic represented in one of three display types (2D coplanar, 3D exocentric, and split-screen) and in one of four conditions (display present/blank crossed with outside world present/blank). Overall, the 2D coplanar display best supported both vertical (compared to 3D) and lateral (compared to split-screen) traffic position estimation performance. Furthermore, although pilots were faster in estimating traffic locations when the display was blank, accuracy was greatest when both the display and outside world were available.

Unmanned Aerial Vehicle Flight Control: False Alarms versus Misses

Thirty-two undergraduate pilots from the University of Illinois School of Aviation performed simulated military reconnaissance missions with an unmanned aerial vehicle (UAV). Pilots were required to: a) navigate the UAV through a series of mission legs, b) search for possible targets of opportunity, and c) monitor system health. They performed the missions under three types of auditory auto-alert aids (a 100% reliable system, a 67% reliable system with automation false alarms, and a 67% reliable system with automation misses), as well as a non-automated baseline condition. Results indicate that while reliable automation can benefit performance relative to baseline in the automated task, the unreliable automation aids reduced performance to that of baseline or worse. The automation false alarms and misses harmed performance in qualitatively different ways, with false alarm prone automation appearing to cause more damage than miss prone automation to both the automated task and the concurrent target search task.

Conformal Symbology, Attention Shifts, and the Head-Up Display

Thirty-two pilots flew instrument approaches in a high-fidelity simulator. Location of flight symbology was manipulated head-up vs. head-down while controlling for optical distance and symbology format. Pilots were assigned to one of two symbology sets, conformal and non-conformal. Each pilot flew half of the trials with the symbology presented in a head-up location and half with the symbology located head-down. Airspeed tracking for both groups was displayed with non-conformal digital symbology. An unexpected far domain event was presented on one trial per pilot. The results revealed that, for flight path control, there was generally a cost associated with head-down location. The magnitude of this cost was larger for conformal than for non-conformal symbology. Head-up presentation resulted in faster transition from instrument to visual flight reference, but poorer airspeed tracking and slower response to the far domain unexpected event and greater error tracking digital airspeed. The results are interpreted with the theoretical framework of object-based and space-based theories of visual attention, and the tradeoffs between clutter of overlapping imagery and information access cost.

The When and How of Using 2-D and 3-D Displays for Operational Tasks

Three different cannonical viewpoints into a 3D domain are defined to create a taxonomy of 3D displays. We then show how the information processing demands of each display viewpoint, provides benefits and or imposes costs on four categories of tasks, involving travel, image matching or situation awareness, visual search, and precise judgments. These task-display interactions are illustrated from experiments in aviation display design, battlefield judgments, and data visualization. Conclusions are offered regarding two possible ways of addressing the task-display interactions in design.

Egocentric and Exocentric Displays for Terminal Area Navigation

Two experiments are reported that contrast egocentric vs. exocentric features of perspective (3D) or plan view (2D) electronic map displays for supporting local guidance and global awareness. Pilots used these displays for a simulated approach to a landing along a curved approach, through a terrain-filled region. The task was simulated on an IRIS visual graphics workstation. In Experiment 1, a rotating vs. fixed-map display was experimentally crossed with a 2D vs. 3D (perspective map) view as 24 pilots were assessed in their ability to maintain the flight path (local guidance) and demonstrate global awareness of surrounding terrain features. Rotating displays supported better flight path guidance in both the lateral and vertical axes, and did not substantially harm performance on the task of recalling the location of terrain features. Map rotation also supported better performance in locating features from an ego-referenced but not a world-referenced perspective. 3D displays provided a slight advantage for lateral guidance but a substantial cost for vertical control, because of the ambiguity with which perspective viewing depicted precise altitude. In Experiment 2, 10 pilots flew with the rotating 2D display, and with an improved version of the rotating 3D display, using color coding to reduce the ambiguity of altitude information. Vertical control improved as a result of the 3D display design improvement, but lateral control did not. Assessment of terrain awareness on a map reconstruction task revealed marginally better performance with the 2D map. The results are discussed in terms of the costs and benefits of presenting information in 3D, ego-referenced format for both local guidance and global awareness tasks.

The Implications of Data-Link for Representing Pilot Request Information on 2D and 3D Air Traffic Control Displays

Twenty-four subjects (seven ATC specialists and 17 pilots trained in fundamental ATC skills), performed a simulation in which they were required to evaluate pilot requests for flight plan changes, issued by aircraft depicted on their display. Some requests could be safely granted, whereas others would bring about a mid-air conflict with other aircraft. Subjects evaluated the requests in the context of an airspace depicted on either a conventional 2D planar display or a 3D perspective display. Requests were presented either as voice messages or were displayed visually, as if relayed via data-link, either printed at the bottom of the display or represented as vectors, emanating in the requested direction from the requesting aircraft. The results indicated that performance was generally equivalent with the 2D and the 3D display and was best with the auditory-verbal request mode. It was considerably slower with the print mode, presumably because of the greater scanning required. The spatial vector mode offered performance that was faster than print, but considerably less accurate. Particular deficiencies were noticed with the vector mode when it was used to present complex 3-dimensional requests in the context of the 3-dimensional display.

Costs and Benefits of Head-Up Display Use: A Meta-Analytic Approach

Costs and benefits associated with head-up display (HUD) use were assessed by analyzing data obtained from studies in the aviation domain. Eighteen studies investigating the use of HUDs, head-down displays, and conformal displays were included in analyses of flight path tracking and event detection performance. HUD and conformality benefits consisted of faster detection responses to changes in symbology and presentation of traffic, as well as increased flight path tracking accuracy. Further investigation into the HUD data revealed a heterogeneous component, suggesting that factors other than display location were contributing to pilot performance. One factor modulating performance was the relative expectancy of the event. HUD use was advantageous if an event in the environment was expected. However, when an event was unexpected, detection was degraded with the presence of HUD instrumentation. This finding reveals a potential cost due to HUD use when confronted with anomalous situations.

Pilot Performance and Workload Using Traffic Displays to Support Free Flight

Arising from the need to employ innovative solutions to safely and efficiently maintain air traffic separation in increasingly denser skyways, the concept of Free Flight involves shifting some air traffic management responsibilities from air traffic control specialists on the ground, to pilots in the cockpit. Such a shift in traffic management responsibilities will be critically dependent upon the development of displays to provide traffic and hazard information to pilots in the cockpit (Wickens, Carbonari, Merwin, Morphew, & O'Brien (1997; Battiste (in progress); Johnson, Battiste, Delzell, Holland, Belcher, & Jordan, 1997). This research examined the effect of different information-varying display aids (predictors and threat vectors) for in-cockpit traffic displays, on pilot performance, workload, attentional demands, and flight safety. Fifteen pilots flew a series of traffic avoidance scenarios in a Free Flight simulation designed to assess the effects of different levels of traffic display information on these pilot variables. Three, 2D-coplanar prototype displays were compared which differed in the level of traffic information provided. Analysis of the data revealed that the traffic display with the most predictive information supported increased safety and decreased workload, without appreciable decrements in flight performance or efficiency.

Visual Analysis of Scientific Data: Comparison of 3D-Topographic, Color and Gray Scale Displays in a Feature Detection Task

Several display techniques were compared for representing scientific data in the context of a feature detection task. The data sets were rendered on a Silicon Graphics workstation using four display formats: linearized gray scale; rainbow scale; reduced hue (blue-green-yellow-white) scale; and a 3D-topographic format viewed in stereo. The task involved searching for features that were embedded in scientific data sets consisting of two spatial and one scalar variable. Data sets were drawn from three scientific domains: Landsat, medical MRI, and global atmospheric data bases. Two types of features were embedded within the data sets: circular (blob-like) discontinuities, and linear (cliff-like) discontinuities. Results showed a general advantage for the gray scale, and a marked disadvantage for the 3D-topographic format in both accuracy and response latency. Performance in the two color scale formats was intermediate, with the reduced hue scale supporting faster, if not more accurate performance than the full rainbow scale. Performance differences were found across data base domains, as well. Directions for future research are discussed.

Unreliable Automated Attention Cueing for Air-Ground Targeting and Traffic Maneuvering

We report two experiments in which pilots' attention is occasionally directed to inappropriate or inaccurate locations in space, replicating the effects of imperfect automation. A general taxonomy of human performance costs in these situations is presented. In Experiment 1, pilots are engaged in an air-ground targeting scenario. Target cueing, based upon semi-reliable sensor information, sometimes directs attention away from the true target. Yet pilots follow such guidance, even knowing its unreliability, a result of the difficulty of the unaided task. In Experiment 2, pilots in a free flight simulation are engaged in a series of traffic conflict avoidance maneuvers, using a cockpit display of traffic information (CDTI). On rare trials the CDTI knowledge of the traffic intruder's intentions, reflected in a predictor symbol, is unreliable and does not correspond with the actual aircraft behavior. Yet pilots' avoidance behavior is governed by the predictor symbol, and a display manipulation that calls attention to the inaccuracy of the predictor does little to influence pilots' reliance upon the predictor symbol although it does reduce visual workload. The data are interpreted in terms of appropriate trust calibration.

Decision and Workload Implications of Free Flight and the Cockpit Display of Traffic Information (Cdti)

Twenty-two certified flight instructors flew a general aviation simulator, either actively selecting and flying maneuvers to avoid traffic conflict, using a cockpit display of traffic information (CDTI) in a free flight scenario, or more passively executing corresponding maneuvers as commanded by a simulated air traffic controller, in a conventional scenario. The maneuvers spontaneously chosen by the pilots in the freeflight scenario were found to be only partially consistent with the maneuvers for traffic avoidance, dictated by current FAA “rules of the road”, with pilots showing a marked preference for making vertical rather than lateral maneuvers, and climbing rather than descending maneuvers. Pilots also preferred to make simpler single axis maneuvers. Visual scanning in both scenarios was measured, revealing that pilots spent a majority of their time fixating on the instrument panel, and in free flight, they distributed their remaining visual attention differentially across the two sources of traffic information, the outside world and the CDTI. In all conditions, they were particularly vulnerable to missing the detection of traffic which was not annunciated on the CDTI, but was visible on the forward view.

Focal and Ambient Visual Contributions and Driver Visual Scanning in Lane Keeping and Hazard Detection

In some previous simulator studies, we showed that drivers, engaged in a secondary in-vehicle task at different display separations, were able to buffer some aspects of the driving task (e.g., vehicle control), but not others (e.g., hazard response; Horrey & Wickens, 2002, Horrey, Alexander, & Wickens, 2003). We speculate that this pattern of results may be a result of effective time-sharing between the ambient visual task of vehicle control and the focal visual in-vehicle task, but task interference between the in-vehicle task and the focal task of hazard detection and identification. Alternatively, drivers may be using an optimal scanning strategy that allows them to effectively monitor both the roadway and the in-vehicle device. In this paper, we detail some of the theoretical background, describe some relevant research, and present a model of scanning behavior.

Graph-Task Dependencies in Three-Dimensional Data: Influence of Three-Dimensionality and Color

This study compared conventional 2D graphs with 3D graphs and color based graphs for presenting 3-dimensional data. These data were in the format representing the effects on Y, of 4 levels of X and 4 levels of Z. Z was represented by a parameter (line type) in the 2D display, by space (depth) in the 3D display, and Y was represented by color in the color display. Subjects answered questions about the displayed data that varied in the degree to which they required focused attention on a single data point, to integration across the entire data space. The results indicated that the 3D display supported slowest and least accurate performance for the focused attention questions, a cost that dissipated when the questions became more integrative. Performance with the color display suffered badly in both speed and accuracy with the most integrative questions. The 2D display performed consistently well in both speed and accuracy. The results are consistent with prior data and with emerging theoretical perspectives on graph-task dependencies.