Impact Mapping for Geospatial Reasoning and Decision Making

OBJECTIVE

The reported study evaluated a novel approach to aiding geospatial reasoning and decision making.

BACKGROUND

Impact mapping aims to alleviate the cognitive demands of geospatial tasks in part by externalizing data in the form of an integrated decision surface. This is achieved by aggregating data across multiple sources of information and visualizing their combined utility rather than objective measurements or individual utility. Previous research has shown that geospatial decisions improve when aided in this manner, but it remains unknown if dynamic decision making, often plagued by fatigue and anchoring bias, would benefit similarly.

METHOD

The experiment implemented a systematic manipulation of the presence of a composite impact map and the number of attributes present in a two-stage disaster relief, resource allocation task to investigate when and how impact mapping is beneficial or deleterious to decision makers.

RESULTS

The presence of the composite impact map increased the utility of selected sites, increased re-planning decisions, reduced information display views, and reduced workload. Generally, the effect of the composite impact map was greater when participants were asked to evaluate more attributes.

CONCLUSION

Composite impact maps appear to improve repeated geospatial reasoning and minimize anchoring bias because they alleviate the cognitive demands otherwise necessary to interpret and maintain information from multiple attributes.

APPLICATION

Data visualization techniques, such as impact mapping, can improve repeated geospatial decision making in environments that include high cognitive demand.

Metrics for Human-Robot Team Design: A Teamwork Perspective on Evaluation of Human-Robot Teams

Metrics for human-robot teaming should extend to teams consisting of multiple human and robotic agents, and to teams working in complex, dynamic work domains. This work proposes that to comprehensively analyze and evaluate multi-human, multi-robot teams, traditional HRI metrics of performance and efficiency must be expanded upon to incorporate metrics of teamwork. We develop five distinct metrics to capture both ecological and cognitive aspects of teamwork found to be important in human-automation interaction, inspired by research in the cognitive systems engineering community. We demonstrate the application of these metrics in a spacecraft maintenance case study comparing multiple human-robot team architectures. The case study demonstrates that the teamwork metrics capture aspects of human-robot interaction not apparent when using only traditional performance and efficiency metrics. The paper concludes that the proposed teamwork metrics are complementary to existing metrics in HRI and should be included in the evaluation of human-robot teams.

Interaction Algorithm Effect on Human Experience with Reinforcement Learning

A goal of interactive machine learning (IML) is to enable people with no specialized training to intuitively teach intelligent agents how to perform tasks. Toward achieving that goal, we are studying how the design of the interaction method for a Bayesian Q-Learning algorithm impacts aspects of the human’s experience of teaching the agent using human-centric metrics such as frustration in addition to traditional ML performance metrics. This study investigated two methods of natural language instruction: critique and action advice. We conducted a human-in-the-loop experiment in which people trained two agents with different teaching methods but, unknown to each participant, the same underlying reinforcement learning algorithm. The results show an agent that learns from action advice creates a better user experience compared to an agent that learns from binary critique in terms of frustration, perceived performance, transparency, immediacy, and perceived intelligence. We identified nine main characteristics of an IML algorithm’s design that impact the human’s experience with the agent, including using human instructions about the future, compliance with input, empowerment, transparency, immediacy, a deterministic interaction, the complexity of the instructions, accuracy of the speech recognition software, and the robust and flexible nature of the interaction algorithm.

Decision Support System Requirements Definition for Human Extravehicular Activity Based on Cognitive Work Analysis

The design and adoption of decision support systems within complex work domains is a challenge for cognitive systems engineering (CSE) practitioners, particularly at the onset of project development. This article presents an example of applying CSE techniques to derive design requirements compatible with traditional systems engineering to guide decision support system development. Specifically, it demonstrates the requirements derivation process based on cognitive work analysis for a subset of human spaceflight operations known as extravehicular activity. The results are presented in two phases. First, a work domain analysis revealed a comprehensive set of work functions and constraints that exist in the extravehicular activity work domain. Second, a control task analysis was performed on a subset of the work functions identified by the work domain analysis to articulate the translation of subject matter states of knowledge to high-level decision support system requirements. This work emphasizes an incremental requirements specification process as a critical component of CSE analyses to better situate CSE perspectives within the early phases of traditional systems engineering design.

Exploring relationships of human-automation interaction consequences on pilots: uncovering subsystems

OBJECTIVE

We attempted to understand the latent structure underlying the systems pilots use to operate in situations involving human-automation interaction (HAI).

BACKGROUND

HAI is an important characteristic of many modern work situations. Of course, the cognitive subsystems are not immediately apparent by observing a functioning system, but correlations between variables may reveal important relations.

METHOD

The current report examined pilot judgments of 11 HAI dimensions (e.g., Workload, Task Management, Stress/Nervousness, Monitoring Automation, and Cross-Checking Automation) across 48 scenarios that required airline pilots to interact with automation on the flight deck.

RESULTS

We found three major clusters of the dimensions identifying subsystems on the flight deck: a workload subsystem, a management subsystem, and an awareness subsystem.

DISCUSSION

Relationships characterized by simple correlations cohered in ways that suggested underlying subsystems consistent with those that had previously been theorized.

APPLICATION

Understanding the relationship among dimensions affecting HAI is an important aspect in determining how a new piece of automation designed to affect one dimension will affect other dimensions as well.

Toward a characterization of adaptive systems: a framework for researchers and system designers

OBJECTIVE

This article presents a systematic framework characterizing adaptive systems.

BACKGROUND

Adaptive systems are those that can appropriately modify their behavior to fit the current context. This concept is appealing because it offers the possibility of creating computer assistants that behave like good human assistants who can provide what is needed without being asked. However, the majority of adaptive systems have been experimental rather than practical because of the technical challenges in accurately perceiving and interpreting users' current cognitive state; integrating cognitive state, environment, and task information; and using it to predict users' current needs. The authors anticipate that recent developments in neurological and physiological sensors to identify users' cognitive state will increase interest in adaptive systems research and practice over the next few years.

METHOD

To inform future efforts in adaptive systems, this work provides an organizing framework for characterizing adaptive systems, identifying considerations and implications, and suggesting future research issues.

RESULTS

A two-part framework is presented that (a) categorizes ways in which adaptive systems can modify their behavior and (b) characterizes trigger mechanisms through which adaptive systems can sense the current situation and decide how to adapt.

CONCLUSION

The framework provided in this article provides a tool for organizing and informing past, present, and future research and development efforts in adaptive systems.