No Effect of Cue Format on Automation Dependence in an Aided Signal Detection Task

Trust with increasing and decreasing reliability

OBJECTIVE

The primary purpose was to determine how trust changes over time when automation reliability increases or decreases. A secondary purpose was to determine how task-specific self-confidence is associated with trust and reliability level.

BACKGROUND

Both overtrust and undertrust can be detrimental to system performance; therefore, the temporal dynamics of trust with changing reliability level need to be explored.

METHOD

Two experiments used a dominant-color identification task, where automation provided a recommendation to users, with the reliability of the recommendation changing over 300 trials. In Experiment 1, two groups of participants interacted with the system: one group started with a 50% reliable system which increased to 100%, while the other used a system that decreased from 100% to 50%. Experiment 2 included a group where automation reliability increased from 70% to 100%.

RESULTS

Trust was initially high in the decreasing group and then declined as reliability level decreased; however, trust also declined in the 50% increasing reliability group. Furthermore, when user self-confidence increased, automation reliability had a greater influence on trust. In Experiment 2, the 70% increasing reliability group showed increased trust in the system.

CONCLUSION

Trust does not always track the reliability of automated systems; in particular, it is difficult for trust to recover once the user has interacted with a low reliability system.

APPLICATIONS

This study provides initial evidence into the dynamics of trust for automation that gets better over time suggesting that users should only start interacting with automation when it is sufficiently reliable.

Simple manipulations of anthropomorphism fail to induce perceptions of humanness or improve trust in an automated agent

Although automation is employed as an aid to human performance, operators often interact with automated decision aids inefficiently. The current study investigated whether anthropomorphic automation would engender higher trust and use, subsequently improving human-automation team performance. Participants performed a multi-element probabilistic signal detection task in which they diagnosed a hypothetical nuclear reactor as in a state of safety or danger. The task was completed unassisted and assisted by a 93%-reliable agent varying in anthropomorphism. Results gave no evidence that participants' perceptions of anthropomorphism differed between conditions. Further, anthropomorphic automation failed to bolster trust and automation-aided performance. Findings suggest that the benefits of anthropomorphism may be limited in some contexts.

The Effect of Informing Participants of the Response Bias of an Automated Target Recognition System on Trust and Reliance Behavior

OBJECTIVE

To determine how changing and informing a user of the false alarm (FA) rate of an automated target recognition (ATR) system affects the user's trust in and reliance on the system and their performance during an underwater mine detection task.

BACKGROUND

ATR systems are designed to operate using a high sensitivity and a liberal decision criterion to reduce the risk of the ATR system missing a target. A high number of FAs in general may lead to a decrease in operator trust and reliance.

METHODS

Participants viewed sonar images and were asked to identify mines in the images. They performed the task without ATR and with ATR at a lower and higher FA rate. The participants were split into two groups-one informed and one uninformed of the changed FA rate. Trust and/or confidence in detecting mines was measured after each block.

RESULTS

When not informed of the FA rate, the FA rate had a significant effect on the participants' response bias. Participants had greater trust in the system and a more consistent response bias when informed of the FA rate. Sensitivity and confidence were not influenced by disclosure of the FA rate but were significantly worse for the high FA rate condition compared with performance without the ATR.

CONCLUSION AND APPLICATION

Informing a user of the FA rate of automation may positively influence the level of trust in and reliance on the aid.

Measuring the Efficiency of Automation-Aided Performance in a Simulated Baggage Screening Task

OBJECTIVE

The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making.

BACKGROUND

Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal.

METHOD

Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid's judgment relative to the raw stimuli, target certainty, and target prevalence.

RESULTS AND CONCLUSION

Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid's diagnosis with a probability of 50%. Performance was similar across experiments.

APPLICATION

Results suggest that human operators' performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.

Adapting to the algorithm: how accuracy comparisons promote the use of a decision aid

In three experiments, we sought to understand when and why people use an algorithm decision aid. Distinct from recent approaches, we explicitly enumerate the algorithm’s accuracy while also providing summary feedback and training that allowed participants to assess their own skills. Our results highlight that such direct performance comparisons between the algorithm and the individual encourages a strategy of selective reliance on the decision aid; individuals ignored the algorithm when the task was easier and relied on the algorithm when the task was harder. Our systematic investigation of summary feedback, training experience, and strategy hint manipulations shows that further opportunities to learn about the algorithm encourage not only increased reliance on the algorithm but also engagement in experimentation and verification of its recommendations. Together, our findings emphasize the decision-maker’s capacity to learn about the algorithm providing insights for how we can improve the use of decision aids.

Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Trust calibration for a human–machine team is the process by which a human adjusts their expectations of the automation’s reliability and trustworthiness; adaptive support for trust calibration is needed to engender appropriate reliance on automation. Herein, we leverage an instance-based learning ACT-R cognitive model of decisions to obtain and rely on an automated assistant for visual search in a UAV interface. This cognitive model matches well with the human predictive power statistics measuring reliance decisions; we obtain from the model an internal estimate of automation reliability that mirrors human subjective ratings. The model is able to predict the effect of various potential disruptions, such as environmental changes or particular classes of adversarial intrusions on human trust in automation. Finally, we consider the use of model predictions to improve automation transparency that account for human cognitive biases in order to optimize the bidirectional interaction between human and machine through supporting trust calibration. The implications of our findings for the design of reliable and trustworthy automation are discussed.

Ironic efficiency in automation-aided signal detection

Decision makers often make poor use of the information provided by an automated signal detection aid; recent studies have found that participants assisted by an automated aid fell well short of best-possible sensitivity levels. The present study tested the generalisability of this finding over varying levels of aid reliability. Participants performed a binary signal detection task either unaided or with assistance from a decision aid that was 60%, 85%, or 96%-reliable. Assistance from a highly reliable aid (85% or 96%) improved discrimination performance, while assistance from a low-reliability aid (60%) did not. Because their ideal strategy is to place less weight on less reliable cues, however, the decision makers' tendency to disuse the aid became more appropriate as the aid's reliability declined. Automation-aided efficiency was thus near to optimal when the aid was close to chance but became highly inefficient, ironically, as the aid's reliability increased. Practitioner Summary: Investigating operators' automation-aided information integration strategies allows human factors practitioners to predict the level of performance the operator will attain. Ironically, in an aided signal detection task, performance when assisted by a highly reliable aid is far less efficient than that obtained when assisted by a far less reliable aid. Abbreviations: OW: optimal weighting; UW: uniform weighting; CC: contingent criterion; BD: best decides; CF: coin flip; PM: probability matching; HDI: highest density interval; MCMC: markov chain monte carlo; HR: hit rate; FAR: false alarm rate.

Automation reliability, human-machine system performance, and operator compliance: A study with airport security screeners supported by automated explosives detection systems for cabin baggage screening

Using a simulated X-ray screening task, we tested 122 airport security screeners working with the support of explosives detection systems for cabin baggage screening (EDSCB) as low-level automation. EDSCB varied systematically on three automation reliability measures: accuracy, d', and positive predictive value (PPV). Results showed that when unaided performance was high, operator confidence was high, and automation provided only small benefits. When unaided performance was lower, operator confidence was lower, and automation with higher d' provided large benefits. Operator compliance depended on the PPV of automation: We found lower compliance for lower PPV. Automation with a high false alarm rate of 20% and a low PPV of .3 resulted in operators ignoring about one-half of the true automation alarms on difficult targets-a strong cry-wolf effect. Our results suggest that automation reliability described by d' and PPV is more valid than using accuracy alone. When the PPV is below .5, operators should receive clear instructions on how to respond to automation alarms.