Representing situation awareness in collaborative systems: a case study in the energy distribution domain

Distributed situation awareness: From awareness in individuals and teams to the awareness of technologies, sociotechnical systems, and societies

A large component of Neville Stanton's work has focused on situation awareness in domains such as defence, transport, and process control. A significant contribution has been to initiate a shift from considering individual human operator situation awareness to considering the situation awareness of human and non-human teams, organisations, and even sociotechnical systems. Though controversial when introduced, the distributed situation awareness model has become increasingly relevant for modern day systems and problems. In this article we reflect on Stanton's contribution and point to a pressing need to consider a. The situation awareness of advanced technologies, and b. situation awareness at a sociotechnical system, societal and even global level. This is demonstrated via discussion on two contemporaneous issues: automated vehicles and the COVID-19 pandemic. It is concluded that, given advances such as artificial intelligence, the increased connectedness of society, emerging issues such as disinformation, and an increasing set of global threats, Stanton's distributed situation awareness model and associated analysis framework provide a useful toolkit for future Human Factors and Ergonomics applications.

Distributed situation awareness: a health-system approach to assessing and designing patient flow management

Patient flow management is a system-wide process but many healthcare providers do not integrate multiple departments into the process to minimise the time between treatments or medical services for maximum patient throughput. This paper presents a case study of applying Distributed Situation Awareness (DSA) to characterise system-wide patient flow management and identify opportunities for improvements in a healthcare system. This case study employed a three-part method of data elicitation, extraction, and representation to investigate DSA. Social, task, and knowledge networks were developed and then combined to characterise patient flow management and identify deficiencies of the command and control centre of a healthcare facility. Social network analysis provided centrality metrics to further characterise patient flow management. The DSA model helped identify design principles and deficiencies in managing patient flow. These findings indicate that DSA is promising for analysing patient flow management from a system-wide perspective. Practitioner summary: This article examines Distribution Situation Awareness (DSA) as an analysis framework to study system-wide patient flow management. The DSA yields social, task, and knowledge networks that can be combined to characterise patient flow and identify deficiencies in the system. DSA appears promising for analysing communication and coordination of complex systems. Abbreviations: CDM: critical decision method; CTaC: carilion transfer and communications center; EAST: event analysis systematic teamwork; ED: emergency department; DES: discrete event simulation; DSA: distributed situation awareness; SA: situation awareness; SNA: social network analysis.

Improving Team-Based Decision Making Using Data Analytics and Informatics: Protocol for a Collaborative Decision Support Design

Background

According to the September 2015 Institute of Medicine report, Improving Diagnosis in Health Care, each of us is likely to experience one diagnostic error in our lifetime, often with devastating consequences. Traditionally, diagnostic decision making has been the sole responsibility of an individual clinician. However, diagnosis involves an interaction among interprofessional team members with different training, skills, cultures, knowledge, and backgrounds. Moreover, diagnostic error is prevalent in the interruption-prone environment, such as the emergency department, where the loss of information may hinder a correct diagnosis.

Objective

The overall purpose of this protocol is to improve team-based diagnostic decision making by focusing on data analytics and informatics tools that improve collective information management.

Methods

To achieve this goal, we will identify the factors contributing to failures in team-based diagnostic decision making (aim 1), understand the barriers of using current health information technology tools for team collaboration (aim 2), and develop and evaluate a collaborative decision-making prototype that can improve team-based diagnostic decision making (aim 3).

Results

Between 2019 to 2020, we are collecting data for this study. The results are anticipated to be published between 2020 and 2021.

Conclusions

The results from this study can shed light on improving diagnostic decision making by incorporating diagnostics rationale from team members. We believe a positive direction to move forward in solving diagnostic errors is by incorporating all team members, and using informatics.

International Registered Report Identifier (IRRID)

DERR1-10.2196/16047

The effects of team co-location and reduced crewing on team communication characteristics

The manner in which control rooms are configured can impact the flow of information between command teams. Previous research revealed bottlenecks of communications between the Sonar Controller (SOC) and the Operations Officer (OPSO) in submarine control rooms. One way to relieve such bottlenecks is to co-locate operators reliant on one another for task relevant information. The aim of the current studies was to use multiple command teams to empirically examine a novel submarine control room configuration and a reduced crew size in comparison to a baseline of contemporary operations to see if such bottlenecks could be removed. Ten teams performed high and low demand Dived Tracking (DT) scenarios in a simulated submarine control room. Activities and communications of the teams were recorded and quantified using the Event Analysis of Systemic Teamwork (EAST) method affording statistical comparisons with a baseline condition of contemporary operations. The findings showed that the co-location of operators relieved the bottleneck of communications between the SOC and the OPSO. Although overall communications increased, this was more balanced across the team and was more adaptive to scenario demand. This was coupled with a significant increase in task completion, even with a reduced crew size, suggesting greater efficiency and productivity. Future research should seek to validate the changes observed with objective measures of task performance.

Radio Gaga? Intra-team communication of Australian Rules Football umpires - effect of radio communication on content, structure and frequency

Intra-team communication plays an important role in team effectiveness in various domains including sport. As such, it is a key consideration when introducing new tools within systems that utilise teams. The difference in intra-team communication of Australian Rules Football (AFL) umpiring teams was studied when umpiring with or without radio communications technology. A cross-sectional observational study was conducted to analyse the verbal communication of seven umpiring teams (20 participants) grouped according to their experience with radio communication. The results identified that radio communication technology increased the frequency and altered the structure of intra-team communication. Examination of the content of the intra-team communication identified impacts on the 'Big Five' teamwork behaviours and associated coordinating mechanisms. Analysis revealed that the communications utilised did not align with the closed-loop form of communication described in the Big Five model. Implications for teamwork models, coaching and training of AFL umpires are discussed. Practitioner Summary: Assessing the impact of technology on performance is of interest to ergonomics practitioners. The impact of radio communications on teamwork is explored in the highly dynamic domain of AFL umpiring. When given radio technology, intra-team communication increased which supported teamwork behaviours, such as backup behaviour and mutual performance monitoring.

State-of-science: situation awareness in individuals, teams and systems

Our review addresses one of the most used, but debated, topics in Ergonomics: Situation Awareness (SA). We examine and elaborate upon key SA models. These models are divided into individual SA, team SA and systems SA categories. Despite, or perhaps because of, the debates surrounding SA it remains an enduring theme for research and practice in the domain of Ergonomics, now for over two decades. A contingent approach, which seeks to match different models of SA to different types of ergonomics problem, enables the differences between positions to be revealed and reconciled, and the practitioner guided towards optimum methodological solutions. Practitioner Summary: Measuring SA in individuals, teams and systems has become a key objective in Ergonomics. One single approach to SA does not fit all problems encountered. This review shows the importance of considering all three types of models and achieving a match between them and the problem at hand.

'We didn't know anything, it was a mess!' Emergent structures and the effectiveness of a rescue operation multi-team system

Multi-team systems (MTS) are used to tackle unpredictable events and to respond effectively to fast-changing environmental contingencies. Their effectiveness is influenced by within as well as between team processes (i.e. communication, coordination) and emergent phenomena (i.e. situational awareness). The present case study explores the way in which the emergent structures and the involvement of bystanders intertwine with the dynamics of processes and emergent states both within and between the component teams. Our findings show that inefficient transition process and the ambiguous leadership generated poor coordination and hindered the development of emergent phenomena within the whole system. Emergent structures and bystanders substituted leadership functions and provided a pool of critical resources for the MTS. Their involvement fostered the emergence of situational awareness and facilitated contingency planning processes. However, bystander involvement impaired the emergence of cross-understandings and interfered with coordination processes between the component teams. Practitioner Summary: Based on a real emergency situation, the present research provides important theoretical and practical insights about the role of bystander involvement in the dynamics of multi-team systems composed to tackle complex tasks and respond to fast changing and unpredictable environmental contingencies.

Exploring the use of situation awareness in behaviors and practices of health and safety leaders

An understanding of how health and safety management systems (HSMS) reduce worksite injuries, illnesses and fatalities may be gained in studying the behaviors of health and safety leaders. These leaders bear the accountability for identifying, understanding and managing the risks of a mining operation. More importantly, they have to transfer this knowledge of perception, recognition and response to risks in the mining environment to their workers. The leaders' efforts to build and maintain a mining operation's workforce that consistently executes safe work practices may be captured through more than just lagging indicators of health and safety performance. This exploratory study interviewed six leaders in occupations such as site-level safety supervisors, mine superintendents and/or general managers at surface and underground stone, sand and gravel and metal/nonmetal mine sites throughout the United States, with employee populations ranging from 40 to 175. In exploring leaders' perspectives on how they systematically manage health and safety, examples such as approaches to task training, handling near-miss incidents, identifying future leaders and providing workers with feedback offer insights into how leaders translate their knowledge and management of site-level risks to others.

Never blame the umpire - a review of Situation Awareness models and methods for examining the performance of officials in sport

As sport becomes more complex, there is potential for ergonomics concepts to help enhance the performance of sports officials. The concept of Situation Awareness (SA) appears pertinent given the requirement for officials to understand what is going on in order to make decisions. Although numerous models exist, none have been applied to examine officials, and only several recent examples have been applied to sport. This paper examines SA models and methods to identify if any have applicability to officials in sport (OiS). Evaluation of the models and methods identified potential applications of individual, team and systems models of SA. The paper further demonstrates that the Distributed Situation Awareness model is suitable for studying officials in fastball sports. It is concluded that the study of SA represents a key area of multidisciplinary research for both ergonomics and sports science in the context of OiS. Practitioner Summary: Despite obvious synergies, applications of cognitive ergonomics concepts in sport are sparse. This is especially so for Officials in Sport (OiS). This article presents an evaluation of Situation Awareness models and methods, providing practitioners with guidance on which are the most suitable for OiS system design and evaluation.

Investigating information-processing performance of different command team structures in the NATO Problem Space

The structure of command teams is a significant factor on their communications and ability to process, and act upon, information. The NATO Problem Space was used in this study to represent three of the main dimensions in the battle-space environment: familiarity, rate of change, and strength of information position. Results show that the five common team structures (chain, Y, circle, wheel and all-connected) did not generally perform as predicted in team literature. Findings suggest that under dynamic and highly variable conditions, high levels of synchronisation and trust should be present. On the other hand, synchronisation and trust are less important in hierarchical, highly centralised structures, because team members are more willing to accept the authority of a single leader and this tight control ensures that these teams can perform well as long as the Problem Space is familiar, information is explicit and the environment does not change. Practitioner Summary: Some types of team structures are better suited to particular constraints of the battle-space than others. This research has shown that the much touted all-connected structure is often the worst performing structure and that the traditional hierarchy of command and control has much merit in the digital information age.

A theoretical framework for negotiating the path of emergency management multi-agency coordination

Multi-agency coordination represents a significant challenge in emergency management. The need for liaison officers working in strategic level emergency operations centres to play organizational boundary spanning roles within multi-agency coordination arrangements that are enacted in complex and dynamic emergency response scenarios creates significant research and practical challenges. The aim of the paper is to address a gap in the literature regarding the concept of multi-agency coordination from a human-environment interaction perspective. We present a theoretical framework for facilitating multi-agency coordination in emergency management that is grounded in human factors and ergonomics using the methodology of core-task analysis. As a result we believe the framework will enable liaison officers to cope more efficiently within the work domain. In addition, we provide suggestions for extending the theory of core-task analysis to an alternate high reliability environment.

The process of processing: exploring the validity of Neisser's perceptual cycle model with accounts from critical decision-making in the cockpit

The perceptual cycle model (PCM) has been widely applied in ergonomics research in domains including road, rail and aviation. The PCM assumes that information processing occurs in a cyclical manner drawing on top-down and bottom-up influences to produce perceptual exploration and actions. However, the validity of the model has not been addressed. This paper explores the construct validity of the PCM in the context of aeronautical decision-making. The critical decision method was used to interview 20 helicopter pilots about critical decision-making. The data were qualitatively analysed using an established coding scheme, and composite PCMs for incident phases were constructed. It was found that the PCM provided a mutually exclusive and exhaustive classification of the information-processing cycles for dealing with critical incidents. However, a counter-cycle was also discovered which has been attributed to skill-based behaviour, characteristic of experts. The practical applications and future research questions are discussed. Practitioner Summary: This paper explores whether information processing, when dealing with critical incidents, occurs in the manner anticipated by the perceptual cycle model. In addition to the traditional processing cycle, a reciprocal counter-cycle was found. This research can be utilised by those who use the model as an accident analysis framework.

Using the event analysis of systemic teamwork (EAST) to explore conflicts between different road user groups when making right hand turns at urban intersections

Collisions between different types of road users at intersections form a substantial component of the road toll. This paper presents an analysis of driver, cyclist, motorcyclist and pedestrian behaviour at intersections that involved the application of an integrated suite of ergonomics methods, the Event Analysis of Systemic Teamwork (EAST) framework, to on-road study data. EAST was used to analyse behaviour at three intersections using data derived from an on-road study of driver, cyclist, motorcyclist and pedestrian behaviour. The analysis shows the differences in behaviour and cognition across the different road user groups and pinpoints instances where this may be creating conflicts between different road users. The role of intersection design in creating these differences in behaviour and resulting conflicts is discussed. It is concluded that currently intersections are not designed in a way that supports behaviour across the four forms of road user studied. Interventions designed to improve intersection safety are discussed. Practitioner Summary: Intersection safety currently represents a key road safety issue worldwide. This paper presents a novel application of a framework of ergonomics methods for studying differences in road user behaviour at intersections. The findings support development of interventions that consider all road users as opposed to one group in isolation.

Representing distributed cognition in complex systems: how a submarine returns to periscope depth

This paper presents the Event Analysis of Systemic Teamwork (EAST) method as a means of modelling distributed cognition in systems. The method comprises three network models (i.e. task, social and information) and their combination. This method was applied to the interactions between the sound room and control room in a submarine, following the activities of returning the submarine to periscope depth. This paper demonstrates three main developments in EAST. First, building the network models directly, without reference to the intervening methods. Second, the application of analysis metrics to all three networks. Third, the combination of the aforementioned networks in different ways to gain a broader understanding of the distributed cognition. Analyses have shown that EAST can be used to gain both qualitative and quantitative insights into distributed cognition. Future research should focus on the analyses of network resilience and modelling alternative versions of a system.

Using social network analysis and agent-based modelling to explore information flow using common operational pictures for maritime search and rescue operations

The concept of common operational pictures (COPs) is explored through the application of social network analysis (SNA) and agent-based modelling to a generic search and rescue (SAR) scenario. Comparing the command structure that might arise from standard operating procedures with the sort of structure that might arise from examining information-in-common, using SNA, shows how one structure could be more amenable to 'command' with the other being more amenable to 'control' - which is potentially more suited to complex multi-agency operations. An agent-based model is developed to examine the impact of information sharing with different forms of COPs. It is shown that networks using common relevant operational pictures (which provide subsets of relevant information to groups of agents based on shared function) could result in better sharing of information and a more resilient structure than networks that use a COP.

PRACTITIONER SUMMARY

SNA and agent-based modelling are used to compare different forms of COPs for maritime SAR operations. Different forms of COP change the communications structures in the socio-technical systems in which they operate, which has implications for future design and development of a COP.

Are posture data from simulated tasks representative of field conditions? Case study for overhead electric utility workers

Many ergonomics studies are conducted in laboratory-simulated work environments to assess risks for the development of musculoskeletal disorders under more controlled conditions. However, the simulated conditions could be of questionable validity with respect to reproduction of field conditions involving risk factors. The objective of this study was to verify whether the postures recorded for neck extension/flexion and upper arm elevation of overhead electric utility workers in a simulated environment were similar to those recorded in a field environment. Of the three frequently performed tasks analysed, two presented similar exposure in both conditions. However, differences were identified for a more complex task (relay replacement). These results suggest that simulated tasks may be more representative for more standardised tasks. This may indicate that researchers investigating risks should avoid simplifying complex tasks when reproducing field posture exposure in laboratories, since omitting extra subtasks may lead to an inaccurate reproduction of field exposure. Practitioner Summary: Studies comparing results from field and simulated environments are necessary to evaluate to what degree postural exposure reproduced in laboratory is representative of the exposure occurring in real work situations. This is particularly relevant for tasks involving training in simulated environment due to safety constraints.

Information chaos in primary care: implications for physician performance and patient safety

PURPOSE

The purpose of this article is to explore the concept of information chaos as it applies to the issues of patient safety and physician workload in primary care and to propose a research agenda.

METHODS

We use a human factors engineering perspective to discuss the concept of information chaos in primary care and explore implications for its impact on physician performance and patient safety.

RESULTS

Information chaos is comprised of various combinations of information overload, information underload, information scatter, information conflict, and erroneous information. We provide a framework for understanding information chaos, its impact on physician mental workload and situation awareness, and its consequences, and we discuss possible solutions and suggest a research agenda that may lead to methods to reduce the problem.

CONCLUSIONS

Information chaos is experienced routinely by primary care physicians. This is not just inconvenient, annoying, and frustrating; it has implications for physician performance and patient safety. Additional research is needed to define methods to measure and eventually reduce information chaos.

What could they have been thinking? How sociotechnical system design influences cognition: a case study of the Stockwell shooting

Understanding why an individual acted in a certain way is of fundamental importance to the human factors community, especially when the choice of action results in an undesirable outcome. This challenge is typically tackled by applying retrospective interview techniques to generate models of what happened, recording deviations from a 'correct procedure'. While such approaches may have great utility in tightly constrained procedural environments, they are less applicable in complex sociotechnical systems that require individuals to modify procedures in real time to respond to a changing environment. For complex sociotechnical systems, a formative approach is required that maps the information available to the individual and considers its impact on performance and action. A context-specific, activity-independent, constraint-based model forms the basis of this approach. To illustrate, an example of the Stockwell shooting is used, where an innocent man, mistaken for a suicide bomber, was shot dead. Transferable findings are then presented. STATEMENT OF RELEVANCE: This paper presents a new approach that can be applied proactively to consider how sociotechnical system design, and the information available to an individual, can affect their performance. The approach is proposed to be complementary to the existing tools in the mental models phase of the cognitive work analysis framework.

Description of dynamic shared knowledge: an exploratory study during a competitive team sports interaction

This exploratory case study describes the sharedness of knowledge within a basketball team (nine players) and how it changes during an official match. To determine how knowledge is mobilised in an actual game situation, the data were collected and processed following course-of-action theory (Theureau 2003). The results were used to characterise the contents of the shared knowledge (i.e. regarding teammate characteristics, team functioning, opponent characteristics, opposing team functioning and game conditions) and to identify the characteristic types of change: (a) the reinforcement of a previous element of shared knowledge; (b) the invalidation of an element of shared knowledge; (c) fragmentation of an element of shared knowledge; (d) the creation of a new element of shared knowledge. The discussion deals with the diverse types of change in shared knowledge and the heterogeneous and dynamic nature of common ground within the team. STATEMENT OF RELEVANCE: The present case study focused on how the cognitions of individual members of a team coordinate to produce a team performance (e.g. surgical teams in hospitals, military teams) and how the shared knowledge changes during team activity. Traditional methods to increase knowledge sharedness can be enhanced by making use of 'opportunities for coordination' to optimise team adaptiveness.

A new approach for designing cognitive artefacts to support disaster management

The public display of information is not a new phenomenon; whiteboards (and blackboards before them) have commonly been used to share information. Once annotated, these collaborative information sources have clear benefits for cognition, reducing the burden on short-term memory and supporting parallel processing. This paper introduces a structured approach for considering the design and development of these cognitive artefacts with the aim of enhancing system performance. To illustrate this approach, a case study of military support to large-scale civilian emergencies is used. Using the introduced process, a number of displays are developed supporting individual and shared understanding of the domain, enhancing the crisis planning and management process. The case study demonstrates how the proposed structured approach can inform the designer and lead to domain specific designs. A clear trail can be plotted between the analysis of the domain and the development of the cognitive artefacts. STATEMENT OF RELEVANCE: This paper presents a generic approach for the design of cognitive artefacts to enhance system performance. Illustrated by a case study, it is contended that the approach, and adaptations of it, are applicable to supporting the design of information aids for a wide range of complex domains.

Investigating accident causation through information network modelling

Management of risk in complex domains such as aviation relies heavily on post-event investigations, requiring complex approaches to fully understand the integration of multi-causal, multi-agent and multi-linear accident sequences. The Event Analysis of Systemic Teamwork methodology (EAST; Stanton et al. 2008) offers such an approach based on network models. In this paper, we apply EAST to a well-known aviation accident case study, highlighting communication between agents as a central theme and investigating the potential for finding agents who were key to the accident. Ultimately, this work aims to develop a new model based on distributed situation awareness (DSA) to demonstrate that the risk inherent in a complex system is dependent on the information flowing within it. By identifying key agents and information elements, we can propose proactive design strategies to optimize the flow of information and help work towards avoiding aviation accidents. Statement of Relevance: This paper introduces a novel application of an holistic methodology for understanding aviation accidents. Furthermore, it introduces an ongoing project developing a nonlinear and prospective method that centralises distributed situation awareness and communication as themes. The relevance of findings are discussed in the context of current ergonomic and aviation issues of design, training and human-system interaction.

Using an integrated methods approach to analyse the emergent properties of military command and control

This paper applies the event analysis for systemic teamwork (EAST) method to an example of military command and control. EAST offers a way to describe system level emergent properties that arise from the complex interactions of system components (human and technical). These are described using an integrated methods approach and modelled using Task, Social and Propositional networks. The current article is divided into three parts: a brief description of the military command and control context, a brief description of the EAST method, and a more in depth presentation of the analysis outcomes. The emergent properties of the military scenario relate to the degree of system reconfigurability, systems level situational awareness and the role of mediating technology. The findings are compared with similar analyses undertaken in civilian domains, in which the latest developments in command and control, under the aegis of Network Enabled Capability (NEC), are already in place.