Herd Those Sheep: Emergent Multiagent Coordination and Behavioral-Mode Switching

A Typology for the Application of Team Coordination Dynamics Across Increasing Levels of Dynamic Complexity

OBJECTIVE

This review and synthesis examines approaches for measuring and assessing team coordination dynamics (TCD). The authors advance a system typology for classifying TCD approaches and their applications for increasing levels of dynamic complexity.

BACKGROUND

There is an increasing focus on how teams adapt their coordination in response to changing and uncertain operational conditions. Understanding coordination is significant because poor coordination is associated with maladaptive responses, whereas adaptive coordination is associated with effective responses. This issue has been met with TCD approaches that handle increasing complexity in the types of TCD teams exhibit.

METHOD

A three-level system typology of TCD approaches for increasing dynamic complexity is provided, with examples of research at each level. For System I TCD, team states converge toward a stable, fixed-point attractor. For System II TCD, team states are periodic, which can appear complex, yet are regular and relatively stable. In System III TCD, teams can exhibit periodic patterns, but those patterns change continuously to maintain effectiveness.

RESULTS

System I and System II are applicable to TCD with known or discoverable behavioral attractors that are stationary across mid-to long-range timescales. System III TCD is the most generalizable to dynamic environments with high requirements for adaptive coordination across a range of timescales.

CONCLUSION

We outline current challenges for TCD and next steps in this burgeoning field of research.

APPLICATION

System III approaches are becoming widespread, as they are generalizable to time- and/or scale-varying TCD and multimodal analyses. Recommendations for deploying TCD in team settings are provided.

Abduction and Deduction in Dynamical Cognitive Science

This paper reviews the recent history of a subset of research in dynamical cognitive science, in particular that subset that allies itself with the sciences of complexity and casts cognitive systems as interaction dominant, noncomputational, and nonmodular. I look at this history in the light of C.S. Peirce's understanding of scientific reasoning as progressing from abduction to deduction to induction. In particular, I examine the development of a controversy concerning the use of the interaction dominance of human cognitive systems as an explanation of the ubiquitous 1/f noise, multifractality, and complexity matching in human behavior.

Mouse tracking performance: A new approach to analyzing continuous mouse tracking data

Mouse tracking is an important source of data in cognitive science. Most contemporary mouse tracking studies use binary-choice tasks and analyze the curvature or velocity of an individual mouse movement during an experimental trial as participants select from one of the two options. However, there are many types of mouse tracking data available beyond what is produced in a binary-choice task, including naturalistic data from web users. In order to utilize these data, cognitive scientists need tools that are robust to the lack of trial-by-trial structure in most normal computer tasks. We use singular value decomposition (SVD) and detrended fluctuation analysis (DFA) to analyze whole time series of unstructured mouse movement data. We also introduce a new technique for describing two-dimensional mouse traces as complex-valued time series, which allows SVD and DFA to be applied in a straightforward way without losing important spatial information. We find that there is useful information at the level of whole time series, and we use this information to predict performance in an online task. We also discuss how the implications of these results can advance the use of mouse tracking research in cognitive science.

Predicting and understanding human action decisions during skillful joint-action using supervised machine learning and explainable-AI

This study investigated the utility of supervised machine learning (SML) and explainable artificial intelligence (AI) techniques for modeling and understanding human decision-making during multiagent task performance. Long short-term memory (LSTM) networks were trained to predict the target selection decisions of expert and novice players completing a multiagent herding task. The results revealed that the trained LSTM models could not only accurately predict the target selection decisions of expert and novice players but that these predictions could be made at timescales that preceded a player's conscious intent. Importantly, the models were also expertise specific, in that models trained to predict the target selection decisions of experts could not accurately predict the target selection decisions of novices (and vice versa). To understand what differentiated expert and novice target selection decisions, we employed the explainable-AI technique, SHapley Additive explanation (SHAP), to identify what informational features (variables) most influenced modelpredictions. The SHAP analysis revealed that experts were more reliant on information about target direction of heading and the location of coherders (i.e., other players) compared to novices. The implications and assumptions underlying the use of SML and explainable-AI techniques for investigating and understanding human decision-making are discussed.

Conversation dynamics in a multiplayer video game with knowledge asymmetry

Despite the challenges associated with virtually mediated communication, remote collaboration is a defining characteristic of online multiplayer gaming communities. Inspired by the teamwork exhibited by players in first-person shooter games, this study investigated the verbal and behavioral coordination of four-player teams playing a cooperative online video game. The game, Desert Herding, involved teams consisting of three ground players and one drone operator tasked to locate, corral, and contain evasive robot agents scattered across a large desert environment. Ground players could move throughout the environment, while the drone operator’s role was akin to that of a “spectator” with a bird’s-eye view, with access to veridical information of the locations of teammates and the to-be-corralled agents. Categorical recurrence quantification analysis (catRQA) was used to measure the communication dynamics of teams as they completed the task. Demands on coordination were manipulated by varying the ground players’ ability to observe the environment with the use of game “fog.” Results show that catRQA was sensitive to changes to task visibility, with reductions in task visibility reorganizing how participants conversed during the game to maintain team situation awareness. The results are discussed in the context of future work that can address how team coordination can be augmented with the inclusion of artificial agents, as synthetic teammates.

Experimental conditions in which dyads outperform individuals in a task of force produced by two people

When participants control periodic isometric force cycling between two target forces, they more accurately control force in a joint action than in an individual action. In some other studies, however, individuals tend to outperform dyads in joint action. The present study thus examined experimental conditions in which dyads outperformed individuals in a task of force produced by two people. This study consisted of two tasks with two target conditions and three force production conditions. The individual task was performed by one participant, and the joint task was performed by two participants. In absolute and relative target conditions, the participants made continuous, discrete, and periodic isometric pressing movements with the index finger. Although no difference was seen in force error between tasks in the continuous condition, the joint task had a smaller error than the individual task in the two other conditions. The joint task had a smaller variable force than the individual task in the periodic conditions, but no difference was seen in force variability between tasks in the two other conditions. Participants mainly controlled force in both tasks in the continuous condition. In the periodic or discrete condition at a prescribed interval, however, participants had to control both force and timing in the individual task, and muscle force must be mainly controlled to compensate for force errors by synchronizing interpersonal force outputs in the joint task. Thus, dyads can reduce the dimensionality of the control problem because they can synchronize their action which provides timing information.

Assessing Team Effectiveness by How Players Structure Their Search in a First-Person Multiplayer Video Game

People working as a team can achieve more than when working alone due to a team's ability to parallelize the completion of tasks. In collaborative search tasks, this necessitates the formation of effective division of labor strategies to minimize redundancies in search. For such strategies to be developed, team members need to perceive the task's relevant components and how they evolve over time, as well as an understanding of what others will do so that they can structure their own behavior to contribute to the team's goal. This study explored whether the capacity for team members to coordinate effectively can be related to how participants structure their search behaviors in an online multiplayer collaborative search task. Our results demonstrated that the structure of search behavior, quantified using detrended fluctuation analysis, was sensitive to contextual factors that limit a participant's ability to gather information. Further, increases in the persistence of movement fluctuations during search behavior were found as teams developed more effective coordinative strategies and were associated with better task performance.

Prospects for Augmenting Team Interactions with Real-Time Coordination-Based Measures in Human-Autonomy Teams

Complex work in teams requires coordination across team members and their technology as well as the ability to change and adapt over time to achieve effective performance. To support such complex interactions, recent efforts have worked toward the design of adaptive human-autonomy teaming systems that can provide feedback in or near real time to achieve the desired individual or team results. However, while significant advancements have been made to better model and understand the dynamics of team interaction and its relationship with task performance, appropriate measures of team coordination and computational methods to detect changes in coordination have not yet been widely investigated. Having the capacity to measure coordination in real time is quite promising as it provides the opportunity to provide adaptive feedback that may influence and regulate teams' coordination patterns and, ultimately, drive effective team performance. A critical requirement to reach this potential is having the theoretical and empirical foundation from which to do so. Therefore, the first goal of the paper is to review approaches to coordination dynamics, identify current research gaps, and draw insights from other areas, such as social interaction, relationship science, and psychotherapy. The second goal is to collate extant work on feedback and advance ideas for adaptive feedback systems that have potential to influence coordination in a way that can enhance the effectiveness of team interactions. In addressing these two goals, this work lays the foundation as well as plans for the future of human-autonomy teams that augment team interactions using coordination-based measures.

Herding stochastic autonomous agents via local control rules and online target selection strategies

We propose a simple yet effective set of local control rules to make a small group of “herder agents” collect and contain in a desired region a large ensemble of non-cooperative, non-flocking stochastic “target agents” in the plane. We investigate the robustness of the proposed strategies to variations of the number of target agents and the strength of the repulsive force they feel when in proximity of the herders. The effectiveness of the proposed approach is confirmed in both simulations in ROS and experiments on real robots.

Coordination dynamics of multi-agent interaction in a musical ensemble

Humans interact with other humans at a variety of timescales and in a variety of social contexts. We exhibit patterns of coordination that may differ depending on whether we are genuinely interacting as part of a coordinated group of individuals vs merely co-existing within the same physical space. Moreover, the local coordination dynamics of an interacting pair of individuals in an otherwise non-interacting group may spread, propagating change in the global coordination dynamics and interaction of an entire crowd. Dynamical systems analyses, such as Recurrence Quantification Analysis (RQA), can shed light on some of the underlying coordination dynamics of multi-agent human interaction. We used RQA to examine the coordination dynamics of a performance of "Welcome to the Imagination World", composed for wind orchestra. This performance enacts a real-life simulation of the transition from uncoordinated, non-interacting individuals to a coordinated, interacting multi-agent group. Unlike previous studies of social interaction in musical performance which rely on different aspects of video and/or acoustic data recorded from each individual, this project analyzes group-level coordination patterns solely from the group-level acoustic data of an audio recording of the performance. Recurrence and stability measures extracted from the audio recording increased when musicians coordinated as an interacting group. Variability in these measures also increased, indicating that the interacting ensemble of musicians were able to explore a greater variety of behavior than when they performed as non-interacting individuals. As an orchestrated (non-emergent) example of coordination, we believe these analyses provide an indication of approximate expected distributions for recurrence patterns that may be measurable before and after truly emergent coordination.

Task dynamics define the contextual emergence of human corralling behaviors

Social animals have the remarkable ability to organize into collectives to achieve goals unobtainable to individual members. Equally striking is the observation that despite differences in perceptual-motor capabilities, different animals often exhibit qualitatively similar collective states of organization and coordination. Such qualitative similarities can be seen in corralling behaviors involving the encirclement of prey that are observed, for example, during collaborative hunting amongst several apex predator species living in disparate environments. Similar encirclement behaviors are also displayed by human participants in a collaborative problem-solving task involving the herding and containment of evasive artificial agents. Inspired by the functional similarities in this behavior across humans and non-human systems, this paper investigated whether the containment strategies displayed by humans emerge as a function of the task's underlying dynamics, which shape patterns of goal-directed corralling more generally. This hypothesis was tested by comparing the strategies naïve human dyads adopt during the containment of a set of evasive artificial agents across two disparate task contexts. Despite the different movement types (manual manipulation or locomotion) required in the different task contexts, the behaviors that humans display can be predicted as emergent properties of the same underlying task-dynamic model.

Navigational Behavior of Humans and Deep Reinforcement Learning Agents

Rapid advances in the field of Deep Reinforcement Learning (DRL) over the past several years have led to artificial agents (AAs) capable of producing behavior that meets or exceeds human-level performance in a wide variety of tasks. However, research on DRL frequently lacks adequate discussion of the low-level dynamics of the behavior itself and instead focuses on meta-level or global-level performance metrics. In doing so, the current literature lacks perspective on the qualitative nature of AA behavior, leaving questions regarding the spatiotemporal patterning of their behavior largely unanswered. The current study explored the degree to which the navigation and route selection trajectories of DRL agents (i.e., AAs trained using DRL) through simple obstacle ridden virtual environments were equivalent (and/or different) from those produced by human agents. The second and related aim was to determine whether a task-dynamical model of human route navigation could not only be used to capture both human and DRL navigational behavior, but also to help identify whether any observed differences in the navigational trajectories of humans and DRL agents were a function of differences in the dynamical environmental couplings.

Self-organized division of cognitive labor

Often members of a group benefit from dividing the group’s task into separate components, where each member specializes their role so as to accomplish only one of the components. While this division of labor phenomenon has been observed with respect to both manual and cognitive labor, there is no clear understanding of the cognitive mechanisms allowing for its emergence, especially when there are multiple divisions possible and communication is limited. Indeed, maximization of expected utility often does not differentiate between alternative ways in which individuals could divide labor. We developed an iterative two-person game in which there are multiple ways of dividing labor, but in which it is not possible to explicitly negotiate a division. We implemented the game both as a human experimental task and as a computational model. Our results show that the majority of human dyads can finish the game with an efficient division of labor. Moreover, we fitted our computational model to the behavioral data, which allowed us to explain how the perceived similarity between a player’s actions and the task’s focal points guided the players’ choices from one round to the other, thus bridging the group dynamics and its underlying cognitive process. Potential applications of this model outside cognitive science include the improvement of cooperation in human groups, multi-agent systems, as well as human-robot collaboration.

Coordination Dynamics: A Foundation for Understanding Social Behavior

Humans' interactions with each other or with socially competent machines exhibit lawful coordination patterns at multiple levels of description. According to Coordination Dynamics, such laws specify the flow of coordination states produced by functional synergies of elements (e.g., cells, body parts, brain areas, people…) that are temporarily organized as single, coherent units. These coordinative structures or synergies may be mathematically characterized as informationally coupled self-organizing dynamical systems (Coordination Dynamics). In this paper, we start from a simple foundation, an elemental model system for social interactions, whose behavior has been captured in the Haken-Kelso-Bunz (HKB) model. We follow a tried and tested scientific method that tightly interweaves experimental neurobehavioral studies and mathematical models. We use this method to further develop a body of empirical research that advances the theory toward more generalized forms. In concordance with this interdisciplinary spirit, the present paper is written both as an overview of relevant advances and as an introduction to its mathematical underpinnings. We demonstrate HKB's evolution in the context of social coordination along several directions, with its applicability growing to increasingly complex scenarios. In particular, we show that accommodating for symmetry breaking in intrinsic dynamics and coupling, multiscale generalization and adaptation are principal evolutions. We conclude that a general framework for social coordination dynamics is on the horizon, in which models support experiments with hypothesis generation and mechanistic insights.

Hopf Bifurcations in Complex Multiagent Activity: The Signature of Discrete to Rhythmic Behavioral Transitions

Most human actions are composed of two fundamental movement types, discrete and rhythmic movements. These movement types, or primitives, are analogous to the two elemental behaviors of nonlinear dynamical systems, namely, fixed-point and limit cycle behavior, respectively. Furthermore, there is now a growing body of research demonstrating how various human actions and behaviors can be effectively modeled and understood using a small set of low-dimensional, fixed-point and limit cycle dynamical systems (differential equations). Here, we provide an overview of these dynamical motorprimitives and detail recent research demonstrating how these dynamical primitives can be used to model the task dynamics of complex multiagent behavior. More specifically, we review how a task-dynamic model of multiagent shepherding behavior, composed of rudimentary fixed-point and limit cycle dynamical primitives, can not only effectively model the behavior of cooperating human co-actors, but also reveals how the discovery and intentional use of optimal behavioral coordination during task learning is marked by a spontaneous, self-organized transition between fixed-point and limit cycle dynamics (i.e., via a Hopf bifurcation).

Physiological and Behavioral Synchrony Predict Group Cohesion and Performance

Interpersonal synchrony contributes to social functioning in dyads, but it remains unknown how synchrony shapes group experiences and performance. To this end, we designed a novel group drumming task in which participants matched their drumming to either predictable or unpredictable tempos. Fifty-one three-person groups were randomly assigned to one of two conditions: synchronized or asynchronized drumming. Outcome measures included electrocardiograms and self-reports of group cohesion and synchrony. The drumming task elicited an increase in physiological synchrony between group members (specifically their hearts' interbeat intervals). We also found that physiological synchronization and behavioral synchronization predicted individuals' experience of group cohesion. Physiological synchrony also predicted performance in a subsequent group task that involved freely drumming together. The findings suggest that the behavioral and physiological consequences of synchronization contribute to the formation of group bonds and coordination. They also confirm that insights from translational social neuroscience can inform our knowledge of the development of cohesive and efficacious groups.

Feedback delays can enhance anticipatory synchronization in human-machine interaction

Research investigating the dynamics of coupled physical systems has demonstrated that small feedback delays can allow a dynamic response system to anticipate chaotic behavior. This counterintuitive phenomenon, termed anticipatory synchronization, has been observed in coupled electrical circuits, laser semi-conductors, and artificial neurons. Recent research indicates that the same process might also support the ability of humans to anticipate the occurrence of chaotic behavior in other individuals. Motivated by this latter work, the current study examined whether the process of feedback delay induced anticipatory synchronization could be employed to develop an interactive artificial agent capable of anticipating chaotic human movement. Results revealed that incorporating such delays within the movement-control dynamics of an artificial agent not only enhances an artificial agent’s ability to anticipate chaotic human behavior, but to synchronize with such behavior in a manner similar to natural human-human anticipatory synchronization. The implication of these findings for the development of human-machine interaction systems is discussed.

A Multi-Agent Based Intelligent Training System for Unmanned Surface Vehicles

The modeling and design of multi-agent systems is imperative for applications in the evolving intelligence of unmanned systems. In this paper, we propose a multi-agent system design that is used to build a system for training a team of unmanned surface vehicles (USVs) where no historical data concerning the behavior is available. In this approach, agents are built as the physical controller of each USV and their cooperative decisions used for the USVs’ group coordination. To make our multi-agent system intelligently coordinate USVs, we built a multi-agent-based learning system. First, an agent-based data collection platform is deployed to gather competition data from agents’ observation for on-line learning tasks. Second, we design a genetic-based fuzzy rule training algorithm that is capable of optimizing agents’ coordination decisions in an accumulated manner. The simulation results of this study demonstrate that our proposed training approach is feasible and able to converge to a stable action selection policy towards efficient multi-USVs’ cooperative decision making.

Human social motor solutions for human-machine interaction in dynamical task contexts

Multiagent activity is commonplace in everyday life and can improve the behavioral efficiency of task performance and learning. Thus, augmenting social contexts with the use of interactive virtual and robotic agents is of great interest across health, sport, and industry domains. However, the effectiveness of human-machine interaction (HMI) to effectively train humans for future social encounters depends on the ability of artificial agents to respond to human coactors in a natural, human-like manner. One way to achieve effective HMI is by developing dynamical models utilizing dynamical motor primitives (DMPs) of human multiagent coordination that not only capture the behavioral dynamics of successful human performance but also, provide a tractable control architecture for computerized agents. Previous research has demonstrated how DMPs can successfully capture human-like dynamics of simple nonsocial, single-actor movements. However, it is unclear whether DMPs can be used to model more complex multiagent task scenarios. This study tested this human-centered approach to HMI using a complex dyadic shepherding task, in which pairs of coacting agents had to work together to corral and contain small herds of virtual sheep. Human-human and human-artificial agent dyads were tested across two different task contexts. The results revealed (i) that the performance of human-human dyads was equivalent to those composed of a human and the artificial agent and (ii) that, using a "Turing-like" methodology, most participants in the HMI condition were unaware that they were working alongside an artificial agent, further validating the isomorphism of human and artificial agent behavior.

The Emergence of Social Norms and Conventions

The utility of our actions frequently depends upon the beliefs and behavior of other agents. Thankfully, through experience, we learn norms and conventions that provide stable expectations for navigating our social world. Here, we review several distinct influences on their content and distribution. At the level of individuals locally interacting in dyads, success depends on rapidly adapting pre-existing norms to the local context. Hence, norms are shaped by complex cognitive processes involved in learning and social reasoning. At the population level, norms are influenced by intergenerational transmission and the structure of the social network. As human social connectivity continues to increase, understanding and predicting how these levels and time scales interact to produce new norms will be crucial for improving communities.

The History and Philosophy of Ecological Psychology

Ecological Psychology is an embodied, situated, and non-representational approach pioneered by J. J. Gibson and E. J. Gibson. This theory aims to offer a third way beyond cognitivism and behaviorism for understanding cognition. The theory started with the rejection of the premise of the poverty of the stimulus, the physicalist conception of the stimulus, and the passive character of the perceiver of mainstream theories of perception. On the contrary, the main principles of ecological psychology are the continuity of perception and action, the organism-environment system as unit of analysis, the study of affordances as the objects of perception, combined with an emphasis on perceptual learning and development. In this paper, first, we analyze the philosophical and psychological influences of ecological psychology: pragmatism, behaviorism, phenomenology, and Gestalt psychology. Second, we summarize the main concepts of the approach and their historical development following the academic biographies of the proponents. Finally, we highlight the most significant developments of this psychological tradition. We conclude that ecological psychology is one of the most innovative approaches in the psychological field, as it is reflected in its current influence in the contemporary embodied and situated cognitive sciences, where the notion of affordance and the work of E. J. Gibson and J. J. Gibson is considered as a historical antecedent.

Information-Based Social Coordination Between Players of Different Skill in Doubles Pong

We studied how teams of two players of different skill level intercepted approaching balls in the doubles-pong task. In this task, the two players moved their on-screen paddles along a shared interception axis, so that the approaching ball was intercepted by one of the paddles and that the paddles did not collide. Earlier work revealed the presence of a fuzzy division of interception space, with a boundary between interception domains located in the space between the two initial paddle positions. In the present study, using the performance of the players in their individual training sessions, we formed teams of players of varying skill level. We considered two accounts of how this boundary should be understood. In a first account, the players have shared knowledge of this boundary. Based on the side of the boundary at which the approaching ball will cross the interception axis, the players would decide whose paddle is to make the interception. Under this account, we expected that a better-skilled player would take responsibility for a larger interception domain, leading to a boundary closer to the lesser-skilled player. However, our analyses did not reveal any systematic effect of skill difference on the location (or degree of fuzziness) of the boundary: location of boundaries and overlap of interception domains varied over teams but were not systematically related to skill differences between team members. We did find effects of ball speed and approach angle. In a second account, the boundary emerges from (information-driven) player–player–ball interactions. An action-based model consistent with this account was able to capture all the patterns in boundary positions and overlaps that we observed. We conclude that the interception patterns that players demonstrate in the doubles-pong task are best understood as emerging from the unfolding of the dynamics of the system of the two players and the ball, coupled through information.

Autonomous Shepherding Behaviors of Multiple Target Steering Robots

This paper presents a distributed coordination methodology for multi-robot systems, based on nearest-neighbor interactions. Among many interesting tasks that may be performed using swarm robots, we propose a biologically-inspired control law for a shepherding task, whereby a group of external agents drives another group of agents to a desired location. First, we generated sheep-like robots that act like a flock. We assume that each agent is capable of measuring the relative location and velocity to each of its neighbors within a limited sensing area. Then, we designed a control strategy for shepherd-like robots that have information regarding where to go and a steering ability to control the flock, according to the robots’ position relative to the flock. We define several independent behavior rules; each agent calculates to what extent it will move by summarizing each rule. The flocking sheep agents detect the steering agents and try to avoid them; this tendency leads to movement of the flock. Each steering agent only needs to focus on guiding the nearest flocking agent to the desired location. Without centralized coordination, multiple steering agents produce an arc formation to control the flock effectively. In addition, we propose a new rule for collecting behavior, whereby a scattered flock or multiple flocks are consolidated. From simulation results with multiple robots, we show that each robot performs actions for the shepherding behavior, and only a few steering agents are needed to control the whole flock. The results are displayed in maps that trace the paths of the flock and steering robots. Performance is evaluated via time cost and path accuracy to demonstrate the effectiveness of this approach.