Investigating Conversational Dynamics: Interactive Alignment, Interpersonal Synergy, and Collective Task Performance

A general model of conversational dynamics and an example application in serious illness communication

Conversation has been a primary means for the exchange of information since ancient times. Understanding patterns of information flow in conversations is a critical step in assessing and improving communication quality. In this paper, we describe COnversational DYnamics Model (CODYM) analysis, a novel approach for studying patterns of information flow in conversations. CODYMs are Markov Models that capture sequential dependencies in the lengths of speaker turns. The proposed method is automated and scalable, and preserves the privacy of the conversational participants. The primary function of CODYM analysis is to quantify and visualize patterns of information flow, concisely summarized over sequential turns from one or more conversations. Our approach is general and complements existing methods, providing a new tool for use in the analysis of any type of conversation. As an important first application, we demonstrate the model on transcribed conversations between palliative care clinicians and seriously ill patients. These conversations are dynamic and complex, taking place amidst heavy emotions, and include difficult topics such as end-of-life preferences and patient values. We use CODYMs to identify normative patterns of information flow in serious illness conversations, show how these normative patterns change over the course of the conversations, and show how they differ in conversations where the patient does or doesn’t audibly express anger or fear. Potential applications of CODYMs range from assessment and training of effective healthcare communication to comparing conversational dynamics across languages, cultures, and contexts with the prospect of identifying universal similarities and unique “fingerprints” of information flow.

Analysis and Design of Social Presence in a Computer-Mediated Communication System

Social presence, or the subjective experience of being present with another existing person, varies with the interaction medium. In general, social presence research has mainly focused on uni-directional aspects of each exchanged message, not on bidirectional interactions. Our primary purpose is to introduce such bidirectional evaluation by quantifying the degree of social presence with a few statistical measures. To this end, we developed a software called “TypeTrace” that records all keystrokes of online chat interactants and reenacts their typing actions and analyzed the results from different chat conditions, mainly focusing on the characterization of bi-directional interactions. We also compared the chat interaction patterns with the patterns from phone call datasets to investigate the difference of live communication in different media. The hypothesis of the experiment was that either richness or concurrency of communication is important for organizing social presence. Richness is defined by the variety of information at a time in communication and the concurrency is the number of temporal thread being processed at the same time. Our results show that when we merely increase the richness of information by presenting the typing process, the cognition of others' presence does not significantly increase. However, when the information concurrency is augmented by introducing the transmission of realtime text, we found that the transfer entropy between the interactants becomes considerably higher, and the social presence and emotional arousal, intimacy increased. High transfer entropy was also observed in the phone call dataset. This result shows that the mere augmentation of information richness does not necessarily lead to increased social presence, and concurrent communication is another critical factor for fostering vivid conversation in digital environments.

Group-level physiological synchrony and individual-level anxiety predict positive affective behaviors during a group decision-making task

Joint performance can lead to the synchronization of physiological processes among group members during a shared task. Recently, it has been shown that synchronization is indicative of subjective ratings of group processes and task performance. However, different methods have been used to quantify synchronization, and little is known about the effects of the choice of method and level of analysis (individuals, dyads, or triads) on the results. In this study, participants performed a decision‐making task in groups of three while physiological signals (heart rate and electrodermal activity), positive affective behavior, and personality traits were measured. First, we investigated the effects of different levels of analysis of physiological synchrony on affective behavior. We computed synchrony measures as (a) individual contributions to group synchrony, (b) the average dyadic synchrony within a group, and (c) group‐level synchrony. Second, we assessed the association between physiological synchrony and positive affective behavior. Third, we investigated the moderating effects of trait anxiety and social phobia on behavior. We discovered that the effects of physiological synchrony on positive affective behavior were particularly strong at the group level but nonsignificant at the individual and dyadic levels. Moreover, we found that heart rate and electrodermal synchronization showed opposite effects on group members' display of affective behavior. Finally, trait anxiety moderated the relationship between physiological synchrony and affective behavior, perhaps due to social uncertainty, while social phobia did not have a moderating effect. We discuss these results regarding the role of different physiological signals and task demands during joint action.

Impact Statement

Despite the inherent multilevel structure of groups, little is known regarding how physiological coupling between group members relates to their behaviors during joint group tasks at multiple levels (individual, dyadic, and group). We showed that the relationship between physiological synchrony and smiling/laughing behaviors made by group members were particularly strong at the group level but nonsignificant at the individual and dyadic levels. By using innovative quantification methods—Multidimensional Recurrence Quantification Analysis—we highlight the importance of modeling data in a way that allows for multilevel considerations within groups.

Musical improvisation enhances interpersonal coordination in subsequent conversation: Motor and speech evidence

This study explored the effects of musical improvisation between dyads of same-sex strangers on subsequent behavioural alignment. Participants-all non-musicians-conversed before and after either improvising music together (Musical Improvisation-MI-group) or doing a motoric non-rhythmic cooperative task (building a tower together using wooden blocks; the Hands-Busy-HB-group). Conversations were free, but initially guided by an adaptation of the Fast Friends Questionnaire for inducing talk among students who are strangers and meeting for the first time. Throughout, participants' motion was recorded with an optical motion-capture system (Mocap) and analysed in terms of speed cross-correlations. Their conversations were also recorded on separate channels using headset microphones and were analysed in terms of the periodicity displayed by rhythmic peaks in the turn transitions across question and answer pairs (Q+A pairs). Compared with their first conversations, the MI group in the second conversations showed: (a) a very rapid, partially simultaneous anatomical coordination between 0 and 0.4 s; (b) delayed mirror motoric coordination between 0.8 and 1.5 s; and (c) a higher proportion of Periodic Q+A pairs. In contrast, the HB group's motoric coordination changed slightly in timing but not in degree of coordination between the first and second conversations, and there was no significant change in the proportion of periodic Q+A pairs they produced. These results show a convergent effect of prior musical interaction on joint body movement and use of shared periodicity across speech turn-transitions in conversations, suggesting that interaction in music and speech may be mediated by common processes.

Coordination and Consonance Between Interacting, Improvising Musicians

Joint action (JA) is ubiquitous in our cognitive lives. From basketball teams to teams of surgeons, humans often coordinate with one another to achieve some common goal. Idealized laboratory studies of group behavior have begun to elucidate basic JA mechanisms, but little is understood about how these mechanisms scale up in more sophisticated and open-ended JA that occurs in the wild. We address this gap by examining coordination in a paragon domain for creative joint expression: improvising jazz musicians. Coordination in jazz music subserves an aesthetic goal: the generation of a collective musical expression comprising coherent, highly nuanced musical structure (e.g., rhythm, harmony). In our study, dyads of professional jazz pianists improvised in a "coupled," mutually adaptive condition, and an "overdubbed" condition that precluded mutual adaptation, as occurs in common studio recording practices. Using a model of musical tonality, we quantify the flow of rhythmic and harmonic information between musicians as a function of interaction condition. Our analyses show that mutually adapting dyads achieve greater temporal alignment and produce more consonant harmonies. These musical signatures of coordination were preferred by independent improvisers and naive listeners, who gave higher quality ratings to coupled interactions despite being blind to condition. We present these results and discuss their implications for music technology and JA research more generally.

Beyond Dyadic Coordination: Multimodal Behavioral Irregularity in Triads Predicts Facets of Collaborative Problem Solving

We hypothesize that effective collaboration is facilitated when individuals and environmental components form a synergy where they work together and regulate one another to produce stable patterns of behavior, or regularity, as well as adaptively reorganize to form new behaviors, or irregularity. We tested this hypothesis in a study with 32 triads who collaboratively solved a challenging visual computer programming task for 20 min following an introductory warm-up phase. Multidimensional recurrence quantification analysis was used to examine fine-grained (i.e., every 10 s) collective patterns of regularity across team members' speech rate, body movement, and team interaction with the shared user interface. We found that teams exhibited significant patterns of regularity as compared to shuffled baselines, but there were no systematic trends in regularity across time. We also found that periods of regularity were associated with a reduction in overall behavior. Notably, the production of irregular behavior predicted expert-coded metrics of collaborative activity, such as teams' ability to construct shared knowledge and effectively negotiate and coordinate execution of solutions, net of overall behavioral production and behavioral self-similarity. Our findings support the theory that groups can interact to form interpersonal synergies and indicate that information about system-level dynamics is a viable way to understand and predict effective collaborative processes.

How does gaze to faces support face-to-face interaction? A review and perspective

Gaze-where one looks, how long, and when-plays an essential part in human social behavior. While many aspects of social gaze have been reviewed, there is no comprehensive review or theoretical framework that describes how gaze to faces supports face-to-face interaction. In this review, I address the following questions: (1) When does gaze need to be allocated to a particular region of a face in order to provide the relevant information for successful interaction; (2) How do humans look at other people, and faces in particular, regardless of whether gaze needs to be directed at a particular region to acquire the relevant visual information; (3) How does gaze support the regulation of interaction? The work reviewed spans psychophysical research, observational research, and eye-tracking research in both lab-based and interactive contexts. Based on the literature overview, I sketch a framework for future research based on dynamic systems theory. The framework holds that gaze should be investigated in relation to sub-states of the interaction, encompassing sub-states of the interactors, the content of the interaction as well as the interactive context. The relevant sub-states for understanding gaze in interaction vary over different timescales from microgenesis to ontogenesis and phylogenesis. The framework has important implications for vision science, psychopathology, developmental science, and social robotics.

Body synchrony in triadic interaction

Humans subtly synchronize body movement during face-to-face conversation. In this context, bodily synchrony has been linked to affiliation and social bonding, task success and comprehension, and potential conflict. Almost all studies of conversational synchrony involve dyads, and relatively less is known about the structure of synchrony in groups larger than two. We conducted an optic flow analysis of body movement in triads engaged in face-to-face conversation, and explored a common measure of synchrony: time-aligned bodily covariation. We correlated this measure of synchrony with a diverse set of covariates related to the outcome of interactions. Triads showed higher maximum cross-correlation relative to a surrogate baseline, and 'meta-synchrony', in that composite dyads in a triad tended to show correlated structure. A windowed analysis also revealed that synchrony varies widely across an interaction. As in prior studies, average synchrony was low but statistically reliable in just a few minutes of interaction. In an exploratory analysis, we investigated the potential function of body synchrony by predicting it from various covariates, such as linguistic style matching, liking, laughter and cooperative play in a behavioural economic game. Exploratory results do not reveal a clear function for synchrony, though colaughter within triads was associated with greater body synchrony, and is consistent with an earlier analysis showing a positive connection between colaughter and cooperation. We end by discussing the importance of expanding and codifying analyses of synchrony and assessing its function.

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the non-linear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique's application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a "Programming Basics" module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.

Tilting Together: An Information-Theoretic Characterization of Behavioral Roles in Rhythmic Dyadic Interaction

Every joint collaborative physical activity performed by a group of people, e.g., carrying a table, typically leads to the emergence of spatiotemporal coordination of individual motor behavior. Such interpersonal coordination can arise solely based on the observation of the partners' and/or the object's movements, without the presence of verbal communication. In this paper, we investigate how the social coupling between two individuals in a collaborative task translates into measured objective and subjective performance indicators recorded in two different studies. We analyse the trends in the dyadic interrelationship based on the information-theoretic measure of transfer entropy and identify emerging leader-follower roles. In our experimental paradigm, the actions of the pair of subjects are continuously and seamlessly fused, resulting in a joint control of an object simulated on a tablet computer. Subjects need to synchronize their movements with a 90° phase difference in order to keep the object (a ball) rotating precisely on a predefined circular or elliptic trajectory on a tablet device. Results demonstrate how the identification of causal dependencies in this social interaction task could reveal specific trends in human behavior and provide insights into the emergence of social sensorimotor contingencies.

Interpersonal Coordination Dynamics in Psychotherapy: A Systematic Review

Background

The aim of this study was to gain a better understanding of interpersonal coordination (i.e., covariation over time) in different modalities (e.g., physiology, language) during psychotherapy and their importance for understanding the dynamics of psychotherapeutic interaction and efficacy.

Methods

We conducted a systematic review of all studies examining some form of interpersonal coordination in a psychotherapeutic context.

Results

We first summarize details of the included studies such as gender composition, therapy types, and methods used. The collation of these studies provided evidence that, during psychotherapeutic contexts, interpersonal coordination occurs in physiology, movements, interpersonal displays, and language/vocalizations. Further, it also showed that movement coordination was most frequently associated with psychotherapy outcomes, physiological coordination was most frequently associated with empathy, and coordination in a variety of modalities including language/vocalizations were most frequently associated with therapeutic alliance.

Conclusions

We discuss these results, shortcomings with the current literature, and highlight three crucial questions for future research. Research on interpersonal coordination in psychotherapy has potential to advance the both the research and practice of psychotherapy.

The Role of the Speaker, the Listener, and Their Joint Contributions During Communicative Interactions: A Tripartite View of Intelligibility in Individuals With Dysarthria

Purpose We present a tripartite view of intelligibility in which the contributions of both the speaker and listener, as well as their joint effort during interaction, are considered. While considerable research has examined communicative interactions in situ, there is a critical gap in current knowledge on how speech intelligibility unfolds during such interactions. Here, we argue that research examining speech intelligibility in communicative interactions may provide important groundwork for advancement in clinical interventions for individuals with dysarthria. Method First, we describe the view and argue for its consideration as a powerful way of thinking about speech intelligibility. We then briefly situate the view in the relevant literature on speech intelligibility and existing theoretical frameworks. We then identify suitable methodological paradigms for studying joint contributions to intelligibility and, lastly, discuss the clinical application and potential impact of this tripartite view. Conclusions Speech communication occurs through interaction; however, in the laboratory and clinic, emphasis is usually placed on individual speakers and listeners. We have proposed that it is critical to consider how the joint contributions of speakers and listeners affect speech intelligibility in communicative interaction. This conceptualization is well aligned with the International Classification of Functioning, Disability and Health, and the findings from such an approach will allow us to better understand how to maximize available resources to enhance speech intelligibility.

Reduced nonverbal interpersonal synchrony in autism spectrum disorder independent of partner diagnosis: a motion energy study

BACKGROUND

One of the main diagnostic features of individuals with autism spectrum disorders is nonverbal behaviour difficulties during naturalistic social interactions. The 'Interactional Heterogeneity Hypothesis' of ASD proposes that the degree to which individuals share a common ground substantially influences their ability to achieve smooth social interactions.

METHODS

To test this hypothesis, we filmed 29 autistic and 29 matched typically developed adults engaged in several conversational tasks. Windowed cross-lagged correlations were computed using the time series of motion energy of both individuals in a dyad. These coefficients were then compared across the three dyad types that were homo- or heterogenous with respect to diagnosis: pairs of two autistic individuals, two typically developed individuals or pairs of one autistic and one typically developed person.

RESULTS

We found that all dyad types achieved above-chance interpersonal synchrony, but that synchrony was more expressed in typical dyads compared to both autistic and mixed dyads.

LIMITATIONS

The method presented here provides only one, albeit objective and robust, approach to explore synchrony. The methodological choices as well as the lack of consideration for other communication modalities may limit our interpretation of the findings. Moreover, the sample size is small with respect to exploring associations between synchrony and various outcome and social skill measures.

CONCLUSIONS

The present results do not provide support for the Interactional Heterogeneity Hypothesis given that autistic individuals do not coordinate better when interacting with another autistic individual, compared to when interacting with a typical individual.

Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis

Voice atypicalities have been a characteristic feature of schizophrenia since its first definitions. They are often associated with core negative symptoms such as flat affect and alogia, and with the social impairments seen in the disorder. This suggests that voice atypicalities may represent a marker of clinical features and social functioning in schizophrenia. We systematically reviewed and meta-analyzed the evidence for distinctive acoustic patterns in schizophrenia, as well as their relation to clinical features. We identified 46 articles, including 55 studies with a total of 1254 patients with schizophrenia and 699 healthy controls. Summary effect sizes (Hedges'g and Pearson's r) estimates were calculated using multilevel Bayesian modeling. We identified weak atypicalities in pitch variability (g = -0.55) related to flat affect, and stronger atypicalities in proportion of spoken time, speech rate, and pauses (g's between -0.75 and -1.89) related to alogia and flat affect. However, the effects were mostly modest (with the important exception of pause duration) compared to perceptual and clinical judgments, and characterized by large heterogeneity between studies. Moderator analyses revealed that tasks with a more demanding cognitive and social component showed larger effects both in contrasting patients and controls and in assessing symptomatology. In conclusion, studies of acoustic patterns are a promising but, yet unsystematic avenue for establishing markers of schizophrenia. We outline recommendations towards more cumulative, open, and theory-driven research.

This is for you: Social modulations of proximal vs. distal space in collaborative interaction

Human spatial representations are shaped by affordances for action offered by the environment. A prototypical example is the organization of space into peripersonal (within reach) and extrapersonal (outside reach) regions, mirrored by proximal (this/here) and distal (that/there) linguistic expressions. The peri-/extrapersonal distinction has been widely investigated in individual contexts, but little is known about how spatial representations are modulated by interaction with other people. Is near/far coding of space dynamically adapted to the position of a partner when space, objects, and action goals are shared? Over two preregistered experiments based on a novel interactive paradigm, we show that, in individual and social contexts involving no direct collaboration, linguistic coding of locations as proximal or distal depends on their distance from the speaker's hand. In contrast, in the context of collaborative interactions involving turn-taking and role reversal, proximal space is shifted towards the partner, and linguistic coding of near space ('this' / 'here') is remapped onto the partner's action space.

Multiscale movement coordination dynamics in collaborative team problem solving

During collaborative problem solving (CPS), coordination occurs at different spatial and temporal scales. This multiscale coordination should play a functional role in facilitating effective collaboration. To evaluate this, we conducted a study of computer-based CPS with 42 dyadic teams. We used cross-wavelet coherence to examine movement coordination, extracted from videos, at several scales, and tested whether the observed coordination was greater than expected due to chance and due to task demands. We found that coordination at scales less than 2s was greater than chance and at most scales (except 16s, 1m, and 2m), was greater than expected due to task demands. Lastly, we observed that coherence at .25s and 1s scales was predictive of performance. However, when including relative phase, our results suggest that higher in-phase movement coordination at the 1s scale was the strongest predictor of CPS performance. Further, we used growth curve modeling to examine how movement coordination changes across the duration of the task and whether this is moderated by CPS performance. We found that coordination over the duration of the CPS task is quadratic (a U shape) and that better performing teams have higher coordination with a shallower curve. We discuss these findings and their relevance to understanding how low-level movement coordination facilitates CPS.

Beyond frequency counts: Novel conceptual recurrence analysis metrics to index semantic coordination in team communications

Semantic alignment is a key process underlying interpersonal and team communication. However, semantic similarity is difficult to quantify, and statistical approaches designed to measure it often rely on methods that make the identification of the relative importance of key words difficult. This study outlines how conceptual recurrence analysis (CRA) can address these issues and can be used to detect conceptual structure in interpersonal communication. We developed several novel CRA metrics to analyze communication data reported previously by Mancuso, Finomore, Rahill, Blair, and Funke (Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58, 405-409, 2014), gathered from teams who worked cooperatively on a logic puzzle under different cognitive biasing contexts. CRA, like other measures of semantic coordination, relies on parameters whose values affect estimates of semantic alignment. We evaluated how the dimensionality of semantic spaces affects metrics quantifying the conceptual similarity of communicative exchanges, and whether metrics calculated from top-down, a priori semantic spaces or bottom-up semantic spaces empirically derived from each data set were more sensitive to biasing context. We found that the novel CRA measures were sensitive to manipulations of cognitive bias, and that higher-dimensional, bottom-up semantic spaces generally yielded more sensitivity to the experimental manipulations, though when the communication was evaluated with respect to specific key concepts, lower-dimensional, top-down spaces performed nearly as well. We conclude that CRA is sensitive to experimental manipulations in ways consistent with prior findings and that it presents a customizable framework for testing predictions about interpersonal communication patterns and other linguistic exchanges.

Multidimensional Cross-Recurrence Quantification Analysis (MdCRQA) - A Method for Quantifying Correlation between Multivariate Time-Series

In this paper, Multidimensional Cross-Recurrence Quantification Analysis (MdCRQA) is introduced. It is an extension of Multidimensional Recurrence Quantification Analysis (MdRQA), which allows to quantify the (auto-)recurrence properties of a single multidimensional time-series. MdCRQA extends MdRQA to bi-variate cases to allow for the quantification of the co-evolution of two multidimensional time-series. Moreover, it is shown how a Diagonal Cross-Recurrence Profile (DCRP) can be computed from the MdCRQA output that allows to capture time-lagged coupling between two multidimensional time-series. The core concepts of these analyses are described, as well as practical aspects of their application. In the supplementary materials to this paper, implementations of MdCRQA and the DCRP as MatLab- and R-functions are provided.

Hearing me hearing you: Reciprocal effects between child and parent language in autism and typical development

Language development in typically developing children (TD) has traditionally been investigated in relation to environmental factors, while language in children with autism spectrum disorder (ASD) has primarily been related to child-based factors. We employ a longitudinal corpus of 32 preschoolers with ASD and 35 linguistically matched TD peers recorded over 6 visits (ranging between 2 and 5 years of age) to investigate the relative importance of child-based and environmental factors in language development for both populations. We also investigate the reciprocal interaction between children's response to parents' input, and parents' response to children's production. We report six major findings. (1) Children's production of word types, tokens, and MLU increased across visits, and were predicted by their Expressive Language (EL) (positively) and diagnosis (negatively) from Visit 1. (2) Parents' production also increased across visits, and was predicted by their child's nonverbal cognition (positively) and diagnosis (negatively) from Visit 1. (3) At all visits and across groups, children and parents matched each other in lexical and syntactic production; (4) Parents who produced longer MLUs during a given visit had children who produced more word types and tokens, and had longer MLUs, at the subsequent visit. (5) When both child EL at Visit 1 and parent MLU were included in the model, both contributed significantly to future child language; however, EL accounted for a greater proportion of the variance. (6) Finally, children's speech significantly predicted parent speech at the next visit. Taken together, these results draw more attention to the importance of child-based factors in the early language development of TD children, and to the importance of parental language factors in the early language development of children with ASD.

Communicative misalignment in Autism Spectrum Disorder

Communication deficits are a defining feature of Autism Spectrum Disorder (ASD), manifest during social interactions. Previous studies investigating communicative deficits have largely focused on the perceptual biases, social motivation, cognitive flexibility, or mentalizing abilities of isolated individuals. By embedding autistic individuals in live non-verbal interactions, we characterized a novel cause for their communication deficits. Adults with ASD matched neurotypical individuals in their ability and propensity to generate and modify intelligible behaviors for a communicative partner. However, they struggled to align the meaning of those behaviors with their partner when meaning required referencing their recent communicative history. This communicative misalignment explains why autistic individuals are vulnerable in everyday interactions, which entail fleeting ambiguities, but succeed in social cognition tests involving stereotyped contextual cues. These findings illustrate the cognitive and clinical importance of considering social interaction as a communicative alignment challenge, and how ineffective human communication is without this key interactional ingredient.

Procedural Coordination in the Matching Task

Participants in conversation who recurrently discuss the same targets require fewer and fewer words to identify them. This has been attributed to the collaborative elaboration of conceptual pacts, that is, semantic coordination. But participants do not only coordinate on the semantics of referring expressions; they also coordinate on how to do the task, that is, on procedural coordination. In a matching task experiment (n = 22 dyads), we examined the development of four aspects of procedural coordination: Card placement (CP), implicit generic coordination (IGC), explicit generic coordination (EGC) and general procedural coordination (GPC) in two conditions (the classic condition where targets remain the same over trials, and a new cards condition, where they change at each trial, thus increasing the difficulty of semantic coordination). Procedural coordination constituted almost 30% of the total amount of talk in the matching task. Procedural coordination was more effortful when semantic coordination was more difficult and the four aspects of procedural coordination developed differently depending on participant roles.

Analyzing Multivariate Dynamics Using Cross-Recurrence Quantification Analysis (CRQA), Diagonal-Cross-Recurrence Profiles (DCRP), and Multidimensional Recurrence Quantification Analysis (MdRQA) - A Tutorial in R

This paper provides a practical, hands-on introduction to cross-recurrence quantification analysis (CRQA), diagonal cross-recurrence profiles (DCRP), and multidimensional recurrence quantification analysis (MdRQA) in R. These methods have enjoyed increasing popularity in the cognitive and social sciences since a recognition that many behavioral and neurophysiological processes are intrinsically time dependent and reliant on environmental and social context has emerged. Recurrence-based methods are particularly suited for time-series that are non-stationary or have complicated dynamics, such as longer recordings of continuous physiological or movement data, but are also useful in the case of time-series of symbolic data, as in the case of text/verbal transcriptions or categorically coded behaviors. In the past, they have been used to assess changes in the dynamics of, or coupling between physiological and behavioral measures, for example in joint action research to determine the co-evolution of the behavior between individuals in dyads or groups, or for assessing the strength of coupling/correlation between two or more time-series. In this paper, we provide readers with a conceptual introduction, followed by a step-by-step explanation on how the analyses are performed in R with a summary of the current best practices of their application.

Calculation of Average Mutual Information (AMI) and False-Nearest Neighbors (FNN) for the Estimation of Embedding Parameters of Multidimensional Time Series in Matlab

Using the method or time-delayed embedding, a signal can be embedded into higher-dimensional space in order to study its dynamics. This requires knowledge of two parameters: The delay parameter τ, and the embedding dimension parameter D. Two standard methods to estimate these parameters in one-dimensional time series involve the inspection of the Average Mutual Information (AMI) function and the False Nearest Neighbor (FNN) function. In some contexts, however, such as phase-space reconstruction for Multidimensional Recurrence Quantification Analysis (MdRQA), the empirical time series that need to be embedded already possess a dimensionality higher than one. In the current article, we present extensions of the AMI and FNN functions for higher dimensional time series and their application to data from the Lorenz system coded in Matlab.

Group benefits in joint perceptual tasks-a review

In daily life, humans often perform perceptual tasks together to reach a shared goal. In these situations, individuals may collaborate (e.g., by distributing task demands) to perform the task better than when the task is performed alone (i.e., attain a group benefit). In this review, we identify the factors influencing if, and to what extent, a group benefit is attained and provide a framework of measures to assess group benefits in perceptual tasks. In particular, we integrate findings from two frequently investigated joint perceptual tasks: visuospatial tasks and decision-making tasks. For both task types, we find that an exchange of information between coactors is critical to improve joint performance. Yet, the type of exchanged information and how coactors collaborate differs between tasks. In visuospatial tasks, coactors exchange information about the performed actions to distribute task demands. In perceptual decision-making tasks, coactors exchange their confidence on their individual perceptual judgments to negotiate a joint decision. We argue that these differences can be explained by the task structure: coactors distribute task demands if a joint task allows for a spatial division and stimuli can be accurately processed by one individual. Otherwise, they perform the task individually and then integrate their individual judgments.

Bridgemanian space constancy as a precursor to extended cognition

A few decades ago, cognitive psychologists generally took for granted that the reason we perceive our visual environment as one contiguous stable whole (i.e., space constancy) is because we have an internal mental representation of the visual environment as one contiguous stable whole. They supposed that the non-contiguous visual images that are gathered during the brief fixations that intervene between pairs of saccadic eye movements (a few times every second) are somehow stitched together to construct this contiguous internal mental representation. Determining how exactly the brain does this proved to be a vexing puzzle for vision researchers. Bruce Bridgeman's research career is the story of how meticulous psychophysical experimentation, and a genius theoretical insight, eventually solved this puzzle. The reason that it was so difficult for researchers to figure out how the brain stitches together these visual snapshots into one accurately-rendered mental representation of the visual environment is that it doesn't do that. Bruce discovered that the brain couldn't do that if it tried. The neural information that codes for saccade amplitude and direction is simply too inaccurate to determine exact relative locations of each fixation. Rather than the perception of space constancy being the result of an internal representation, Bruce determined that it is the result of a brain that simply assumes that external space remains constant, and it rarely checks to verify this assumption. In our extension of Bridgeman's formulation, we suggest that objects in the world often serve as their own representations, and cognitive operations can be performed on those objects themselves, rather than on mental representations of them.

The Bursts and Lulls of Multimodal Interaction: Temporal Distributions of Behavior Reveal Differences Between Verbal and Non-Verbal Communication

Recent studies of naturalistic face-to-face communication have demonstrated coordination patterns such as the temporal matching of verbal and non-verbal behavior, which provides evidence for the proposal that verbal and non-verbal communicative control derives from one system. In this study, we argue that the observed relationship between verbal and non-verbal behaviors depends on the level of analysis. In a reanalysis of a corpus of naturalistic multimodal communication (Louwerse, Dale, Bard, & Jeuniaux, ), we focus on measuring the temporal patterns of specific communicative behaviors in terms of their burstiness. We examined burstiness estimates across different roles of the speaker and different communicative modalities. We observed more burstiness for verbal versus non-verbal channels, and for more versus less informative language subchannels. Using this new method for analyzing temporal patterns in communicative behaviors, we show that there is a complex relationship between verbal and non-verbal channels. We propose a "temporal heterogeneity" hypothesis to explain how the language system adapts to the demands of dialog.

How to Create Shared Symbols

Human cognition and behavior are dominated by symbol use. This paper examines the social learning strategies that give rise to symbolic communication. Experiment 1 contrasts an individual-level account, based on observational learning and cognitive bias, with an inter-individual account, based on social coordinative learning. Participants played a referential communication game in which they tried to communicate a range of recurring meanings to a partner by drawing, but without using their conventional language. Individual-level learning, via observation and cognitive bias, was sufficient to produce signs that became increasingly effective, efficient, and shared over games. However, breaking a referential precedent eliminated these benefits. The most effective, most efficient, and most shared signs arose when participants could directly interact with their partner, indicating that social coordinative learning is important to the creation of shared symbols. Experiment 2 investigated the contribution of two distinct aspects of social interaction: behavior alignment and concurrent partner feedback. Each played a complementary role in the creation of shared symbols: Behavior alignment primarily drove communication effectiveness, and partner feedback primarily drove the efficiency of the evolved signs. In conclusion, inter-individual social coordinative learning is important to the evolution of effective, efficient, and shared symbols.

Coordinated Interpersonal Behaviour in Collective Dance Improvisation: The Aesthetics of Kinaesthetic Togetherness

Collective dance improvisation (e.g., traditional and social dancing, contact improvisation) is a participatory, relational and embodied art form which eschews standard concepts in aesthetics. We present our ongoing research into the mechanisms underlying the lived experience of “togetherness” associated with such practices. Togetherness in collective dance improvisation is kinaesthetic (based on movement and its perception), and so can be simultaneously addressed from the perspective of the performers and the spectators, and be measured. We utilise these multiple levels of description: the first-person, phenomenological level of personal experiences, the third-person description of brain and body activity, and the level of interpersonal dynamics. Here, we describe two of our protocols: a four-person mirror game and a ‘rhythm battle’ dance improvisation score. Using an interpersonal closeness measure after the practice, we correlate subjective sense of individual/group connectedness and observed levels of in-group temporal synchronization. We propose that kinaesthetic togetherness, or interpersonal resonance, is integral to the aesthetic pleasure of the participants and spectators, and that embodied feeling of togetherness might play a role more generally in aesthetic experience in the performing arts.

Editor's Introduction and Review: Coordination and Context in Cognitive Science

The role of coordination in cognitive science has been on the rise in recent years, in terms of coordination among neurons, coordination among sensory and motor systems, and coordination among individuals. Research has shown that coordination patterns corresponding to cognitive activities depend on the various contexts in which the underlying interactions are situated. The present issue of Topics in Cognitive Science centers on studies of coordination that address the role of context in shaping or interpreting dynamical patterns of human behavior. This introductory article reviews some of the prior literature leading up to current and future research on coordination and context in cognitive science.

Interpersonal Movement Synchrony Responds to High- and Low-Level Conversational Constraints

Much work on communication and joint action conceptualizes interaction as a dynamical system. Under this view, dynamic properties of interaction should be shaped by the context in which the interaction is taking place. Here we explore interpersonal movement coordination or synchrony-the degree to which individuals move in similar ways over time-as one such context-sensitive property. Studies of coordination have typically investigated how these dynamics are influenced by either high-level constraints (i.e., slow-changing factors) or low-level constraints (i.e., fast-changing factors like movement). Focusing on nonverbal communication behaviors during naturalistic conversation, we analyzed how interacting participants' head movement dynamics were shaped simultaneously by high-level constraints (i.e., conversation type; friendly conversations vs. arguments) and low-level constraints (i.e., perceptual stimuli; non-informative visual stimuli vs. informative visual stimuli). We found that high- and low-level constraints interacted non-additively to affect interpersonal movement dynamics, highlighting the context sensitivity of interaction and supporting the view of joint action as a complex adaptive system.

Conversation level syntax similarity metric

The syntax and semantics of human language can illuminate many individual psychological differences and important dimensions of social interaction. Accordingly, psychological and psycholinguistic research has begun incorporating sophisticated representations of semantic content to better understand the connection between word choice and psychological processes. In this work we introduce ConversAtion level Syntax SImilarity Metric (CASSIM), a novel method for calculating conversation-level syntax similarity. CASSIM estimates the syntax similarity between conversations by automatically generating syntactical representations of the sentences in conversation, estimating the structural differences between them, and calculating an optimized estimate of the conversation-level syntax similarity. After introducing and explaining this method, we report results from two method validation experiments (Study 1) and conduct a series of analyses with CASSIM to investigate syntax accommodation in social media discourse (Study 2). We run the same experiments using two well-known existing syntactic metrics, LSM and Coh-Metrix, and compare their results to CASSIM. Overall, our results indicate that CASSIM is able to reliably measure syntax similarity and to provide robust evidence of syntax accommodation within social media discourse.

Conversing with a devil's advocate: Interpersonal coordination in deception and disagreement

This study investigates the presence of dynamical patterns of interpersonal coordination in extended deceptive conversations across multimodal channels of behavior. Using a novel "devil's advocate" paradigm, we experimentally elicited deception and truth across topics in which conversational partners either agreed or disagreed, and where one partner was surreptitiously asked to argue an opinion opposite of what he or she really believed. We focus on interpersonal coordination as an emergent behavioral signal that captures interdependencies between conversational partners, both as the coupling of head movements over the span of milliseconds, measured via a windowed lagged cross correlation (WLCC) technique, and more global temporal dependencies across speech rate, using cross recurrence quantification analysis (CRQA). Moreover, we considered how interpersonal coordination might be shaped by strategic, adaptive conversational goals associated with deception. We found that deceptive conversations displayed more structured speech rate and higher head movement coordination, the latter with a peak in deceptive disagreement conversations. Together the results allow us to posit an adaptive account, whereby interpersonal coordination is not beholden to any single functional explanation, but can strategically adapt to diverse conversational demands.

Two Trackers Are Better than One: Information about the Co-actor's Actions and Performance Scores Contribute to the Collective Benefit in a Joint Visuospatial Task

When humans collaborate, they often distribute task demands in order to reach a higher performance compared to performing the same task alone (i.e., a collective benefit). Here, we tested to what extent receiving information about the actions of a co-actor, performance scores, or receiving both types of information impacts the collective benefit in a collaborative multiple object tracking task. In a between-subject design, pairs of individuals jointly tracked a subset of target objects among several moving distractor objects on a computer screen for a 100 trials. At the end of a trial, pairs received performance scores (Experiment 1), information about their partner's target selections (Experiment 2), or both types of information (Experiment 3). In all experiments, the performance of the pair exceeded the individual performances and the simulated performance of two independent individuals combined. Initially, when receiving both types of information (Experiment 3), pairs achieved the highest performance and divided task demands most efficiently compared to the other two experiments. Over time, performances and the ability to divide task demands for pairs receiving a single type of information converged with those receiving both, suggesting that pairs' coordination strategies become equally effective over time across experiments. However, pairs' performances never reached a theoretical limit of performance in all experiments. For distributing task demands, members of a pair predominantly used a left-right division of labor strategy (i.e., the leftmost targets were tracked by one co-actor while the rightmost targets were tracked by the other co-actor). Overall, findings of the present study suggest that receiving information about actions of a co-actor, performance scores, or receiving both enables pairs to devise effective division of labor strategies in a collaborative visuospatial task. However, when pairs had both types of information available, the formation of division of labor strategies was facilitated, indicating that pairs benefited the most from having both types of information available (i.e., actions about the co-actor and performance scores). Findings are applicable to circumstances in which humans need to perform collaborative visuospatial tasks that are time-critical and/or only allow a very limited exchange of information between co-actors.

Problem-Solving Phase Transitions During Team Collaboration

Multiple theories of problem-solving hypothesize that there are distinct qualitative phases exhibited during effective problem-solving. However, limited research has attempted to identify when transitions between phases occur. We integrate theory on collaborative problem-solving (CPS) with dynamical systems theory suggesting that when a system is undergoing a phase transition it should exhibit a peak in entropy and that entropy levels should also relate to team performance. Communications from 40 teams that collaborated on a complex problem were coded for occurrence of problem-solving processes. We applied a sliding window entropy technique to each team's communications and specified criteria for (a) identifying data points that qualify as peaks and (b) determining which peaks were robust. We used multilevel modeling, and provide a qualitative example, to evaluate whether phases exhibit distinct distributions of communication processes. We also tested whether there was a relationship between entropy values at transition points and CPS performance. We found that a proportion of entropy peaks was robust and that the relative occurrence of communication codes varied significantly across phases. Peaks in entropy thus corresponded to qualitative shifts in teams' CPS communications, providing empirical evidence that teams exhibit phase transitions during CPS. Also, lower average levels of entropy at the phase transition points predicted better CPS performance. We specify future directions to improve understanding of phase transitions during CPS, and collaborative cognition, more broadly.

A reduced-dimensionality approach to uncovering dyadic modes of body motion in conversations

Face-to-face conversations are central to human communication and a fascinating example of joint action. Beyond verbal content, one of the primary ways in which information is conveyed in conversations is body language. Body motion in natural conversations has been difficult to study precisely due to the large number of coordinates at play. There is need for fresh approaches to analyze and understand the data, in order to ask whether dyads show basic building blocks of coupled motion. Here we present a method for analyzing body motion during joint action using depth-sensing cameras, and use it to analyze a sample of scientific conversations. Our method consists of three steps: defining modes of body motion of individual participants, defining dyadic modes made of combinations of these individual modes, and lastly defining motion motifs as dyadic modes that occur significantly more often than expected given the single-person motion statistics. As a proof-of-concept, we analyze the motion of 12 dyads of scientists measured using two Microsoft Kinect cameras. In our sample, we find that out of many possible modes, only two were motion motifs: synchronized parallel torso motion in which the participants swayed from side to side in sync, and still segments where neither person moved. We find evidence of dyad individuality in the use of motion modes. For a randomly selected subset of 5 dyads, this individuality was maintained for at least 6 months. The present approach to simplify complex motion data and to define motion motifs may be used to understand other joint tasks and interactions. The analysis tools developed here and the motion dataset are publicly available.

Mirroring and beyond: coupled dynamics as a generalized framework for modelling social interactions

When people observe one another, behavioural alignment can be detected at many levels, from the physical to the mental. Likewise, when people process the same highly complex stimulus sequences, such as films and stories, alignment is detected in the elicited brain activity. In early sensory areas, shared neural patterns are coupled to the low-level properties of the stimulus (shape, motion, volume, etc.), while in high-order brain areas, shared neural patterns are coupled to high-levels aspects of the stimulus, such as meaning. Successful social interactions require such alignments (both behavioural and neural), as communication cannot occur without shared understanding. However, we need to go beyond simple, symmetric (mirror) alignment once we start interacting. Interactions are dynamic processes, which involve continuous mutual adaptation, development of complementary behaviour and division of labour such as leader-follower roles. Here, we argue that interacting individuals are dynamically coupled rather than simply aligned. This broader framework for understanding interactions can encompass both processes by which behaviour and brain activity mirror each other (neural alignment), and situations in which behaviour and brain activity in one participant are coupled (but not mirrored) to the dynamics in the other participant. To apply these more sophisticated accounts of social interactions to the study of the underlying neural processes we need to develop new experimental paradigms and novel methods of data analysis.

Beyond Synchrony: Joint Action in a Complex Production Task Reveals Beneficial Effects of Decreased Interpersonal Synchrony

A variety of joint action studies show that people tend to fall into synchronous behavior with others participating in the same task, and that such synchronization is beneficial, leading to greater rapport, satisfaction, and performance. It has been noted that many of these task environments require simple interactions that involve little planning of action coordination toward a shared goal. The present study utilized a complex joint construction task in which dyads were instructed to build model cars while their hand movements and heart rates were measured. Participants built these models under varying conditions, delimiting how freely they could divide labor during a build session. While hand movement synchrony was sensitive to the different tasks and outcomes, the heart rate measure did not show any effects of interpersonal synchrony. Results for hand movements show that the more participants were constrained by a particular building strategy, the greater their behavioral synchrony. Within the different conditions, the degree of synchrony was predictive of subjective satisfaction and objective product outcomes. However, in contrast to many previous findings, synchrony was negatively associated with superior products, and, depending on the constraints on the interaction, positively or negatively correlated with higher subjective satisfaction. These results show that the task context critically shapes the role of synchronization during joint action, and that in more complex tasks, not synchronization of behavior, but rather complementary types of behavior may be associated with superior task outcomes.