Action Co-representation is Tuned to Other Humans

Joint spatial-temporal association of response codes (STEARC) effect: Mental timelines embodied interpersonally

A co-action task was used to explore the effect of social interactions on temporal judgements, in comparison with an individual-task condition. In Experiment 1, the co-actors sat either individually (individual condition) or alongside a partner (joint condition) in front of a monitor and then responded to time-related words (e.g. yesterday, tomorrow). In Experiment 2, co-actors sat separately in front of two monitors and categorized the words either individually or jointly. Participants' response times to past- and future-related words in the individual conditions of both experiments had no significant difference. However, in the joint conditions, the responses were faster when the past-time words were mapped toward the participants on the left than when future-time words were mapped toward them. Our data support the existence of a specific mapping between past-time-left space and future-time-right space. This suggests that the two cooperators probably shared a similar mental timeline.

Modulations of one's sense of agency during human-machine interactions: A behavioural study using a full humanoid robot

Although previous investigations reported a reduced sense of agency when individuals act with traditional machines, little is known about the mechanisms underpinning interactions with human-like automata. The aim of this study was twofold: (1) to investigate the effect of the machine's physical appearance on the individuals' sense of agency and (2) to explore the cognitive mechanisms underlying the individuals' sense of agency when they are engaged in a joint task. Twenty-eight participants performed a joint Simon task together with another human or an automated artificial system as a co-agent. The physical appearance of the automated artificial system was manipulated so that participants could cooperate either with a servomotor or a full humanoid robot during the joint task. Both participants' response times and temporal estimations of action-output delays (i.e., an implicit measure of agency) were collected. Results showed that participants' sense of agency for self- and other-generated actions sharply declined during interactions with the servomotor compared with the human-human interactions. Interestingly, participants' sense of agency for self- and other-generated actions was reinforced when participants interacted with the humanoid robot compared with the servomotor. These results are discussed further.

Co-actors represent each other's task regularity through social statistical learning

Numerous joint action studies have demonstrated that certain low-level aspects (e.g., stimuli and responses) of a co-actor's task can be automatically and implicitly represented by us as actors, biasing our own task performance in a joint action setup. However, it remains unclear whether individuals also represent more abstract, high-level aspects of a co-actor's task, such as regularity. In the first five experiments, participants participated alongside their co-actors and responded to a mixed shape sequence generated by randomly interleaving two fixed order sequences of shapes in both the pre- and post-test sessions. But different intermediate practice sessions were undergone by participants across experiments. When practicing their own fixed order sequences in a mixed shape sequence, either together with another person (Experiment 1) or alone but informed that their partner was performing the same practice task in a different room (Experiment 4), participants exhibited a learning effect on their co-actors' practiced sequences. This indirect learning effect was absent when one of the co-actors did not participate due to either being removed from the practice (Experiment 2) or sitting still without offering responses (Experiment 3), as well as when the two co-actors practiced together but responded to two distinct properties of stimuli (e.g., colour and shape, respectively), with one having regularity and the other not. Finally, participants exhibited comparable direct learning effects on their own practiced sequences for Experiments 1-5 as when performing the pre-test, practice, and post-test sessions alone for Experiment 6. These results demonstrate that, when practicing together, or even when believing that they are acting together with a partner, co-actors do represent the task regularity of one another through social statistical learning and transfer this learned regularity to subsequent task performances. The present study extends our understanding of co-representation in the joint action context in terms of the more abstract and high-level task features people co-represent, such as a co-actor's task regularity.

Action prediction modulates self-other integration in joint action

People often coordinate actions with others, requiring an adjustable amount of self-other integration between actor's and co-actor's actions. Previous research suggests that such self-other integration (indexed by the joint Simon effect) is enhanced by agent similarity of the co-actor (e.g., high in intentionality). In this study, we aimed to extend this line of research by testing whether experiencing agency over a co-actor's actions (vicarious agency) and/or action prediction strengthens the joint Simon effect. For this purpose, we manipulated experienced agency by varying the experienced control over a co-actor's actions (Experiment 1), and action prediction regarding the co-actor's actions (Experiment 2). Vicarious agency could effectively be induced, but did not modulate the size of the joint Simon effect. The joint Simon effect was decreased when the co-actor's actions were unpredictable (vs. predictable) during joint task performance. These findings suggest social agency can be induced and effectively measured in joint action. Action prediction can act as an effective agency cue modulating the amount of self-other integration in joint action.

Human-like behavioral variability blurs the distinction between a human and a machine in a nonverbal Turing test

Variability is a property of biological systems, and in animals (including humans), behavioral variability is characterized by certain features, such as the range of variability and the shape of its distribution. Nevertheless, only a few studies have investigated whether and how variability features contribute to the ascription of humanness to robots in a human-robot interaction setting. Here, we tested whether two aspects of behavioral variability, namely, the standard deviation and the shape of distribution of reaction times, affect the ascription of humanness to robots during a joint action scenario. We designed an interactive task in which pairs of participants performed a joint Simon task with an iCub robot placed by their side. Either iCub could perform the task in a preprogrammed manner, or its button presses could be teleoperated by the other member of the pair, seated in the other room. Under the preprogrammed condition, the iCub pressed buttons with reaction times falling within the range of human variability. However, the distribution of the reaction times did not resemble a human-like shape. Participants were sensitive to humanness, because they correctly detected the human agent above chance level. When the iCub was controlled by the computer program, it passed our variation of a nonverbal Turing test. Together, our results suggest that hints of humanness, such as the range of behavioral variability, might be used by observers to ascribe humanness to a humanoid robot.

Reaction time coupling in a joint stimulus-response task: A matter of functional actions or likable agents?

Shaping one owns actions by observing others’ actions is driven by the deep-rooted mechanism of perception-action coupling. It typically occurs automatically, expressed as for example the unintentional synchronization of reaction times in interactive games. Theories on perception-action coupling highlight its benefits such as the joint coordination of actions to cooperatively perform tasks properly, the learning of novel actions from others, and the bonding with likable others. However, such functional aspects and how they shape perception-action coupling have never been compared quantitatively. Here we tested a total of hundred-fifteen participants that played a stimulus-response task while, in parallel, they observed videos of agents that played the exact same task several milliseconds in advance. We compared to what degree the reaction times of actions of agents, who varied their behavior in terms of functionality and likability in preceding prisoner dilemma games and quizzes, shape the reaction times of human test participants. To manipulate functionality and likability, we varied the predictability of cooperative behavior and correctness of actions of agents, respectively, resulting in likable (cooperative), dislikable (uncooperative), functional (correct actions), and dysfunctional (incorrect actions) agents. The results of three experiments showed that the participants’ reaction times correlated most with the reaction times of agents that expressed functional behavior. However, the likability of agents had no effects on reaction time correlations. These findings suggest that, at least in the current computer task, participants are more likely to adopt the timing of actions from people that perform correct actions than from people that they like.

The evolutionary origins of syntax: Event cognition in nonhuman primates

Languages tend to encode events from the perspective of agents, placing them first and in simpler forms than patients. This agent bias is mirrored by cognition: Agents are more quickly recognized than patients and generally attract more attention. This leads to the hypothesis that key aspects of language structure are fundamentally rooted in a cognition that decomposes events into agents, actions, and patients, privileging agents. Although this type of event representation is almost certainly universal across languages, it remains unclear whether the underlying cognition is uniquely human or more widespread in animals. Here, we review a range of evidence from primates and other animals, which suggests that agent-based event decomposition is phylogenetically older than humans. We propose a research program to test this hypothesis in great apes and human infants, with the goal to resolve one of the major questions in the evolution of language, the origins of syntax.

Impact of hierarchical processing levels on division of labor: neural electrophysiological evidence from a joint rule shifting

OBJECTIVES

Although previous studies have revealed the behavioral and neural mechanisms underlying cognitive control, the neural electrophysiological process in joint task switching has rarely been discussed. Moreover, the hierarchy process remains unclear. Therefore, the present study aimed to explore the neural mechanism of a target when two actors use division of labor between action selections in a joint hierarchical rule-shifting situation.

METHODS

The present study employed a joint hierarchical rule shifting paradigm, and each participant was responsible for the selection of one type of action (left key and right key). The cue was the letter 'R' with three features, which indicated the hierarchical rule of the current trial. The target was an Arabic numeral (1-9, except 5). Participants made a magnitude judgement or a parity judgment based on the high and low hierarchical features of cues.

RESULTS

The P2 component was smaller in the high-shift condition than in the repeat condition; however, the high-shift condition elicited a larger N2 amplitude than the low-shift and repeat conditions. We believe that the current P2 is related to selective attention to the hierarchical features of rules, and N2 is related to conflict control. In addition, the high-shift condition had the largest P3 amplitude, which indicates that this condition has the largest extent of update and leads to the most unstable association between rule and stimuli.

CONCLUSIONS

The study demonstrated that participants had dynamic brain responses to target stimuli in the joint rule-shifting situation. In other words, participants first confirmed the actor of the current target and then processed the target stimuli themselves.

Social Action Effects: Representing Predicted Partner Responses in Social Interactions

The sociomotor framework outlines a possible role of social action effects on human action control, suggesting that anticipated partner reactions are a major cue to represent, select, and initiate own body movements. Here, we review studies that elucidate the actual content of social action representations and that explore factors that can distinguish action control processes involving social and inanimate action effects. Specifically, we address two hypotheses on how the social context can influence effect-based action control: first, by providing unique social features such as body-related, anatomical codes, and second, by orienting attention towards any relevant feature dimensions of the action effects. The reviewed empirical work presents a surprisingly mixed picture: while there is indirect evidence for both accounts, previous studies that directly addressed the anatomical account showed no signs of the involvement of genuinely social features in sociomotor action control. Furthermore, several studies show evidence against the differentiation of social and non-social action effect processing, portraying sociomotor action representations as remarkably non-social. A focus on enhancing the social experience in future studies should, therefore, complement the current database to establish whether such settings give rise to the hypothesized influence of social context.

Individual differences in co-representation in three monkey species (Callithrix jacchus, Sapajus apella and Macaca tonkeana) in the joint Simon task: the role of social factors and inhibitory control

Behavioral coordination is involved in many forms of primate interactions. Co-representation is the simultaneous mental representation of one’s own and the partner’s task and actions. It often underlies behavioral coordination and cooperation success. In humans, the dyadic social context can modulate co-representation. Here, we first investigated whether individual differences in co-representation in the joint Simon task in capuchin monkeys and Tonkean macaques can be explained by social factors, namely dyadic grooming and sociality index, rank difference and eigenvector centrality. These factors did not predict variation in co-representation. However, in this specific task, co-representation reduces rather than facilitates joint performance. Automatic co-representation therefore needs to be inhibited or suppressed to maximize cooperation success. We therefore also investigated whether general inhibitory control (detour-reaching) would predict co-representation in the joint Simon task in Tonkean macaques, brown capuchin and marmoset monkeys. Inhibitory control did neither explain individual differences nor species differences, since marmosets were most successful in their joint performance despite scoring lowest on inhibitory control. These results suggest that the animals’ ability to resolve conflicts between self and other representation to increase cooperation success in this task is gradually learned due to frequent exposure during shared infant care, rather than determined by strong general inhibitory control. Further, we conclude that the joint Simon task, while useful to detect co-representation non-invasively, is less suitable for identifying the factors explaining individual differences and thus a more fruitful approach to identify these factors is to design tasks in which co-representation favors, rather than hinders cooperation success.

Spatial auditory presentation of a partner's presence induces the social Simon effect

Social presence is crucial for smooth communications in virtual reality (VR). Current telecommunication systems rarely submit spatial auditory information originating from remote people. However, such information may enhance social presence in VR. In this study, we constructed a dynamic binaural synthesis system and investigated the effect of spatial auditory information of a remote partner on a participant's behavior using the social Simon effect (SSE). The SSE is a spatial stimulus-response compatibility effect between two persons. The SSE occurs when one perceives that their partner is present. Several studies have confirmed the SSE in actual environments. We presented partner sounds diotically (i.e., without spatial information) to one group or binaurally (i.e., with spatial information) to another group through headphones without providing visual information about the partner. The results showed that the SSE was induced only in the binaural group in the current auditory VR (Experiment 1), whereas both groups exhibited the SSE in an actual environment (Experiment 2). These results suggest that the auditory spatial information of remote people is sufficient to induce the SSE and has a potential to enhance social presence.

Interference in the shared-Stroop task: a comparison of self- and other-monitoring

Co-actors represent and integrate each other's actions, even when they need not monitor one another. However, monitoring is important for successful interactions, particularly those involving language, and monitoring others' utterances probably relies on similar mechanisms as monitoring one's own. We investigated the effect of monitoring on the integration of self- and other-generated utterances in the shared-Stroop task. In a solo version of the Stroop task (with a single participant responding to all stimuli; Experiment 1), participants named the ink colour of mismatching colour words (incongruent stimuli) more slowly than matching colour words (congruent). In the shared-Stroop task, one participant named the ink colour of words in one colour (e.g. red), while ignoring stimuli in the other colour (e.g. green); the other participant either named the other ink colour or did not respond. Crucially, participants either provided feedback about the correctness of their partner's response (Experiment 3) or did not (Experiment 2). Interference was greater when both participants responded than when they did not, but only when their partners provided feedback. We argue that feedback increased interference because monitoring one's partner enhanced representations of the partner's target utterance, which in turn interfered with self-monitoring of the participant's own utterance.

Reluctance against the machine: Retrieval of observational stimulus-response episodes in online settings emerges when interacting with a human, but not with a computer partner

Observing how another person responds to a stimulus creates stimulus–response (SR) episodes. These can be retrieved from memory on later occasions, which means that observed responses are utilized for regulating one’s own actions. Until now, evidence for storage and retrieval of observationally acquired SR episodes was limited to dyadic face-to-face interactions between two partners who respond in an alternating fashion. In two preregistered studies (total N = 252), we demonstrate for the first time that observational SR episodes can also be acquired in online interactions: Robust retrieval effects emerged when observers believe to be interacting with another person. In turn, retrieval effects were absent when observers believe to be interacting with a computer. Our findings show that feature-based binding and retrieval principles are pervasive and also apply to social interactions, even under purely virtual conditions. We discuss implications of our findings for different explanatory accounts of social modulations of automatic imitation.

Joint Simon effect in movement trajectories

In joint action literature it is often assumed that acting together is driven by pervasive and automatic process of co-representation, that is, representing the co-actor's part of the task in addition to one's own. Much of this research employs joint stimulus-response compatibility tasks varying the stimuli employed or the physical and social relations between participants. In this study we test the robustness of co-representation effects by focusing instead on variation in response modality. Specifically, we implement a mouse-tracking version of a Joint Simon Task in which participants respond by producing continuous movements with a computer mouse rather than pushing discrete buttons. We have three key findings. First, in a replication of an earlier study we show that in a classical individual Simon Task movement trajectories show greater curvature on incongruent trials, paralleling longer response times. Second, this effect largely disappears in a Go-NoGo Simon Task, in which participants respond to only one of the cues and refrain from responding to the other. Third, contrary to previous studies that use button pressing responses, we observe no overall effect in the joint variants of the task. However, we also detect a notable diversity in movement strategies adopted by the participants, with some participants showing the effect on the individual level. Our study casts doubt on the pervasiveness of co-representation, highlights the usefulness of mouse-tracking methodology and emphasizes the need for looking at individual variation in task performance.

Number to me, space to you: Joint representation of spatial-numerical associations

Recent work has shown that number concepts activate both spatial and magnitude representations. According to the social co-representation literature which has shown that participants typically represent task components assigned to others together with their own, we asked whether explicit magnitude meaning and explicit spatial coding must be present in a single mind, or can be distributed across two minds, to generate a spatial-numerical congruency effect. In a shared go/no-go task that eliminated peripheral spatial codes, we assigned explicit magnitude processing to participants and spatial processing to either human or non-human co-agents. The spatial-numerical congruency effect emerged only with human co-agents. We demonstrate an inter-personal level of conceptual congruency between space and number that arises from a shared conceptual representation not contaminated by peripheral spatial codes. Theoretical implications of this finding for numerical cognition are discussed.

Robots facilitate human language production

Despite recent developments in integrating autonomous and human-like robots into many aspects of everyday life, social interactions with robots are still a challenge. Here, we focus on a central tool for social interaction: verbal communication. We assess the extent to which humans co-represent (simulate and predict) a robot's verbal actions. During a joint picture naming task, participants took turns in naming objects together with a social robot (Pepper, Softbank Robotics). Previous findings using this task with human partners revealed internal simulations on behalf of the partner down to the level of selecting words from the mental lexicon, reflected in partner-elicited inhibitory effects on subsequent naming. Here, with the robot, the partner-elicited inhibitory effects were not observed. Instead, naming was facilitated, as revealed by faster naming of word categories co-named with the robot. This facilitation suggests that robots, unlike humans, are not simulated down to the level of lexical selection. Instead, a robot's speaking appears to be simulated at the initial level of language production where the meaning of the verbal message is generated, resulting in facilitated language production due to conceptual priming. We conclude that robots facilitate core conceptualization processes when humans transform thoughts to language during speaking.

Interactive brains, social minds: Neural and physiological mechanisms of interpersonal action coordination

It is now widely accepted that inter-brain synchronization is an important and inevitable mechanism of interpersonal action coordination and social interaction behavior. This review of the current literature focuses first on the forward model for interpersonal action coordination and functional system theory for biological systems, two broadly similar concepts for adaptive system behavior. Further, we review interacting-brain and/or hyper-brain dynamics studies, to show the interplay between intra- and inter-brain connectivity resulting in hyper-brain network structure and network topology dynamics, and consider the functioning of interacting brains as a superordinate system. The concept of a superordinate system, or superorganism, is then evaluated with respect to neuronal and physiological systems group dynamics, which show further accompanying mechanisms of interpersonal interaction. We note that fundamental problems need to be resolved to better understand the neural mechanisms of interpersonal action coordination.

Action co-representation under threat: A Social Simon study

Several studies have shown that individuals automatically integrate the actions of other individuals into their own action plans, thus facilitating action coordination. What happens to this mechanism in situations of danger? This capacity could either be reduced, in order to allocate more cognitive resources for individualistic actions, or be maintained or enhanced to enable cooperation under threat. In order to determine the impact of the perception of danger on this capacity, two groups of participants carried out, in pairs, the Social Simon task, which provides a measure of co-representation. The task was performed during so-called 'threat blocks' (during which participants could be exposed at any time to an aversive stimulus) and so-called 'safety blocks' (during which no aversive stimulation could occur). In a first group of participants, both individuals were exposed at the same time to threat blocks. In a second group, only one of the two participants was exposed to them at a time. Our results indicate that co-representation, an important cognitive mechanism for cooperation, (i) is preserved in situations of danger; and (ii) may even be increased in participants who are confronted alone to threat but in the presence of a safe partner. Contrarily to popular belief, danger does not shut down our capacities for social interaction.

Excluded but not alone. Does social exclusion prevent the occurrence of a Joint Simon Effect (JSE)?

The Joint Simon Effect (JSE) is known to reflect the natural and spontaneous tendency to integrate actions from another individual into our own action system during joint action. This tendency to co-represent others' actions positively correlates with the dispositional tendency to take others' perspective. However, a quick episode of social exclusion has been shown to erase the tendency to co-represent others' actions. From a theoretical viewpoint, excluded individuals should still show signs of co-representation as ostracism tends to increase attention paid to others and Perspective Taking. In this context, this study challenges the idea that social exclusion eliminates all forms of co-representation (or JSE) by using an in-person version of the Task. We also intend to replicate findings on a positive link between Perspective Taking and the size of the JSE. To this end, participants played either an inclusive or exclusive version of the Cyberball game, then performed a joint go-nogo Simon Task. Perspective Taking was measured by using the Interpersonal Reactivity Index. By contrast to previous results, our results revealed that excluded individuals co-represented others' actions as much as included ones. However, a positive correlation between Perspective Taking and the JSE was limited to included participants. We discuss these findings in terms of methodological differences and in the light of the social exclusion literature.

Processing and utilization of auditory action effects in individual and social tasks

The influence of action-effect integration on motor control and sensory processing is often investigated in arrangements featuring human-machine interactions. Such experiments focus on predictable sensory events produced through participants' interactions with simple response devices. Action-effect integration may, however, also occur when we interact with human partners. The current study examined the similarities and differences in perceptual and motor control processes related to generating sounds with or without the involvement of a human partner. We manipulated the complexity of the causal chain of events between the initial motor and the final sensory event. In the self-induced condition participants generated sounds directly by pressing a button, while in the interactive condition sounds resulted from a paired reaction-time task, that is, the final sound was generated indirectly, by relying on the contribution of the partner. Auditory event-related potentials (ERPs) and force application patterns were similar in the two conditions, suggesting that social action effects produced with the involvement of a second human agent in the causal sequence are processed, and utilized as action feedback in the same way as direct consequences of one's actions. The only reflection of a processing difference between the two conditions was a slow, posterior ERP waveform that started before the presentation of the auditory stimulus, which may reflect differences in stimulus expectancy or task difficulty.

The influence of social context and social connection on visual perceptual processes

The aim of this study was twofold: (1) to re-test whether a minimal social context has an influence on gaze patterns, and (2) to determine if a social connection (i.e., friendship) has a modulatory effect on gaze patterns in a minimal social context. In Experiment 1, two unacquainted participants were paired and seated at separate testing stations in the same room. At the beginning of each trial, participants were informed whether they were looking at different image sets (solo trials), or the same image set (joint trials). Image sets consisted of a positive, a negative, and two neutral images. No explicit task instructions were provided and there was no interaction between participants during the task. Experiment 2 was identical to Experiment 1, except that participants were paired with a friend. The fixation-based metrics of interest were time to first fixation (TFF) and total fixation duration (TFD). The findings revealed that social context has a modulatory effect on attentional capture (i.e., TFF) irrespective of social connection. Unexpectedly, a negativity bias was found to hold attention (i.e., TFD) regardless of social context. However, having a social connection did increase the time spent looking at positive images on the joint trials. Having a social connection with another person seems to alter looking behaviour such that more time is spent looking at positive images on the joint trials compared to the solo trials. These findings highlight the importance of evaluating both TFD and TFF to develop a better understanding of the factors underlying joint perception.

Sequential aiming in pairs: the multiple levels of joint action

The task constraints imposed upon a co-actor can often influence our own actions. Likewise, the observation of somebody else's movements can involuntarily contaminate the execution of our own movements. These joint action outcomes have rarely been considered in unison. The aim of the present study was to simultaneously examine the underlying processes contributing to joint action. We had pairs of participants work together to execute sequential aiming movements between two targets-the first person's movement was contingent upon the anticipation of the second person's movement (leader), while the second person's movement was contingent upon the direct observation of the first person's movement (follower). Participants executed separate blocks of two-target aiming movements under different contexts; that is, solely on their own using one (2T1L) and two (2T2L) of their upper limbs, or with another person (2T2P). The first movement segment generally indicated a more abrupt approach (shorter time after peak velocity, greater displacement and magnitude of peak velocity), which surprisingly coincided with lower spatial variability, for the 2T2P context. Meanwhile, the second segment indicated a similar kinematic profile as the first segment for the 2T2P context. The first movement of the leader appeared to accommodate the follower for their movement, while the second movement of the follower was primed by the observation of the leader's movement. These findings collectively advocate two distinct levels of joint action including the anticipation (top-down) and mapping (bottom-up) of other people's actions.

Effect of perceived interpersonal closeness on the joint Simon effect in adolescents and adults

Here, we explored the role of perceived interpersonal closeness in joint action using the joint Simon task in adolescents and adults. In a two-choice reaction time task, spatially assigned responses to non-spatial stimulus features are faster when the stimulus and response are in congruent locations than not. This phenomenon is called Simon effect and is absent or strongly attenuated when a participant responds to only one of the stimuli. However, the effect reappears when two participants carry out the same go/no-go tasks cooperatively. This re-emergence of the Simon effect in joint action is called the joint Simon effect (JSE). In this study, we first replicated the standard and joint Simon effects in adolescents (n = 43), as well as adults (n = 39) with similar magnitude of the effects in the two age groups. The magnitude of the JSE was positively correlated with the level of closeness as measured by Inclusion of Other in the Self scale. This correlation was not significantly different in adolescents (n = 73) compared to adults (n = 71). Our findings show that joint action is sensitive to the social factor such as interpersonal closeness, and the underlying mechanisms are already mature by adolescence.

Vicarious representation: A new theory of social cognition

Theory of mind, the attribution of mental states to others is one form of social cognition. The aim of this paper is to highlight the importance of another, much simpler, form of social cognition, which I call vicarious representation. Vicarious representation is the attribution of other-centered properties to objects. This mental capacity is different from, and much simpler than, theory of mind as it does not imply the understanding (or representation) of the mental (or even perceptual) states of another agents. I argue that the most convincing experiments that are supposed to show that non-human primates have theory of mind in fact demonstrate that they are capable of vicarious representation. The same is true for the experiments about the theory of mind of infants under 12 months.

The effect of co-actor group membership on the social inhibition of return effect

Being part of a group is a crucial factor in human social interaction. In the current study we explored whether group membership affects reflexive automatic cognitive functioning, and specifically the social inhibition of return effect (SIOR; Welsh et al., 2005). SIOR is characterized by slower reaction times (RTs) to a location already searched by another agent. To examine whether group membership modulates SIOR, we recruited Muslim and Jewish students from the University of Haifa to perform a task with either an in-group member or an out-group member. Both IOR and SIOR were suggested to act as a foraging facilitator (Klein, 2000; Welsh et al., 2005). Accordingly, we predicted that the SIOR effect would be larger when performing the task with an in-group member than with an out-group member. The results confirmed our prediction by indicating that the co-actor's group membership modulated the SIOR effect. These findings are consistent with the notion that social factors play a critical role in producing the SIOR effect and provide a novel indication of the influence of social factors such as group membership on basic reflexive cognitive processes.

The role of the co-actor's response reachability in the joint Simon effect: remapping of working space by tool use

The Simon effect, that is the advantage of the spatial correspondence between stimulus and response locations when stimulus location is task irrelevant, occurs even when the task is performed by two participants, each performing a go/no-go task. This effect, known as the joint Simon effect, does not emerge when participants sit outside each other's peripersonal space, thus suggesting that the presence of an active confederate in peripersonal space might provide a reference for response coding. The present study investigated whether this finding is due to the distance separating the participants and/or to the distance separating each participant and the other agent's response. In two experiments, pairs of participants performed a social detection task sitting outside each other's arm reach, with response keys located close to the participants or outside arm reach. When the response key was located outside the participant's arm reach, he/she could reach it by means of a tool. In Experiment 1, by means of a tool, participants could reach their response key only, while in Experiment 2, they could reach also their co-agent's response key. The joint Simon effect did not emerge when participants could not reach the co-actor's response, while it emerged when they could potentially reach the other participant's response using the tool, but only when turn taking was required. These results may be taken as evidence that the possibility to reach and act upon the co-actor's response key may be at the bases of compatibility effects observed in joint action contexts requiring complementary responses.

A matter of you versus me? Experiences of control in a joint go/no-go task

When interacting with others, people represent their own as well as their interaction partners' actions. Such joint action representation is essential for action coordination, but may also interfere with action control. We investigated how joint action representations affect experienced control over people's own actions and their interaction partners' actions. Participants performed a joint go/no-go task, which is commonly used to measure to what extent people represent their own actions in spatial reference to their interaction partner (e.g., as 'left' vs. 'right'). After each second trial, participants indicated experienced control over their own action, their interaction partner's action, or over action inhibition. Despite this frequent interruption of the go/no-go task, we found strong evidence for the spatial representation of joint actions. However, this joint action representation did not affect experiences of control. Possible explanations and implications of these findings are discussed.

The social Simon effect in the tactile sensory modality: a negative finding

This study seeks to investigate whether users activate cognitive representations of their partner's action when they are involved in tactile collaborative tasks. The social Simon effect is a spatial stimulus-response interference induced by the mere presence of a partner in a go/nogo task. It has been extensively studied in the visual and auditory sensory modalities, but never before in the tactile modality. We compared the performances of 28 participants in three tasks: (1) a standard Simon task where participants responded to two different tactile stimuli applied to their fingertips with either their left or right foot, (2) an individual go/nogo task where participants responded to only one stimulus and (3) a social go/nogo task where they again responded to only one stimulus, but were partnered with another person who responded to the complementary stimulus. The interference effect due to spatial incongruence between the side where participants received the stimulus and the foot used to answer increased significantly in the standard Simon task compared to the social go/nogo task. Such a difference was not observed between the social and individual go/nogo tasks. Performances were nevertheless enhanced in the social go/nogo task, but irrespectively of the stimulus-response congruency. This study is the first to report a negative result for the social Simon effect in the tactile modality. Results suggest that cognitive representation of the co-actor is weaker in this modality.

Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Despite considerable interest in both action perception and social attention over the last 2 decades, there has been surprisingly little investigation concerning how the manual actions of other humans orient visual attention. The present review draws together studies that have measured the orienting of attention, following observation of another's goal-directed action. Our review proposes that, in line with the literature on eye gaze, action is a particularly strong orienting cue for the visual system. However, we additionally suggest that action may orient visual attention using mechanisms, which gaze direction does not (i.e., neural direct mapping and corepresentation). Finally, we review the implications of these gaze-independent mechanisms for the study of attention to action. We suggest that our understanding of attention to action may benefit from being studied in the context of joint action paradigms, where the role of higher level action goals and social factors can be investigated.

The Multimodal Go-Nogo Simon Effect: Signifying the Relevance of Stimulus Features in the Go-Nogo Simon Paradigm Impacts Event Representations and Task Performance

Numerous studies have shown that stimulus-response-compatibility (SRC) effects in the go-nogo version of the Simon task can be elicited as a result of performing the task together with another human or non-human agent (e.g., a Japanese-waving-cat, a working-clock, or a ticking-metronome). A parsimonious explanation for both social and non-social SRC effects is that highlighting the spatial significance of alternative (non-/social) action events makes action selection more difficult. This holds even when action events are task-irrelevant. Recent findings, however, suggest that this explanation holds only for cases of a modality correspondence between the Simon task as such (i.e., auditory or visual) and the alternative (non-/social) action event that needs to be discriminated. However, based on the fact that perception and action are represented by the same kind of codes, an event that makes the go-nogo decision more challenging should impact go-nogo Simon task performance. To tackle this issue, the present study tested if alternative stimulus events that come from a different sensory modality do impact SRC effects in the go-nogo version of the Simon task. This was tested in the presence and absence of alternative action events of a human co-actor. In a multimodal (auditory–visual) go-nogo Simon paradigm, participants responded to their assigned stimulus – e.g., a single auditory stimulus while ignoring the alternative visual stimulus or vice versa – in the presence or absence of a human co-actor (i.e., joint and single go-nogo condition). Results showed reliable SRCs in both, single and joint go-nogo Simon task conditions independent of the modality participants had to respond to. Although a correspondence between stimulus material and attention-grabbing event might be an efficient condition for SRCs to emerge, the driving force underlying the emergence of SRCs rather appears to be whether the attentional focus prevents or facilitates alternative events to be integrated. Thus, under task conditions in which the attentional focus is sufficiently broad to enable the integration and thus cognitive representation of alternative events, go-nogo decisions become more difficult, resulting in reliable SRCs in single and joint go-nogo Simon tasks.

Sharing a mental number line across individuals? The role of body position and empathy in joint numerical cognition

A growing body of research shows that the human brain acts differently when performing a task together with another person than when performing the same task alone. In this study, we investigated the influence of a co-actor on numerical cognition using a joint random number generation (RNG) task. We found that participants generated relatively smaller numbers when they were located to the left (vs. right) of a co-actor (Experiment 1), as if the two individuals shared a mental number line and predominantly selected numbers corresponding to their relative body position. Moreover, the mere presence of another person on the left or right side or the processing of numbers from loudspeaker on the left or right side had no influence on the magnitude of generated numbers (Experiment 2), suggesting that a bias in RNG only emerged during interpersonal interactions. Interestingly, the effect of relative body position on RNG was driven by participants with high trait empathic concern towards others, pointing towards a mediating role of feelings of sympathy for joint compatibility effects. Finally, the spatial bias emerged only after the co-actors swapped their spatial position, suggesting that joint spatial representations are constructed only after the spatial reference frame became salient. In contrast to previous studies, our findings cannot be explained by action co-representation because the consecutive production of numbers does not involve conflict at the motor response level. Our results therefore suggest that spatial reference coding, rather than motor mirroring, can determine joint compatibility effects. Our results demonstrate how physical properties of interpersonal situations, such as the relative body position, shape seemingly abstract cognition.

Intentionality of a co-actor influences sensorimotor synchronisation with a virtual partner

Interpersonal sensorimotor synchronisation requires individuals to anticipate and adapt to their partner's movement timing. Research has demonstrated that the intentionality of a co-actor affects joint action planning, however, less is known about whether co-actor intentionality affects sensorimotor synchronisation. Explicit and implicit knowledge of a synchronisation partner's intentionality may influence coordination by modulating temporal anticipation and adaptation processes. We used a computer-controlled virtual partner (VP) consisting of tempo-changing auditory pacing sequences to simulate either an intentional or unintentional synchronisation partner. The VP was programmed to respond to the participant with low or moderate degrees of error correction, simulating a slightly or moderately adaptive human, respectively. In addition, task instructions were manipulated so that participants were told they were synchronising with either another person or a computer. Results indicated that synchronisation performance improved with the more adaptive VP. In addition, there was an influence of the explicit partner instruction, but this was dependent upon the degree of VP adaptivity and was modulated by subjective preferences for either the human or the computer partner. Beliefs about the intentionality of a synchronisation partner may thus influence interpersonal sensorimotor synchronisation in a manner that is modulated by preferences for interacting with intentional agents.

Co-actors represent the order of each other's actions

Previous research has shown that people represent each other’s tasks and actions when acting together. However, less is known about how co-actors represent each other’s action sequences. Here, we asked whether co-actors represent the order of each other’s actions within an action sequence, or whether they merely represent the intended end state of a joint action together with their own contribution. In the present study, two co-actors concurrently performed action sequences composed of two actions. We predicted that if co-actors represent the order of each other’s actions, they should experience interference when the order of their actions differs. Supporting this prediction, the results of six experiments consistently showed that co-actors moved more slowly when performing the same actions in a different order compared to performing the same actions in the same order. In line with findings from bimanual movement tasks, our results indicate that interference can arise due to differences in movement parameters and due to differences in the perceptual characteristics of movement goals. The present findings extend previous research on co-representation, providing evidence that people represent not only the elements of another’s task, but also their temporal structure.

Are You Keeping an Eye on Me? The Influence of Competition and Cooperation on Joint Simon Task Performance

Social interaction plays an important role in human life. While there are instances that require cooperation, there are others that force people to compete rather than to cooperate, in order to achieve certain goals. A key question is how the deployment of attention differs between cooperative and competitive situation; however, empirical investigations have yielded inconsistent results. By manipulating the (in-)dependence of individuals via performance-contingent incentives, in a visual go–nogo Simon task the current study aimed at improving our understanding of complementary task performance in a joint action context. In the independent condition each participant received what s/he achieves; in the cooperative condition each participant received the half of what both achieved, and in the competitive condition participants were instructed that the winner takes it all. Extending previous findings, we found sequential processing adjustments of the Simon effect as a function of the interdependency (i.e., competition, cooperation) and transition between (i.e., go–nogo requirements) interacting individuals. While sequential processing adjustments of the Simon effect in both the competition and cooperation condition were unaffected when alternating between responsible actors (i.e., nogo–go transition), sequential processing adjustments were enlarged under competition for repeating responsibilities of one and the same actor (i.e., go–go transitions). In other words, the prospect of performance-contingent reward in a competitive context exclusively impacts flexible behavioral adjustments of one’s own actions. Rather than fostering the consideration and differentiation of the other actor, pushing one’s own performance to the limit appears to be the suitable strategy in competitive instances of complementary tasks. Therefore, people keep their eyes on themselves when aiming at beating a co-actor and emerging as the winner.

Bimanual joint action: correlated timing or "bimanual" movements accomplished by two people

A crew of two rowing together in perfect synchrony is an example of a task that requires each performer to maintain meticulous timing when coordinating their movements with the other. At the individual level, temporal coordination of the limbs has been observed in bimanual pointing movements even when made to targets of different distance. Timing of the arms is not independent; rather there is a natural temporal coupling. The aim of this experiment was to investigate whether the temporal characteristics of pointing movements can be observed under joint conditions. Sixteen pairs of participants made short and long, unimanual and bimanual pointing movements. In the unimanual and bimanual solo conditions, participants made the movements alone. In the joint condition, each participant contributed one arm to the joint "bimanual" movements. Absolute temporal coupling at movement initiation and termination was measured by the differences in reaction time and total response time. Relative temporal coupling at movement initiation and termination was measured by correlating reaction time and total response time of the left and right limbs. Pointing movements had synchronous movement termination in the bimanual solo conditions and asynchronous termination in the unimanual solo and bimanual joint conditions. The initiation and termination of the arms were not correlated in the unimanual solo condition (initiation r = 0.01, termination r = 0.03). Small-to-medium correlations (r = 0.19, r = 0.24) were observed in the bimanual joint condition, and they were larger than the unimanual solo condition (p = 0.022, p = 0.063). As expected, there were large correlations in the bimanual solo conditions (r = 0.91, r = 0.81). Our findings suggest that absolute temporal coupling does not occur between individuals, but there is evidence for relative temporal coupling in the bimanual joint condition.

When a Social Experimenter Overwrites Effects of Salient Objects in an Individual Go/No-Go Simon Task - An ERP Study

When two persons share a Simon task, a joint Simon effect occurs. The task co-representation account assumes that the joint Simon effect is the product of a vicarious representation of the co-actor's task. In contrast, recent studies show that even (non-human) event-producing objects could elicit a Simon effect in an individual go/no-go Simon task arguing in favor of the referential coding account. For the human-induced Simon effect, a modulation of the P300 component in Electroencephalography (EEG) is typically considered as a neural indicator of the joint Simon effect and task co-representation. Showing that the object-induced Simon effects also modulates the P300 would lead to a re-evaluation of the interpretation of the P300 in individual go/no-go and joint Simon task contexts. To do so, the present study conceptually replicated Experiment 1 from Dolk et al. (2013a) adding EEG recordings and an experimenter controlling the EEG computer to test whether a modulation of the P300 can also be elicited by adding a Japanese waving cat to the task context. Subjects performed an individual go/no-go Simon task with or without a cat placed next to them. Results show an overall Simon effect regardless of the cat's presence and no modulatory influence of the cat on the P300 (Experiment 1), even when conceivably interfering context factors are diminished (Experiment 2). These findings may suggest that the presence of a spatially aligned experimenter in the laboratory may produce an overall Simon effect overwriting a possible modulation of the Japanese waving cat.

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the "like me" hypothesis, and discuss the key role played by the Theory-of-Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long-term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents.

Corepresentation During Joint Action in Marmoset Monkeys ( Callithrix jacchus)

Behavioral coordination is a fundamental element of human cooperation. It is facilitated when individuals represent not only their own actions but also those of their partner. Identifying whether action corepresentation is unique to humans or also present in other species is therefore necessary to fully understand the evolution of human cooperation. We used the auditory joint Simon task to assess whether action corepresentation occurs in common marmosets, a monkey species that engages extensively in coordinated action during cooperative infant care. We found that marmosets indeed show a joint Simon effect. Furthermore, when coordinating their behavior in the joint task, they were more likely to look at their partner than in a joint control condition. Corepresentation is thus not unique to humans but also present in the cooperatively breeding marmosets. Since marmosets are small-brained monkeys, our results suggest that routine coordination in space and time, rather than complex cognitive abilities, plays a role in the evolution of corepresentation.

Temporal binding effect in the action observation domain: Evidence from an action-based somatosensory paradigm

Temporal binding is understood as an effect in which a temporal interval between a voluntary action and its consequent effect is perceived as compressed. It denotes an implicit measure of a sense of agency. When people observe someone else performing an action that generates an effect, temporal binding also takes place. We aimed to test whether the interaction between observed actions and tactile sensation influences temporal binding. Participants observed finger tapping movements (of a human or wooden hand), in parallel to receiving tactile stimulations on their fingertip. These stimulations were either congruent or incongruent with the tactile consequences of the observed movement. The finger tapping movement was followed by a tone. Participants estimated the intervals between the observed action and the tone. We found that temporal binding for observed actions depends on the congruency between the perceived touch and tactile consequences of observed actions restricted to intentional actors.

The influence of passionate love on self-other discrimination during joint action

Prior research on romantic relationships suggests that being in love involves a blurring of self-other cognitive boundaries. However, this research has focused so far on conceptual self-representation, related to the individual's traits or interests. The present study tested the hypothesis that passionate love involves a reduced discrimination between the self and the romantic partner at a bodily level, as indexed by an increased Joint Simon effect (JSE), and we further examined whether this self-other discrimination correlated with the passion felt for the partner. As predicted, we found an increased JSE when participants performed the Joint Simon Task with their romantic partner compared with a friend of the opposite sex. Providing support for the self-expansion model of love (Aron and Aron in Pers Relatsh 3(1):45-58, 1996), this result indicates that romantic relationships blur the boundaries between the self and the romantic partner at a bodily level. Furthermore, the strength of romantic feelings was positively correlated with the magnitude of the JSE when sharing the task with the romantic partner.

You Are Right!

In the social Simon task, two participants perform a spatial compatibility task together, each of them responding to only one stimulus (e.g., one participant reacts to red, the other to green stimuli). Participants show joint spatial compatibility effects (SCEs), that is, they respond faster when their go-stimulus appears on their half of the screen. Effects are absent when the same go/no-go task is performed without a coactor. Joint SCEs were originally explained in terms of shared task representations, but recent research suggests that effects result from spatial response coding: in joint go/no-go tasks, participants perceive themselves as the right/left participant operating a right/left response key. While previous research showed that the spatial alignment of keys and seats influences the effect, the present research demonstrates that merely instructing participants to be the right/left participant operating a right/left response key instead of labeling participants and keys with arbitrary numbers substantially increases joint SCEs.

No evidence for shared representations of task sets in joint task switching

It has been suggested that actors co-represent a shared task context when they perform a task in a joint fashion. The present study examined the possibility of co-representation in joint task switching, in which two actors shared two tasks that switched randomly across trials. Experiment 1 showed that when an actor performed the tasks individually, switch costs were obtained if the actors responded on the previous trial (go trial), but not if they did not respond (no-go trial). When two actors performed the tasks jointly, switch costs were obtained if the actor responded on the previous trial (actor-repeat trials) but not if the co-actor responded (actor-switch trials). In Experiment 2, a single actor performed both tasks of the joint condition to test whether the findings of Experiment 1 were due to the use of different response sets by the two actors. Switch costs were obtained for both repetitions and alternations of the response set, which rules out this possibility. Taken together, our findings provided little support for the idea that actors co-represent the task sets of their co-actors.

The Joint Action Effect on Memory as a Social Phenomenon: The Role of Cued Attention and Psychological Distance

In contrast to individual tasks, a specific social setting is created when two partners work together on a task. How does such a social setting affect memory for task-related information? We addressed this issue in a distributed joint-action paradigm, where two team partners respond to different types of information within the same task. Previous work has shown that joint action in such a task enhances memory for items that are relevant to the partner’s task but not to the own task. By removing critical, non-social confounds, we wanted to pinpoint the social nature of this selective memory advantage. Specifically, we created joint task conditions in which participants were aware of the shared nature of the concurrent task but could not perceive sensory cues to the other’s responses. For a differentiated analysis of the social parameters, we also varied the distance between partners. We found that the joint action effect emerged even without sensory cues from the partner, and it declined with increasing distance between partners. These results support the notion that the joint-action effect on memory is in its core driven by the experience of social co-presence, and does not simply emerge as a by-product of partner-generated sensory cues.

Pointing out mechanisms underlying joint action

Recent studies have shown that spatial and feature-based attention can contribute to inducing a spatial compatibility effect in both the standard Simon task and the joint Simon task. Less work generally has been devoted to investigating how attention modulates spatial compatibility effects. In the present study, we aimed to explore whether indirectly manipulating the degree of attention necessary to respond to a compatible or an incompatible stimulus can modulate the spatial compatibility effect in a joint Simon task and elicit a compatibility effect in the individual go-no-go Simon task. To this end, we biased spatial attention to the compatible stimulus by asking participants to perform a pointing response always toward the compatible side, regardless of the stimulus location. Crucially, reaction times-recorded at gesture onset-showed a compatibility effect pattern in the individual condition and an additional modulation in the joint condition. These results show that the spatial attention intrinsic to action planning can affect both individual and joint Simon tasks.