When Pinocchio acts like a human, a wooden hand becomes embodied. Action co-representation for non-biological agents

Joint action with human and robotic co-actors: Self-other integration is immune to the perceived humanness of the interacting partner

When performing a joint action task, we automatically represent the action and/or task constraints of the co-actor with whom we are interacting. Current models suggest that, not only physical similarity, but also abstract, conceptual features shared between self and the interacting partner play a key role in the emergence of joint action effects. Across two experiments, we investigated the influence of the perceived humanness of a robotic agent on the extent to which we integrate the action of that agent into our own action/task representation, as indexed by the Joint Simon Effect (JSE). The presence (vs. absence) of a prior verbal interaction was used to manipulate robot's perceived humanness. In Experiment 1, using a within-participant design, we had participants perform the joint Go/No-go Simon task with two different robots. Before performing the joint task, one robot engaged in a verbal interaction with the participant and the other robot did not. In Experiment 2, we employed a between-participants design to contrast these two robot conditions as well as a human partner condition. In both experiments, a significant Simon effect emerged during joint action and its amplitude was not modulated by the humanness of the interacting partner. Experiment 2 further showed that the JSE obtained in robot conditions did not differ from that measured in the human partner condition. These findings contradict current theories of joint action mechanisms according to which perceived self-other similarity is a crucial determinant of self-other integration in shared task settings.

Action prediction modulates self-other integration in joint action

People often coordinate actions with others, requiring an adjustable amount of self-other integration between actor's and co-actor's actions. Previous research suggests that such self-other integration (indexed by the joint Simon effect) is enhanced by agent similarity of the co-actor (e.g., high in intentionality). In this study, we aimed to extend this line of research by testing whether experiencing agency over a co-actor's actions (vicarious agency) and/or action prediction strengthens the joint Simon effect. For this purpose, we manipulated experienced agency by varying the experienced control over a co-actor's actions (Experiment 1), and action prediction regarding the co-actor's actions (Experiment 2). Vicarious agency could effectively be induced, but did not modulate the size of the joint Simon effect. The joint Simon effect was decreased when the co-actor's actions were unpredictable (vs. predictable) during joint task performance. These findings suggest social agency can be induced and effectively measured in joint action. Action prediction can act as an effective agency cue modulating the amount of self-other integration in joint action.

Reaction time coupling in a joint stimulus-response task: A matter of functional actions or likable agents?

Shaping one owns actions by observing others’ actions is driven by the deep-rooted mechanism of perception-action coupling. It typically occurs automatically, expressed as for example the unintentional synchronization of reaction times in interactive games. Theories on perception-action coupling highlight its benefits such as the joint coordination of actions to cooperatively perform tasks properly, the learning of novel actions from others, and the bonding with likable others. However, such functional aspects and how they shape perception-action coupling have never been compared quantitatively. Here we tested a total of hundred-fifteen participants that played a stimulus-response task while, in parallel, they observed videos of agents that played the exact same task several milliseconds in advance. We compared to what degree the reaction times of actions of agents, who varied their behavior in terms of functionality and likability in preceding prisoner dilemma games and quizzes, shape the reaction times of human test participants. To manipulate functionality and likability, we varied the predictability of cooperative behavior and correctness of actions of agents, respectively, resulting in likable (cooperative), dislikable (uncooperative), functional (correct actions), and dysfunctional (incorrect actions) agents. The results of three experiments showed that the participants’ reaction times correlated most with the reaction times of agents that expressed functional behavior. However, the likability of agents had no effects on reaction time correlations. These findings suggest that, at least in the current computer task, participants are more likely to adopt the timing of actions from people that perform correct actions than from people that they like.

Spatial auditory presentation of a partner's presence induces the social Simon effect

Social presence is crucial for smooth communications in virtual reality (VR). Current telecommunication systems rarely submit spatial auditory information originating from remote people. However, such information may enhance social presence in VR. In this study, we constructed a dynamic binaural synthesis system and investigated the effect of spatial auditory information of a remote partner on a participant's behavior using the social Simon effect (SSE). The SSE is a spatial stimulus-response compatibility effect between two persons. The SSE occurs when one perceives that their partner is present. Several studies have confirmed the SSE in actual environments. We presented partner sounds diotically (i.e., without spatial information) to one group or binaurally (i.e., with spatial information) to another group through headphones without providing visual information about the partner. The results showed that the SSE was induced only in the binaural group in the current auditory VR (Experiment 1), whereas both groups exhibited the SSE in an actual environment (Experiment 2). These results suggest that the auditory spatial information of remote people is sufficient to induce the SSE and has a potential to enhance social presence.

Reluctance against the machine: Retrieval of observational stimulus-response episodes in online settings emerges when interacting with a human, but not with a computer partner

Observing how another person responds to a stimulus creates stimulus–response (SR) episodes. These can be retrieved from memory on later occasions, which means that observed responses are utilized for regulating one’s own actions. Until now, evidence for storage and retrieval of observationally acquired SR episodes was limited to dyadic face-to-face interactions between two partners who respond in an alternating fashion. In two preregistered studies (total N = 252), we demonstrate for the first time that observational SR episodes can also be acquired in online interactions: Robust retrieval effects emerged when observers believe to be interacting with another person. In turn, retrieval effects were absent when observers believe to be interacting with a computer. Our findings show that feature-based binding and retrieval principles are pervasive and also apply to social interactions, even under purely virtual conditions. We discuss implications of our findings for different explanatory accounts of social modulations of automatic imitation.

Joint Simon effect in movement trajectories

In joint action literature it is often assumed that acting together is driven by pervasive and automatic process of co-representation, that is, representing the co-actor's part of the task in addition to one's own. Much of this research employs joint stimulus-response compatibility tasks varying the stimuli employed or the physical and social relations between participants. In this study we test the robustness of co-representation effects by focusing instead on variation in response modality. Specifically, we implement a mouse-tracking version of a Joint Simon Task in which participants respond by producing continuous movements with a computer mouse rather than pushing discrete buttons. We have three key findings. First, in a replication of an earlier study we show that in a classical individual Simon Task movement trajectories show greater curvature on incongruent trials, paralleling longer response times. Second, this effect largely disappears in a Go-NoGo Simon Task, in which participants respond to only one of the cues and refrain from responding to the other. Third, contrary to previous studies that use button pressing responses, we observe no overall effect in the joint variants of the task. However, we also detect a notable diversity in movement strategies adopted by the participants, with some participants showing the effect on the individual level. Our study casts doubt on the pervasiveness of co-representation, highlights the usefulness of mouse-tracking methodology and emphasizes the need for looking at individual variation in task performance.

Number to me, space to you: Joint representation of spatial-numerical associations

Recent work has shown that number concepts activate both spatial and magnitude representations. According to the social co-representation literature which has shown that participants typically represent task components assigned to others together with their own, we asked whether explicit magnitude meaning and explicit spatial coding must be present in a single mind, or can be distributed across two minds, to generate a spatial-numerical congruency effect. In a shared go/no-go task that eliminated peripheral spatial codes, we assigned explicit magnitude processing to participants and spatial processing to either human or non-human co-agents. The spatial-numerical congruency effect emerged only with human co-agents. We demonstrate an inter-personal level of conceptual congruency between space and number that arises from a shared conceptual representation not contaminated by peripheral spatial codes. Theoretical implications of this finding for numerical cognition are discussed.

How vertical elevation affects self-other integration as measured by the joint Simon effect

Earlier findings suggest that positions of power decrease self-other integration and increase psychological distance to others. Until now, however, evidence for this relation rests exclusively on subjective measures. The current research instead employed a vertical joint Simon task to measure self-other integration. This task assesses the extent to which people represent their own actions in reference to their co-actor's, also referred to as the joint Simon effect. Building on cultural associations between power and vertical elevation, we manipulated whether participants were in an elevated (high-power) or lower (low-power) seating position. Experiments 1a and 1b reanalyzed existing datasets and found that elevated (vs. lower) seating position decreased the joint Simon effect, consistent with predictions. Experiment 2 provides a high-powered replication of this finding. Yet, further analyses revealed that feelings of power - measured as a manipulation check and indeed demonstrating that the manipulation was successful - did not mediate or moderate the effect of seating position on the joint Simon effect. Therefore, it is possible that the effect of seating elevation was driven through other aspects of that manipulation than feelings of power. We discuss these and suggest ways to test these alternative explanations.

Robots facilitate human language production

Despite recent developments in integrating autonomous and human-like robots into many aspects of everyday life, social interactions with robots are still a challenge. Here, we focus on a central tool for social interaction: verbal communication. We assess the extent to which humans co-represent (simulate and predict) a robot's verbal actions. During a joint picture naming task, participants took turns in naming objects together with a social robot (Pepper, Softbank Robotics). Previous findings using this task with human partners revealed internal simulations on behalf of the partner down to the level of selecting words from the mental lexicon, reflected in partner-elicited inhibitory effects on subsequent naming. Here, with the robot, the partner-elicited inhibitory effects were not observed. Instead, naming was facilitated, as revealed by faster naming of word categories co-named with the robot. This facilitation suggests that robots, unlike humans, are not simulated down to the level of lexical selection. Instead, a robot's speaking appears to be simulated at the initial level of language production where the meaning of the verbal message is generated, resulting in facilitated language production due to conceptual priming. We conclude that robots facilitate core conceptualization processes when humans transform thoughts to language during speaking.

Action co-representation under threat: A Social Simon study

Several studies have shown that individuals automatically integrate the actions of other individuals into their own action plans, thus facilitating action coordination. What happens to this mechanism in situations of danger? This capacity could either be reduced, in order to allocate more cognitive resources for individualistic actions, or be maintained or enhanced to enable cooperation under threat. In order to determine the impact of the perception of danger on this capacity, two groups of participants carried out, in pairs, the Social Simon task, which provides a measure of co-representation. The task was performed during so-called 'threat blocks' (during which participants could be exposed at any time to an aversive stimulus) and so-called 'safety blocks' (during which no aversive stimulation could occur). In a first group of participants, both individuals were exposed at the same time to threat blocks. In a second group, only one of the two participants was exposed to them at a time. Our results indicate that co-representation, an important cognitive mechanism for cooperation, (i) is preserved in situations of danger; and (ii) may even be increased in participants who are confronted alone to threat but in the presence of a safe partner. Contrarily to popular belief, danger does not shut down our capacities for social interaction.

Does agency matter? Neural processing of robotic movements in 4- and 8-year olds

Despite the increase in interactions between children and robots, our understanding of children's neural processing of robotic movements is limited. The current study theorized that motor resonance hinges on the agency of an actor: its ability to perform actions volitionally. As one of the first studies with a cross-sectional sample of preschoolers and older children and with a specific focus on robotic action (rather than abstract non-human action), the current study investigated whether the perceived agency of a robot moderated children's motor resonance for robotic movements, and whether this changed with age. Motor resonance was measured using electroencephalography (EEG) by assessing mu power while 4 and 8-year-olds observed actions performed by agentic versus non-agentic robots and humans. Results show that older children resonated more strongly with non-agentic than agentic robotic or human movement, while no such differences were found for preschoolers. This outcome is discussed in terms of a predictive coding account of motor resonance. Importantly, these findings contribute to the existing set of studies on this topic by showing that, while keeping all kinematic information constant, there is a clear developmental difference in how children process robotic movement depending on the level of agency of a robot.

Metacontrol and joint action: how shared goals transfer from one task to another?

In most of our daily activities and in team sports, we interact with other individuals and do not act in isolation. Using a social variant of the standard two-choice Simon task, this study aims to test if competitive/cooperative processing modes (i.e., metacontrol states) change the degree of bodily self-other integration between two persons in joint action. In addition, and more exploratory the study tested if this effect depends on a shared group experience with the partner. Two participants shared a visual Simon task, so that each person basically performed complementary parts of the task, which transfers the paradigm into a go/no-go Simon task for each person. Before running this joint Simon task, we set both participants either in a competitive or a cooperative control state by means of a dyadic game, a manipulation aimed at testing possible goal transfer across tasks. We found significant joint Simon effects for participants who were in a competitive state and for participants who were in a cooperative state. The joint Simon effect for participants being in a competitive state was significantly smaller than for participants being in a cooperative state. When experiencing the goal induction together with the partner, the joint Simon effect was significantly decreased as when the induction was performed alone. Both effects (metacontrol state induction and shared experience) seem to be statistically independent of each other. In line with predictions of metacontrol state theory, our study indicated that abstract cognitive goal states can be transferred from one task to another task, able to affect the degree of bodily self-other integration across different task situations.

Pre-exposure to Ambiguous Faces Modulates Top-Down Control of Attentional Orienting to Counterpredictive Gaze Cues

Understanding and reacting to others' nonverbal social signals, such as changes in gaze direction (i.e., gaze cue), are essential for social interactions, as it is important for processes such as joint attention and mentalizing. Although attentional orienting in response to gaze cues has a strong reflexive component, accumulating evidence shows that it can be top-down controlled by context information regarding the signals' social relevance. For example, when a gazer is believed to be an entity "with a mind" (i.e., mind perception), people exert more top-down control on attention orienting. Although increasing an agent's physical human-likeness can enhance mind perception, it could have negative consequences on top-down control of social attention when a gazer's physical appearance is categorically ambiguous (i.e., difficult to categorize as human or nonhuman), as resolving this ambiguity would require using cognitive resources that otherwise could be used to top-down control attention orienting. To examine this question, we used mouse-tracking to explore if categorically ambiguous agents are associated with increased processing costs (Experiment 1), whether categorically ambiguous stimuli negatively impact top-down control of social attention (Experiment 2), and if resolving the conflict related to the agent's categorical ambiguity (using exposure) would restore top-down control to orient attention (Experiment 3). The findings suggest that categorically ambiguous stimuli are associated with cognitive conflict, which negatively impact the ability to exert top-down control on attentional orienting in a counterpredicitive gaze-cueing paradigm; this negative impact, however, is attenuated when being pre-exposed to the stimuli prior to the gaze-cueing task. Taken together, these findings suggest that manipulating physical human-likeness is a powerful way to affect mind perception in human-robot interaction (HRI) but has a diminishing returns effect on social attention when it is categorically ambiguous due to drainage of cognitive resources and impairment of top-down control.

Is young children's helping affected by helpees' need? Preschoolers, but not infants selectively help needy others

Infants and toddlers engage in instrumental helping, that is, help others in achieving an action-based goal. The underlying psychological mechanisms are unclear and hotly debated. The present study examined whether young children's helping is affected by others' need. To this end, 1.5- and 3.5-year-old children (n = 101) were simultaneously confronted with a needy and a non-needy other in a variety of helping tasks. The results show that the 3.5-year-old, but not the 1.5-year-old children preferentially helped the needy person. This suggests developmental changes in the psychological mechanisms underlying early instrumental helping. The results are explained by a developmental account according to which helping only gradually becomes an other-oriented and need-based behavior in the first years of life.

I, Robot: How Human Appearance and Mind Attribution Relate to the Perceived Danger of Robots

Social robots become increasingly human-like in appearance and behaviour. However, a large body of research shows that these robots tend to elicit negative feelings of eeriness, danger, and threat. In the present study, we explored whether and how human-like appearance and mind-attribution contribute to these negative feelings and clarified possible underlying mechanisms. Participants were presented with pictures of mechanical, humanoid, and android robots, and physical anthropomorphism (Studies 1–3), attribution of mind perception of agency and experience (Studies 2 and 3), threat to human–machine distinctiveness, and damage to humans and their identity were assessed for all three robot types. Replicating earlier research, human–machine distinctiveness mediated the influence of anthropomorphic appearance on the perceived damage for humans and their identity, and this mediation was due to anthropomorphic appearance of the robot. Perceived agency and experience did not show similar mediating effects on human–machine distinctiveness, but a positive relation with perceived damage for humans and their identity. Possible explanations are discussed.

Saliency at first sight: instant identity referential advantage toward a newly met partner

Neutral information enjoys beneficial processing when it is associated with self and significant others, but less is known about how the identity referential advantage is constructed in the initial stages of a relationship. We offer a novel solution by asking if a newly met stranger could provide a processing advantage in a shape-identity matching task where shapes were associated with the names of different identities. Each participant was paired with a newly met partner in a joint shape-identity matching task in which three shapes were associated with the names of the participant or his/her best friend, the partner, and a stranger, respectively. The participants judged whether or not the shape and name correctly matched.

Intriguingly, the trials related to a newly met partner exhibited instant referential saliency, which was more accurate and faster than that related to the stranger’s name (baseline) when the partner was physically present (experiments 1, 2, 4, 5), but not when the partner was absent (experiment 3). Self-advantage, however, was robust and lasting. The precursor of physical presence when forming referential saliency toward a stranger and its distinct temporal dynamics imply a novel referential benefit unendowed with familiarity, which is qualitatively different from the well-documented self/friend-advantage effect.

Saccadic eye movements do not trigger a joint Simon effect

Although the joint Simon task (JST) has been investigated for more than a decade, its cause is still unclear. According to ideomotor views of action control, action effects are a commonly cited explanation. However, action effects are usually confounded with the actions producing such effects. We combined a JST with eye tracking and asked participants to respond by performing specific saccades. Saccades were followed by visual feedback (central vs. lateral feedback), serving as the action effect. This arrangement allowed us to isolate actions from action effects and, also to prevent each actor from seeing the reciprocal actions of the other actor. In this saccadic JST, we found a significant compatibility effect in the individual setting. The typical enhanced compatibility effect in the joint setting of the JST was absent with central action feedback and even when lateralized visual action feedback was provided. Our findings suggest that the perception of action effects alone might not be sufficient to modulate compatibility effects for eye movements. The presence of a compatibility effect in the individual setting shows the specific requirements of a saccadic compatibility task - the requirement to perform prosaccades to compatible and antisaccades to incompatible target locations. The lack of a difference between compatibility effects in joint and individual settings and the lack of a modulation of the compatibility effect through lateralized visual action feedback shows that the finding of a joint Simon effect that has frequently been reported for manual responses is absent for saccadic responses.

Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Despite considerable interest in both action perception and social attention over the last 2 decades, there has been surprisingly little investigation concerning how the manual actions of other humans orient visual attention. The present review draws together studies that have measured the orienting of attention, following observation of another's goal-directed action. Our review proposes that, in line with the literature on eye gaze, action is a particularly strong orienting cue for the visual system. However, we additionally suggest that action may orient visual attention using mechanisms, which gaze direction does not (i.e., neural direct mapping and corepresentation). Finally, we review the implications of these gaze-independent mechanisms for the study of attention to action. We suggest that our understanding of attention to action may benefit from being studied in the context of joint action paradigms, where the role of higher level action goals and social factors can be investigated.

The Multimodal Go-Nogo Simon Effect: Signifying the Relevance of Stimulus Features in the Go-Nogo Simon Paradigm Impacts Event Representations and Task Performance

Numerous studies have shown that stimulus-response-compatibility (SRC) effects in the go-nogo version of the Simon task can be elicited as a result of performing the task together with another human or non-human agent (e.g., a Japanese-waving-cat, a working-clock, or a ticking-metronome). A parsimonious explanation for both social and non-social SRC effects is that highlighting the spatial significance of alternative (non-/social) action events makes action selection more difficult. This holds even when action events are task-irrelevant. Recent findings, however, suggest that this explanation holds only for cases of a modality correspondence between the Simon task as such (i.e., auditory or visual) and the alternative (non-/social) action event that needs to be discriminated. However, based on the fact that perception and action are represented by the same kind of codes, an event that makes the go-nogo decision more challenging should impact go-nogo Simon task performance. To tackle this issue, the present study tested if alternative stimulus events that come from a different sensory modality do impact SRC effects in the go-nogo version of the Simon task. This was tested in the presence and absence of alternative action events of a human co-actor. In a multimodal (auditory–visual) go-nogo Simon paradigm, participants responded to their assigned stimulus – e.g., a single auditory stimulus while ignoring the alternative visual stimulus or vice versa – in the presence or absence of a human co-actor (i.e., joint and single go-nogo condition). Results showed reliable SRCs in both, single and joint go-nogo Simon task conditions independent of the modality participants had to respond to. Although a correspondence between stimulus material and attention-grabbing event might be an efficient condition for SRCs to emerge, the driving force underlying the emergence of SRCs rather appears to be whether the attentional focus prevents or facilitates alternative events to be integrated. Thus, under task conditions in which the attentional focus is sufficiently broad to enable the integration and thus cognitive representation of alternative events, go-nogo decisions become more difficult, resulting in reliable SRCs in single and joint go-nogo Simon tasks.

How (not) to increase older adults' tendency to anthropomorphise in serious games

Among elderly, the use of serious games steadily increases. Research shows that anthropomorphising digital agents (i.e., ascribing human characteristics to them) has positive short-term consequences on interactions with digital agents. However, whether these effects can also be observed over a long-term period and in a real-life setting is unknown. In two studies, we investigated the important long-term consequences of anthropomorphism among older adults (age > 50) to increase involvement in serious games. Participants read either a story that highly anthropomorphized the digital agent of a training game, or a low anthropomorphism story about that agent. To investigate long-term effect, they played the training game for three weeks, and gaming data was assessed (number of games played, time of playing, points gained). While on the short-term, the anthropomorphic story increased the humanness of the agent (Study 1), no long-term effects where found (Study 2). Furthermore, an anthropomorphic story had no influence on the gaming outcome. Our results inform app developers about which techniques are useful to humanise digital agents.

When a Social Experimenter Overwrites Effects of Salient Objects in an Individual Go/No-Go Simon Task - An ERP Study

When two persons share a Simon task, a joint Simon effect occurs. The task co-representation account assumes that the joint Simon effect is the product of a vicarious representation of the co-actor's task. In contrast, recent studies show that even (non-human) event-producing objects could elicit a Simon effect in an individual go/no-go Simon task arguing in favor of the referential coding account. For the human-induced Simon effect, a modulation of the P300 component in Electroencephalography (EEG) is typically considered as a neural indicator of the joint Simon effect and task co-representation. Showing that the object-induced Simon effects also modulates the P300 would lead to a re-evaluation of the interpretation of the P300 in individual go/no-go and joint Simon task contexts. To do so, the present study conceptually replicated Experiment 1 from Dolk et al. (2013a) adding EEG recordings and an experimenter controlling the EEG computer to test whether a modulation of the P300 can also be elicited by adding a Japanese waving cat to the task context. Subjects performed an individual go/no-go Simon task with or without a cat placed next to them. Results show an overall Simon effect regardless of the cat's presence and no modulatory influence of the cat on the P300 (Experiment 1), even when conceivably interfering context factors are diminished (Experiment 2). These findings may suggest that the presence of a spatially aligned experimenter in the laboratory may produce an overall Simon effect overwriting a possible modulation of the Japanese waving cat.

Spontaneous Theory of Mind is reduced for nonhuman-like agents as compared to human-like agents

Theory of Mind research has shown that we spontaneously take into account other's beliefs. In the current study, we investigate, with a spontaneous Theory of Mind (ToM) task, if this belief representation also applies to nonhuman-like agents. In a series of three experiments, we show here that we do not spontaneously take into account beliefs of nonhuman-like others, or at least we do it to a lesser extent than for human and human-like agents. Further, the experience we have with the other agent, in our case a dog, does not modulate spontaneous ToM: the same pattern of results was obtained when dog owners and no owners were compared. However, when more attention was attracted to the dog behavior, participants' behavior was influenced by the beliefs of the dog. In sum, spontaneous belief representation seems to be primarily restricted to human and human-like agents, but can be facilitated when more attention is drawn to a nonhuman-like agent.

The influence of passionate love on self-other discrimination during joint action

Prior research on romantic relationships suggests that being in love involves a blurring of self-other cognitive boundaries. However, this research has focused so far on conceptual self-representation, related to the individual's traits or interests. The present study tested the hypothesis that passionate love involves a reduced discrimination between the self and the romantic partner at a bodily level, as indexed by an increased Joint Simon effect (JSE), and we further examined whether this self-other discrimination correlated with the passion felt for the partner. As predicted, we found an increased JSE when participants performed the Joint Simon Task with their romantic partner compared with a friend of the opposite sex. Providing support for the self-expansion model of love (Aron and Aron in Pers Relatsh 3(1):45-58, 1996), this result indicates that romantic relationships blur the boundaries between the self and the romantic partner at a bodily level. Furthermore, the strength of romantic feelings was positively correlated with the magnitude of the JSE when sharing the task with the romantic partner.

No evidence for shared representations of task sets in joint task switching

It has been suggested that actors co-represent a shared task context when they perform a task in a joint fashion. The present study examined the possibility of co-representation in joint task switching, in which two actors shared two tasks that switched randomly across trials. Experiment 1 showed that when an actor performed the tasks individually, switch costs were obtained if the actors responded on the previous trial (go trial), but not if they did not respond (no-go trial). When two actors performed the tasks jointly, switch costs were obtained if the actor responded on the previous trial (actor-repeat trials) but not if the co-actor responded (actor-switch trials). In Experiment 2, a single actor performed both tasks of the joint condition to test whether the findings of Experiment 1 were due to the use of different response sets by the two actors. Switch costs were obtained for both repetitions and alternations of the response set, which rules out this possibility. Taken together, our findings provided little support for the idea that actors co-represent the task sets of their co-actors.

No evidence of task co-representation in a joint Stroop task

People working together on a task must often represent the goals and salient items of their partner. The aim of the present study was to study the influence of joint task representations in an interference task in which the congruency relies on semantic identity. If task representations are shared between partners in a joint Stroop task (co-representation account), we hypothesized that items in the response set of one partner might influence performance of the other. In Experiment 1, pairs of participants sat side by side. Each participant was instructed to press one of two buttons to indicate which of two colors assigned to them was present, ignoring the text and responding only to the pixel color. There were three types of incongruent distractor words: names of colors from their own response set, names of colors from the other partner's response set, and neutral words for colors not used as font colors. The results of Experiment 1 showed that when people were doing this task together, distractor words from the partner's response set interfered more than neutral words and just as much as the words from their own response color set. However, in three follow-up experiments (Experiments 2a, 2b, and 2c), we found an elevated interference for the other response-set words even though no co-actor was present. The overall pattern of results across our study suggests that an alternative response set, regardless of whether it belonged to a co-actor or to a non-social no-go condition, evoked equal amounts of interference comparable to those of the own response set. Our findings are in line with a theory of common coding, in which all events-irrespective of their social nature-are represented and can influence behavior.

Enhanced encoding of the co-actor's target stimuli during a shared non-motor task

Task co-representation has been proposed to rely on the motor brain areas’ capacity to represent others’ action plans similarly to one's own. The joint memory (JM) effect suggests that working in parallel with others influences the depth of incidental encoding: Other-relevant items are better encoded than non-task-relevant items. Using this paradigm, we investigated whether task co-representation could also emerge for non-motor tasks. In Experiment 1, we found enhanced recall performance to stimuli relevant to the co-actor also when the participants’ task required non-motor responses (counting the target words) instead of key-presses. This suggests that the JM effect did not depend on simulating the co-actor's motor responses. In Experiment 2, direct visual access to the co-actor and his actions was found to be unnecessary to evoke the JM effect in case of the non-motor, but not in case of the motor task. Prior knowledge of the co-actor's target category is sufficient to evoke deeper incidental encoding. Overall, these findings indicate that the capacity of task co-representation extends beyond the realm of motor tasks: Simulating the other's motor actions is not necessary in this process.

Social Transmission of Experience of Agency: An Experimental Study

The sense of controlling one’s own actions is fundamental to normal human mental function, and also underlies concepts of social responsibility for action. However, it remains unclear how the wider social context of human action influences sense of agency. Using a simple experimental design, we investigated, for the first time, how observing the action of another person or a robot could potentially influence one’s own sense of agency. We assessed how observing another’s action might change the perceived temporal relationship between one’s own voluntary actions and their outcomes, which has been proposed as an implicit measure of sense of agency. Working in pairs, participants chose between two action alternatives, one rewarded more frequently than the other, while watching a rotating clock hand. They judged, in separate blocks, either the time of their own action, or the time of a tone that followed the action. These were compared to baseline judgements of actions alone, or tones alone, to calculate the perceptual shift of action toward outcome and vice versa. Our design focused on how these two dependent variables, which jointly provide an implicit measure of sense of agency, might be influenced by observing another’s action. In the observational group, each participant could see the other’s actions. Multivariate analysis showed that the perceived time of action and tone shifted progressively toward the actual time of outcome with repeated experience of this social situation. No such progressive change occurred in other groups for whom a barrier hid participants’ actions from each other. However, a similar effect was observed in the group that viewed movements of a human-like robotic hand, rather than actions of another person. This finding suggests that observing the actions of others increases the salience of the external outcomes of action and this effect is not unique to observing human agents. Social contexts in which we see others controlling external events may play an important role in mentally representing the impact of our own actions on the external world.

Joint Simon effects for non-human co-actors

Social interactions with non-biological agents and interactions with technical devices have become increasingly important over the last years. Recent studies investigating the interactions between humans and non-human agents showed rather inconsistent results. While the joint Simon effect (cSE) was found to be absent for non-human co-actors like virtual wooden hands, other studies showed pronounced cSEs when the co-actor was a real event-producing object. However, an often overlooked difference between these studies is the way these co-actors delivered response events. Studies replacing the co-actor by event-producing objects used a continuous response mode, while in studies using wooden hands, the co-actor always produced action effects in a task-related, turn-taking mode. In a series of four experiments, we systematically tested the effects of the response mode on the size of the cSE. The cSE was larger when the co-actor produced events in a turn-taking response mode than in a continuous response mode. Furthermore, we consistently found reliable cSEs for different kinds of virtual non-human co-actors (including a Japanese waving cat, scrambled patterns, and a wooden hand), and found no difference in the size of the cSE between human and non-human co-actors. We discuss possible mechanisms explaining why a cSE might be present or absent when sharing tasks with virtual non-human co-actors.

The joint flanker effect and the joint Simon effect: On the comparability of processes underlying joint compatibility effects

Previous studies observed compatibility effects in different interference paradigms such as the Simon and flanker task even when the task was distributed across two co-actors. In both Simon and flanker tasks, performance is improved in compatible trials relative to incompatible trials if one actor works on the task alone as well as if two co-actors share the task. These findings have been taken to indicate that actors automatically co-represent their co-actor's task. However, recent research on the joint Simon and joint flanker effect suggests alternative non-social interpretations. To which degree both joint effects are driven by the same underlying processes is the question of the present study, and it was scrutinized by manipulating the visibility of the co-actor. While the joint Simon effect was not affected by the visibility of the co-actor, the joint flanker effect was reduced when participants did not see their co-actors but knew where the co-actors were seated. These findings provide further evidence for a spatial interpretation of the joint Simon effect. In contrast to recent claims, however, we propose a new explanation of the joint flanker effect that attributes the effect to an impairment in the focusing of spatial attention contingent on the visibility of the co-actor.

Representation and Incorporation of Close Others’ Responses

We propose a new model of social influence, which can occur spontaneously and in the absence of typically assumed motives. We assume that perceivers routinely construct representations of other people’s experiences and responses (beliefs, attitudes, emotions, and behaviors), when observing others’ responses or simulating the responses of unobserved others. Like representations made accessible by priming, these representations may then influence the process that generates perceivers’ own responses, without intention or awareness, especially when there is a strong social connection to the other. We describe evidence for the basic properties and important moderators of this process, which distinguish it from other mechanisms such as informational, normative, or social identity influence. The model offers new perspectives on the role of others’ values in producing cultural differences, the persistence and power of stereotypes, the adaptive reasons for being influenced by others’ responses, and the impact of others’ views about the self.

With peppermints you're not my prince: aroma modulates self-other integration

Recent studies showed that self-other integration, as indexed by the joint Simon effect (JSE), can be modulated by biasing participants towards particular (integrative vs. exclusive) cognitive-control states. Interestingly, there is evidence suggesting that such control states can be induced by particular odors: stimulating odors (e.g., peppermint aroma) seem to induce a more focused, exclusive state; relaxing odors (e.g., lavender aroma) are thought to induce a broader, more integrative state. In the present study, we tested the possible impact of peppermint and lavender aromas on self-other integration. Pairs of participants performed the joint Simon task in an either peppermint- or lavender-scented testing room. Results showed that both aromas modulated the size of the JSE, although they had a dissociable effect on reaction times (RTs) and percentage of errors (PEs). Whilst the JSE in RTs was found to be less pronounced in the peppermint group, compared to the lavender and no-aroma groups, the JSE in PEs was significantly more pronounced in the lavender group, compared to the peppermint and no-aroma group. These results are consistent with the emerging literature suggesting that the degree of self-other integration does not reflect a trait but a particular cognitive state, which can be biased towards excluding or integrating the other in one’s self-representation.

Competitive Game Play Attenuates Self-Other Integration during Joint Task Performance

Joint task performance is facilitated by sharing and integrating each other’s action representations. Research has shown that the amount of this so-called self-other integration depends on situational aspects related to the social context, including differences in the social relationship between co-acting individuals. There are indications that a cooperative relationship facilitates self-other integration while a competitive relationship results in more individualistic task performance. However, findings from previous studies in which the cooperative or competitive element was manipulated during task performance are inconsistent. Therefore, the present study aimed to manipulate the social relationship between two individuals prior to performing a social Simon task. This task is frequently used to measure self-other integration and distinction processes. A mixed-within-and-between-subjects design was used in which three groups of participants performed both a standard Simon task and a social Simon task after having played a Tetris game either individually, in cooperation with a co-actor, or in competition against another participant. Performance on the standard Simon task was not affected by the Tetris manipulation. However, a sustained effect of the induced cooperative versus competitive relationship was found on the social Simon Task. Less self-other integration was found in participants who had first played a competitive Tetris game compared to participants who had played a cooperative or solo version of the game. The current study thus demonstrates that an established cooperative or competitive relationship is sufficient to modulate the degree of self-other integration on subsequent joint task performance. Importantly, by using Tetris, attention to others’ actions was beneficial both during cooperative and competitive game play and can thus not explain the competition-induced reduction of self-other integration.

Individual differences in action co-representation: not personal distress or subclinical psychotic experiences but sex composition modulates joint action performance

Successful social interaction requires the ability to integrate as well as distinguish own and others’ actions. Normally, the integration and distinction of self and other are a well-balanced process, occurring without much effort or conscious attention. However, not everyone is blessed with the ability to balance self–other distinction and integration, resulting in personal distress in reaction to other people’s emotions or even a loss of self [e.g., in (subclinical) psychosis]. Previous research has demonstrated that the integration and distinction of others’ actions cause interference with one’s own action performance (commonly assessed with a social Simon task). The present study had two goals. First, as previous studies on the social Simon effect employed relatively small samples (N < 50 per test), we aimed for a sample size that allowed us to test the robustness of the action interference effect. Second, we tested to what extent action interference reflects individual differences in traits related to self–other distinction (i.e., personal distress in reaction to other people’s emotions and subclinical psychotic symptoms). Based on a questionnaire study among a large sample (N = 745), we selected a subsample (N = 130) of participants scoring low, average, or high on subclinical psychotic symptoms, or on personal distress. The selected participants performed a social Simon task. Results showed a robust social Simon effect, regardless of individual differences in personal distress or subclinical psychotic symptoms. However, exploratory analyses revealed that the sex composition of interaction pairs modulated social Simon effects. Possible explanations for these findings are discussed.

Exploring social influences on the joint Simon task: empathy and friendship

Tasks for which people must act together to achieve a goal are a feature of daily life. The present study explored social influences on joint action using a Simon procedure for which participants (n = 44) were confronted with a series of images of hands and asked to respond via button press whenever the index finger wore a ring of a certain color (red or green) regardless of pointing direction (left or right). In an initial joint condition they performed the task while sitting next to another person (friend or stranger) who responded to the other color. In a subsequent individual condition they repeated the task on their own; additionally, they completed self-report tests of empathy. Consistent with past research, participants reacted more quickly when the finger pointed toward them rather than their co-actor (the Simon Effect or SE). The effect remained robust when the co-actor was no longer present and was unaffected by degree of acquaintance; however, its magnitude was correlated positively with empathy only among friends. For friends, the SE was predicted by cognitive perspective taking when the co-actor was present and by propensity for fantasizing when the co-actor was absent. We discuss these findings in relation to social accounts (e.g., task co-representation) and non-social accounts (e.g., referential coding) of joint action.

How watching Pinocchio movies changes our subjective experience of extrapersonal space

The way we experience the space around us is highly subjective. It has been shown that motion potentialities that are intrinsic to our body influence our space categorization. Furthermore, we have recently demonstrated that in the extrapersonal space, our categorization also depends on the movement potential of other agents. When we have to categorize the space as "Near" or "Far" between a reference and a target, the space categorized as "Near" is wider if the reference corresponds to a biological agent that has the potential to walk, instead of a biological and non-biological agent that cannot walk. But what exactly drives this "Near space extension"? In the present paper, we tested whether abstract beliefs about the biological nature of an agent determine how we categorize the space between the agent and an object. Participants were asked to first read a Pinocchio story and watch a correspondent video in which Pinocchio acts like a real human, in order to become more transported into the initial story. Then they had to categorize the location ("Near" or "Far") of a target object located at progressively increasing or decreasing distances from a non-biological agent (i.e., a wooden dummy) and from a biological agent (i.e., a human-like avatar). The results indicate that being transported into the Pinocchio story, induces an equal "Near" space threshold with both the avatar and the wooden dummy as reference frames.

Interacting hands: the role of attention for the joint Simon effect

Recent research in monkeys and humans has shown that the presence of the hands near an object enhances spatial processing for objects presented near the hand. This study aimed to test the effect of hand position on the joint Simon effect. In Experiment 1, two human co-actors shared a Simon task while placing their response hands either near the objects appearing on the monitor or away from the monitor. Experiment 2 varied each co-actor’s hand position independently. Experiment 3 tested whether enhanced spatial processing for objects presented near the hand is obtained when replacing one of the two co-actors by a non-human event-producing rubber hand. Experiment 1 provided evidence for a Simon effect. Hand position significantly modulated the size of the Simon effect in the joint Simon task showing an increased Simon effect when the hands of both actors were located near the objects on the monitor, than when they were located away from the monitor. Experiment 2 replicated this finding showing an increased Simon effect when the actor’s hand was located near the objects on the monitor, but only when the co-actor also produced action events in spatial reference. A similar hand position effect was observed in Experiment 3 when a non-human rubber hand replaced the human co-actor. These findings suggest that external action events that are produced in spatial reference bias the distribution of attention to the area near the hand. This strengthens the weight of the spatial response codes (referential coding) and hence increases the joint Simon effect.

The joint Simon effect: a review and theoretical integration

The social or joint Simon effect has been developed to investigate how and to what extent people mentally represent their own and other persons' action/task and how these cognitive representations influence an individual's own behavior when interacting with another person. Here, we provide a review of the available evidence and theoretical frameworks. Based on this review, we suggest a comprehensive theory that integrates aspects of earlier approaches–the Referential Coding Account. This account provides an alternative to the social interpretation of the (joint) go-nogo Simon effect (aka the social Simon effect) and is able to integrate seemingly opposite findings on joint action.

The joint Simon effect depends on perceived agency, but not intentionality, of the alternative action

A co-actor's intentionality has been suggested to be a key modulating factor for joint action effects like the joint Simon effect (JSE). However, in previous studies intentionality has often been confounded with agency defined as perceiving the initiator of an action as being the causal source of the action. The aim of the present study was to disentangle the role of agency and intentionality as modulating factors of the JSE. In Experiment 1, participants performed a joint go/nogo Simon task next to a co-actor who either intentionally controlled a response button with own finger movements (agency+/intentionality+) or who passively placed the hand on a response button that moved up and down on its own as triggered by computer signals (agency−/intentionality−). In Experiment 2, we included a condition in which participants believed that the co-actor intentionally controlled the response button with a Brain-Computer Interface (BCI) while placing the response finger clearly besides the response button, so that the causal relationship between agent and action effect was perceptually disrupted (agency−/intentionality+). As a control condition, the response button was computer controlled while the co-actor placed the response finger besides the response button (agency−/intentionality−). Experiment 1 showed that the JSE is present with an intentional co-actor and causality between co-actor and action effect, but absent with an unintentional co-actor and a lack of causality between co-actor and action effect. Experiment 2 showed that the JSE is absent with an intentional co-actor, but no causality between co-actor and action effect. Our findings indicate an important role of the co-actor's agency for the JSE. They also suggest that the attribution of agency has a strong perceptual basis.

The role of attention in a joint-action effect

The most common explanation for joint-action effects has been the action co-representation account in which observation of another's action is represented within one's own action system. However, recent evidence has shown that the most prominent of these joint-action effects (i.e., the Social Simon effect), can occur when no co-actor is present. In the current work we examined whether another joint-action phenomenon (a movement congruency effect) can be induced when a participant performs their part of the task with a different effector to that of their co-actor and when a co-actor's action is replaced by an attention-capturing luminance signal. Contrary to what is predicted by the action co-representation account, results show that the basic movement congruency effect occurred in both situations. These findings challenge the action co-representation account of this particular effect and suggest instead that it is driven by bottom-up mechanisms.

When co-action eliminates the Simon effect: disentangling the impact of co-actor's presence and task sharing on joint-task performance

This study aimed at assessing whether the mere belief of performing a task with another person, who is in charge of the complementary part of the task, is sufficient for the so-called joint Simon effect to occur. In all three experiments of the study, participants sat alone in a room and underwent two consecutive Go/NoGo tasks that were identical except for the instructions. In Experiment 1, participants performed the task first individually (baseline task), and then either co-acting with another person who responded from an unknown location to the NoGo stimuli (joint task) or imaging themselves responding to the NoGo stimuli (imaginative task). Relative to the baseline, the instructions of the imaginative task made the Simon effect occur, while those of the joint task were ineffective in eliciting the effect. This result suggests that sharing a task with a person who is known to be in charge of the complementary task, but is not physically present, is not sufficient to induce the representation of an alternative response able to produce interference, which happens instead when the instructions explicitly require to imagine such a response. Interestingly, we observed that when the Simon effect was already present in the baseline task (i.e., when the response alternative to the Go response was represented in the individual task due to non-social factors), it disappeared in the joint task. We propose that, when no information about the co-actor's position is available, the division of labor between the participant and co-actor allows participants to filter out the possible (incidental) representation of the alternative response from their task representation, thus eliminating potential sources of interference. This account is supported by the results of Experiments 2 and 3 and suggests that under certain circumstances task-sharing may reduce the interference produced by the irrelevant information, rather than increase it.

Neural correlates of non-verbal social interactions: a dual-EEG study

Successful non-verbal social interaction between human beings requires dynamic and efficient encoding of others' gestures. Our study aimed at identifying neural markers of social interaction and goal variations in a non-verbal task. For this, we recorded simultaneously the electroencephalogram from two participants (dual-EEG), an actor and an observer, and their arm/hand kinematics in a real face-to-face paradigm. The observer watched "biological actions" performed by the human actor and "non-biological actions" performed by a robot. All actions occurred within an interactive or non-interactive context depending on whether the observer had to perform a complementary action or not (e.g., the actor presents a saucer and the observer either places the corresponding cup or does nothing). We analysed the EEG signals of both participants (i.e., beta (~20 Hz) oscillations as an index of cortical motor activity and motor related potentials (MRPs)). We identified markers of social interactions by synchronising EEG to the onset of the actor's movement. Movement kinematics did not differ in the two context conditions and the MRPs of the actor were similar in the two conditions. For the observer, however, an observation-related MRP was measured in all conditions but was more negative in the interactive context over fronto-central electrodes. Moreover, this feature was specific to biological actions. Concurrently, the suppression of beta oscillations was observed in the actor's EEG and the observer's EEG rapidly after the onset of the actor's movement. Critically, this suppression was stronger in the interactive than in the non-interactive context despite the fact that movement kinematics did not differ in the two context conditions. For the observer, this modulation was observed independently of whether the actor was a human or a robot. Our results suggest that acting in a social context induced analogous modulations of motor and sensorimotor regions in observer and actor. Sharing a common goal during an interaction seems thus to evoke a common representation of the global action that includes both actor and observer movements.