FMRI supports the sensorimotor theory of motor resonance

Closing the loop in minimally supervised human-robot interaction: formative and summative feedback

Human instructors fluidly communicate with hand gestures, head and body movements, and facial expressions, but robots rarely leverage these complementary cues. A minimally supervised social robot with such skills could help people exercise and learn new activities. Thus, we investigated how nonverbal feedback from a humanoid robot affects human behavior. Inspired by the education literature, we evaluated formative feedback (real-time corrections) and summative feedback (post-task scores) for three distinct tasks: positioning in the room, mimicking the robot's arm pose, and contacting the robot's hands. Twenty-eight adults completed seventy-five 30-s-long trials with no explicit instructions or experimenter help. Motion-capture data analysis shows that both formative and summative feedback from the robot significantly aided user performance. Additionally, formative feedback improved task understanding. These results show the power of nonverbal cues based on human movement and the utility of viewing feedback through formative and summative lenses.

The vocal side of empathy: neural correlates of pain perception in spoken complaints

In the extensive neuroimaging literature on empathy for pain, few studies have investigated how this phenomenon may relate to everyday social situations such as spoken interactions. The present study used functional Magnetic Resonance Imaging (fMRI) to assess how complaints, as vocal expressions of pain, are empathically processed by listeners and how these empathic responses may vary based on speakers' vocal expression and cultural identity. Twenty-four French participants listened to short utterances describing a painful event, which were either produced in a neutral-sounding or complaining voice by both in-group (French) and out-group (French Canadian) speakers. Results suggest that the perception of suffering from a complaining voice increased activity in the emotional voice areas, composed of voice-sensitive temporal regions interacting with prefrontal cortices and the amygdala. The Salience and Theory of Mind networks, associated with affective and cognitive aspects of empathy, also showed prosody-related activity and specifically correlated with behavioral evaluations of suffering by listeners. Complaints produced by in- vs out-group speakers elicited sensorimotor and default mode activity, respectively, suggesting accent-based changes in empathic perspective. These results, while reaffirming the role of key networks in tasks involving empathy, highlight the importance of vocal expression information and social categorization processes when perceiving another's suffering during social interactions.

Partial repetition between action plans delays responses to ideomotor compatible stimuli

Often one must depart from an intended course of events to react to sudden situational demands before resuming his or her original action retained in working memory. Retaining an action plan in working memory (WM) can delay or facilitate the execution of an intervening action when the action features of the two action plans partly overlap (partial repetition) compared to when they do not overlap. We investigated whether partial repetition costs (PRCs) or benefits (PRBs) occur when the intervening event is an ideomotor-compatible stimulus that is a biological representation of the response required by the participant. Participants viewed two visual events and retained an action plan to the first event (A) while executing a speeded response to the second, intervening event (B). In Experiment 1A, the two visual events were ideomotor compatible, non-ideomotor compatible (abstract), or one was ideomotor compatible, and the other abstract. Results showed PRCs for all event A-B stimulus combinations with reduced PRCs for intervening, ideomotor compatible events. In contrast to previous research, there was no evidence that ideomotor-compatible actions were automatic and bypassed the selection bottleneck. Experiment 1B confirmed PRCs for ideomotor compatible stimuli that more accurately mimicked the required response. Findings suggest that mechanisms for activating, selecting, and retaining action plans are similar between ideomotor compatible and abstract visual events. We conclude that PRCs occur in response to intervening events when action plans are generated offline and rely on WM, including those for ideomotor-compatible stimuli; but PRBs may be restricted to actions generated online. This conclusion is consistent with the perceptual-motor framework by Goodale and Milner (Trends in Neuroscience 15:22-25, 1992).

Coding of Self and Other's Future Choices in Dorsal Premotor Cortex during Social Interaction

Representing others' intentions is central to primate social life. We explored the role of dorsal premotor cortex (PMd) in discriminating between self and others' behavior while two male rhesus monkeys performed a non-match-to-goal task in a monkey-human paradigm. During each trial, two of four potential targets were randomly presented on the right and left parts of a screen, and the monkey or the human was required to choose the one that did not match the previously chosen target. Each agent had to monitor the other's action in order to select the correct target in that agent's own turn. We report neurons that selectively encoded the future choice of the monkey, the human agent, or both. Our findings suggest that PMd activity shows a high degree of self-other differentiation during face-to-face interactions, leading to an independent representation of what others will do instead of entailing self-centered mental rehearsal or mirror-like activities.

The role of sensorimotor experience in the development of mimicry in infancy

During social interactions we often have an automatic and unconscious tendency to copy or 'mimic' others' actions. The dominant view on the neural basis of mimicry appeals to an automatic coupling between perception and action. It has been suggested that this coupling is formed through associative learning during correlated sensorimotor experience. Although studies with adult participants have provided support for this hypothesis, little is known about the role of sensorimotor experience in supporting the development of perceptual-motor couplings, and consequently mimicry behaviour, in infancy. Here we investigated whether the extent to which an observed action elicits mimicry depends on the opportunity an infant has had to develop perceptual-motor couplings for this action through correlated sensorimotor experience. We found that mothers' tendency to imitate their 4-month-olds' facial expressions during a parent-child interaction session was related to infants' facial mimicry as measured by electromyography. Maternal facial imitation was not related to infants' mimicry of hand actions, and instead we found preliminary evidence that infants' tendency to look at their own hands may be related to their tendency to mimic hand actions. These results are consistent with the idea that mimicry is supported by perceptual-motor couplings that are formed through correlated sensorimotor experience obtained by observing one's own actions and imitative social partners.

Investigating Mirror System (MS) Activity in Adults with ASD When Inferring Others' Intentions Using Both TMS and EEG

ASD is associated with mentalizing deficits that may correspond with atypical mirror system (MS) activation. We investigated MS activity in adults with and without ASD when inferring others' intentions using TMS-induced motor evoked potentials (MEPs) and mu suppression measured by EEG. Autistic traits were measured for all participants. Our EEG data show, high levels of autistic traits predicted reduced right mu (8-10 Hz) suppression when mentalizing. Higher left mu (8-10 Hz) suppression was associated with superior mentalizing performances. Eye-tracking and TMS data showed no differences associated with autistic traits. Our data suggest ASD is associated with reduced right MS activity when mentalizing, TMS-induced MEPs and mu suppression measure different aspects of MS functioning and the MS is directly involved in inferring intentions.

Influence of anxiety and alexithymia on brain activations associated with the perception of others' pain in autism

The circumstances under which empathy is altered in ASD remain unclear, as previous studies did not systematically find differences in brain activation between ASD and controls in empathy-eliciting paradigms, and did not always monitor whether differences were primarily due to ASD "per se", or to conditions overlapping with ASD, such as alexithymia and anxiety. Here, we collected fMRI data from 47 participants (22 ASD) viewing pictures depicting hands and feet of unknown others in painful, disgusting, or neutral situations. We computed brain activity for painful and disgusting stimuli (vs. neutral) in whole brain and in regions of interest among the brain areas typically activated during the perception of nociceptive stimuli. Group differences in brain activation disappeared when either alexithymia or anxiety - both elevated in the ASD group - were controlled for. Regression analyses indicated that the influence of symptoms was mainly shared between autistic symptomatology, alexithymia and anxiety or driven by unique contributions from alexithymia or anxiety. Our results suggest that affective empathy may be affected in ASD, but that this association is complex. The respective contribution of alexithymia and anxiety to decreased affective empathy of people with ASD may be due to the association of those psychiatric conditions with reduced motor resonance/Theory of Mind.

Mirroring 'meaningful' actions: Sensorimotor learning modulates imitation of goal-directed actions

Imitation is important in the development of social and technological skills throughout the lifespan. Experiments investigating the acquisition and modulation of imitation (and of its proposed neural substrate, the mirror neuron system) have produced evidence that the capacity for imitation depends on associative learning in which connections are formed between sensory and motor representations of actions. However, evidence that the development of imitation depends on associative learning has been found only for non-goal-directed actions. One reason for the lack of research on goal-directed actions is that imitation of such actions is commonly confounded with the tendency to respond in a spatially compatible manner. However, since the most prominent account of mirror neuron function, and hence of imitation, suggests that these cells encode goal-directed actions, it is important to establish whether sensorimotor learning can also modulate imitation of goal-directed actions. Experiment 1 demonstrated that imitation of goal-directed grasping can be measured while controlling for spatial compatibility, and Experiment 2 showed that this imitation effect can be modulated by sensorimotor training. Together, these data support the hypothesis that the capacity for behavioural imitation and the properties of the mirror neuron system are constructed in the course of development through associative learning.

Mirror neuron activation as a function of explicit learning: changes in mu-event-related power after learning novel responses to ideomotor compatible, partially compatible, and non-compatible stimuli

Questions regarding the malleability of the mirror neuron system (MNS) continue to be debated. MNS activation has been reported when people observe another person performing biological goal-directed behaviors, such as grasping a cup. These findings support the importance of mapping goal-directed biological behavior onto one's motor repertoire as a means of understanding the actions of others. Still, other evidence supports the Associative Sequence Learning (ASL) model which predicts that the MNS responds to a variety of stimuli after sensorimotor learning, not simply biological behavior. MNS activity develops as a consequence of developing stimulus-response associations between a stimulus and its motor outcome. Findings from the ideomotor literature indicate that stimuli that are more ideomotor compatible with a response are accompanied by an increase in response activation compared to less compatible stimuli; however, non-compatible stimuli robustly activate a constituent response after sensorimotor learning. Here, we measured changes in the mu-rhythm, an EEG marker thought to index MNS activity, predicting that stimuli that differ along dimensions of ideomotor compatibility should show changes in mirror neuron activation as participants learn the respective stimulus-response associations. We observed robust mu-suppression for ideomotor-compatible hand actions and partially compatible dot animations prior to learning; however, compatible stimuli showed greater mu-suppression than partially or non-compatible stimuli after explicit learning. Additionally, non-compatible abstract stimuli exceeded baseline only after participants explicitly learned the motor responses associated with the stimuli. We conclude that the empirical differences between the biological and ASL accounts of the MNS can be explained by Ideomotor Theory.

Baby steps: investigating the development of perceptual-motor couplings in infancy

There are cells in our motor cortex that fire both when we perform and when we observe similar actions. It has been suggested that these perceptual-motor couplings in the brain develop through associative learning during correlated sensorimotor experience. Although studies with adult participants have provided support for this hypothesis, there is no direct evidence that associative learning also underlies the initial formation of perceptual-motor couplings in the developing brain. With the present study we addressed this question by manipulating infants' opportunities to associate the visual and motor representation of a novel action, and by investigating how this influenced their sensorimotor cortex activation when they observed this action performed by others. Pre-walking 7-9-month-old infants performed stepping movements on an infant treadmill while they either observed their own real-time leg movements (Contingent group) or the previously recorded leg movements of another infant (Non-contingent control group). Infants in a second control group did not perform any steps and only received visual experience with the stepping actions. Before and after the training period we measured infants' sensorimotor alpha suppression, as an index of sensorimotor cortex activation, while they watched videos of other infants' stepping actions. While we did not find greater sensorimotor alpha suppression following training in the Contingent group as a whole, we nevertheless found that the strength of the visuomotor contingency experienced during training predicted the amount of sensorimotor alpha suppression at post-test in this group. We did not find any effects of motor experience alone. These results suggest that the development of perceptual-motor couplings in the infant brain is likely to be supported by associative learning during correlated visuomotor experience.

Mirror neurons: From origin to function

This article argues that mirror neurons originate in sensorimotor associative learning and therefore a new approach is needed to investigate their functions. Mirror neurons were discovered about 20 years ago in the monkey brain, and there is now evidence that they are also present in the human brain. The intriguing feature of many mirror neurons is that they fire not only when the animal is performing an action, such as grasping an object using a power grip, but also when the animal passively observes a similar action performed by another agent. It is widely believed that mirror neurons are a genetic adaptation for action understanding; that they were designed by evolution to fulfill a specific socio-cognitive function. In contrast, we argue that mirror neurons are forged by domain-general processes of associative learning in the course of individual development, and, although they may have psychological functions, they do not necessarily have a specific evolutionary purpose or adaptive function. The evidence supporting this view shows that (1) mirror neurons do not consistently encode action “goals”; (2) the contingency- and context-sensitive nature of associative learning explains the full range of mirror neuron properties; (3) human infants receive enough sensorimotor experience to support associative learning of mirror neurons (“wealth of the stimulus”); and (4) mirror neurons can be changed in radical ways by sensorimotor training. The associative account implies that reliable information about the function of mirror neurons can be obtained only by research based on developmental history, system-level theory, and careful experimentation.

Tinbergen on mirror neurons

Fifty years ago, Niko Tinbergen defined the scope of behavioural biology with his four problems: causation, ontogeny, survival value and evolution. About 20 years ago, there was another highly significant development in behavioural biology-the discovery of mirror neurons (MNs). Here, I use Tinbergen's original four problems (rather than the list that appears in textbooks) to highlight the differences between two prominent accounts of MNs, the genetic and associative accounts; to suggest that the latter provides the defeasible 'best explanation' for current data on the causation and ontogeny of MNs; and to argue that functional analysis, of the kind that Tinbergen identified somewhat misleadingly with studies of 'survival value', should be a high priority for future research. In this kind of functional analysis, system-level theories would assign MNs a small, but potentially important, role in the achievement of action understanding-or another social cognitive function-by a production line of interacting component processes. These theories would be tested by experimental intervention in human and non-human animal samples with carefully documented and controlled developmental histories.

Effect of levodopa on both verbal and motor representations of action in Parkinson's disease: a fMRI study

Previous studies have demonstrated that non-demented Parkinson's disease (PD) patients have a specific impairment of verb production compared with noun generation. One interpretation of this deficit suggested the influence of striato-frontal dysfunction on action-related verb processing. The aim of our study was to investigate cerebral changes after motor improvement due to dopaminergic medication on the neural circuitry supporting action representation in the brain as mediated by verb generation and motor imagery in PD patients. Functional magnetic resonance imaging on 8 PD patients in "ON" dopaminergic treatment state (DTS) and in "OFF" DTS was used to explore the brain activity during three different tasks: Object Naming (ObjN), Generation of Action Verbs (GenA) in which patients were asked to overtly say an action associated with a picture and mental simulation of action (MSoA) was investigated by asking subjects to mentally simulate an action related to a depicted object. The distribution of brain activities associated with these tasks whatever DTS was very similar to results of previous studies. The results showed that brain activity related to semantics of action is modified by dopaminergic treatment in PD patients. This cerebral reorganisation concerns mainly motor and premotor cortex suggesting an involvement of the putaminal motor loop according to the "motor" theory of verb processing.

The functional architecture of S1 during touch observation described with 7 T fMRI

Recent studies indicate that the primary somatosensory cortex (S1) is active not only when touch is physically perceived but also when it is merely observed to be experienced by another person. This social responsivity of S1 has important implications for our understanding of S1 functioning. However, S1 activity during touch observation has not been characterized in great detail to date. We focused on two features of the S1 functional architecture during touch observation, namely the topographical arrangement of index and middle finger receptive fields (RFs), and their dynamic shrinkage during concurrent activation. Both features have important implications for human behavior. We conducted two fMRI studies at 7 T, one where touch was physically perceived, and one where touch was observed. In the two experiments, participants either had their index finger and/or middle finger stimulated using paintbrushes, or just observed similar touch events on video. Our data show that observing and physically experiencing touch elicits overlapping activity changes in S1. In addition, observing touch to the index finger or the middle finger alone evoked topographically arranged activation foci in S1. Importantly, when co-activated, the index and middle finger RFs not only shrank during physical touch perception, but also during touch observation. Our data, therefore, indicate a similarity between the functional architecture of S1 during touch observation and physical touch perception with respect to single-digit topography and RF shrinkage. These results may allow the tentative conclusion that even primary somatosensory experiences, such as physical touch perception, can be shared amongst individuals.

FMRI evidence of 'mirror' responses to geometric shapes

Mirror neurons may be a genetic adaptation for social interaction [1]. Alternatively, the associative hypothesis [2], [3] proposes that the development of mirror neurons is driven by sensorimotor learning, and that, given suitable experience, mirror neurons will respond to any stimulus. This hypothesis was tested using fMRI adaptation to index populations of cells with mirror properties. After sensorimotor training, where geometric shapes were paired with hand actions, BOLD response was measured while human participants experienced runs of events in which shape observation alternated with action execution or observation. Adaptation from shapes to action execution, and critically, observation, occurred in ventral premotor cortex (PMv) and inferior parietal lobule (IPL). Adaptation from shapes to execution indicates that neuronal populations responding to the shapes had motor properties, while adaptation to observation demonstrates that these populations had mirror properties. These results indicate that sensorimotor training induced populations of cells with mirror properties in PMv and IPL to respond to the observation of arbitrary shapes. They suggest that the mirror system has not been shaped by evolution to respond in a mirror fashion to biological actions; instead, its development is mediated by stimulus-general processes of learning within a system adapted for visuomotor control.

Sensorimotor learning and the ontogeny of the mirror neuron system

Mirror neurons, which have now been found in the human and songbird as well as the macaque, respond to both the observation and the performance of the same action. It has been suggested that their matching response properties have evolved as an adaptation for action understanding; alternatively, these properties may arise through sensorimotor experience. Here I review mirror neuron response characteristics from the perspective of ontogeny; I discuss the limited evidence for mirror neurons in early development; and I describe the growing body of evidence suggesting that mirror neuron responses can be modified through experience, and that sensorimotor experience is the critical type of experience for producing mirror neuron responses.