Sense of Agency Beyond Sensorimotor Process: Decoding Self-Other Action Attribution in the Human Brain

Illusory Directional Sensation Induced by Asymmetric Vibrations Influences Sense of Agency and Velocity in Wrist Motions

Illusory directional sensations are generated through asymmetric vibrations applied to the fingertips and have been utilized to induce upper-limb motions in the rehabilitation and training of patients with visual impairment. However, its effects on motor control remain unclear. This study aimed to verify the effects of illusory directional sensations on wrist motion. We conducted objective and subjective evaluations of wrist motion during a motor task, while inducing an illusory directional sensation that was congruent or incongruent with wrist motion. We found that, when motion and illusory directional sensations were congruent, the sense of agency for motion decreased. This indicates an induction sensation of the hand being moved by the illusion. Interestingly, although no physical force was applied to the hand, the angular velocity of the wrist was higher in the congruent condition than that in the no-stimulation condition. The angular velocity of the wrist and electromyography signals of the agonist muscles were weakly positively correlated, suggesting that the participants may have increased their wrist velocity. In other words, the congruence between the direction of motion and illusory directional sensation induced the sensation of the hand being moved, even though the participants' wrist-motion velocity increased. This phenomenon can be explained by the discrepancy between the sensation of active motion predicted by the efferent copy, and that of actual motion caused by the addition of the illusion. The findings of this study can guide the design of novel rehabilitation methods.

Neural correlates of an illusionary sense of agency caused by virtual reality

Sense of agency (SoA) is the sensation that self-actions lead to ensuing perceptual consequences. The prospective mechanism emphasizes that SoA arises from motor prediction and its comparison with actual action outcomes, while the reconstructive mechanism stresses that SoA emerges from retrospective causal processing about the action outcomes. Consistent with the prospective mechanism, motor planning regions were identified by neuroimaging studies using the temporal binding (TB) effect, a behavioral measure often linked to implicit SoA. Yet, TB also occurs during passive observation of another's action, lending support to the reconstructive mechanism, but its neural correlates remain unexplored. Here, we employed virtual reality (VR) to modulate such observation-based SoA and examined it with functional magnetic resonance imaging (fMRI). After manipulating an avatar hand in VR, participants passively observed an avatar's "action" and showed a significant increase in TB. The binding effect was associated with the right angular gyrus and inferior parietal lobule, which are critical nodes for inferential and agency processing. These results suggest that the experience of controlling an avatar may potentiate inferential processing within the right inferior parietal cortex and give rise to the illusionary SoA without voluntary action.

Effects of Vibration-Based Generation of Timing of Tactile Perception on Upper Limb Function After Stroke: A Case Study

Sensorimotor dysfunction of the fingers and hands hinders the recovery of motor function post-stroke. Generally, hemiplegic patients are unable to properly control the dynamic friction generated between their fingers and objects during hand/finger muscle activity. In addition to sensory information, a sense of agency generated by the temporal synchronization of sensory prediction and sensory feedback is required to control this dynamic friction. In the present study, we utilized a novel rehabilitation device that transmits real-time fingertip contact information to a transducer in a case of stroke hemiplegia with sensorimotor deficits and stagnated hand/finger motor performance. Post-intervention, the patient's upper extremity motor function score (FMA-UE), which had previously been in a state of arrested recovery, improved from 51/66 to 61/66, especially in the wrist joints. Excessive grip force during object grasping and frequency of falling objects was notably decreased post-intervention. We believe that rehabilitation tasks using perceptual generation via transducer will be a new tool for the rehabilitation of post-stroke hand/finger sensorimotor deficits.

Temporal Dynamics of Brain Activity Predicting Sense of Agency over Muscle Movements

Our muscles are the primary means through which we affect the external world, and the sense of agency (SoA) over the action through those muscles is fundamental to our self-awareness. However, SoA research to date has focused almost exclusively on agency over action outcomes rather than over the musculature itself, as it was believed that SoA over the musculature could not be manipulated directly. Drawing on methods from human-computer interaction and adaptive experimentation, we use human-in-the-loop Bayesian optimization to tune the timing of electrical muscle stimulation so as to robustly elicit a SoA over electrically actuated muscle movements in male and female human subjects. We use time-resolved decoding of subjects' EEG to estimate the time course of neural activity which predicts reported agency on a trial-by-trial basis. Like paradigms which assess SoA over action consequences, we found that the late (post-conscious) neural activity predicts SoA. Unlike typical paradigms, however, we also find patterns of early (sensorimotor) activity with distinct temporal dynamics predicts agency over muscle movements, suggesting that the "neural correlates of agency" may depend on the level of abstraction (i.e., direct sensorimotor feedback versus downstream consequences) most relevant to a given agency judgment. Moreover, fractal analysis of the EEG suggests that SoA-contingent dynamics of neural activity may modulate the sensitivity of the motor system to external input.SIGNIFICANCE STATEMENT The sense of agency, the feeling of "I did that," when directing one's own musculature is a core feature of human experience. We show that we can robustly manipulate the sense of agency over electrically actuated muscle movements, and we investigate the time course of neural activity that predicts the sense of agency over these actuated movements. We find evidence of two distinct neural processes: a transient sequence of patterns that begins in the early sensorineural response to muscle stimulation and a later, sustained signature of agency. These results shed light on the neural mechanisms by which we experience our movements as volitional.

Proprioception: A New Era Set in Motion by Emerging Genetic and Bionic Strategies?

The generation of an internal body model and its continuous update is essential in sensorimotor control. Although known to rely on proprioceptive sensory feedback, the underlying mechanism that transforms this sensory feedback into a dynamic body percept remains poorly understood. However, advances in the development of genetic tools for proprioceptive circuit elements, including the sensory receptors, are beginning to offer new and unprecedented leverage to dissect the central pathways responsible for proprioceptive encoding. Simultaneously, new data derived through emerging bionic neural machine-interface technologies reveal clues regarding the relative importance of kinesthetic sensory feedback and insights into the functional proprioceptive substrates that underlie natural motor behaviors.

Auditory Feedback for Enhanced Sense of Agency in Shared Control

There is a growing need for robots that can be remotely controlled to perform tasks of one's own choice. However, the SoA (Sense of Agency: the sense of recognizing that the motion of an observed object is caused by oneself) is reduced because the subject of the robot motion is identified as external due to shared control. To address this issue, we aimed to suppress the decline in SoA by presenting auditory feedback that aims to blur the distinction between self and others. We performed the tracking task in a virtual environment under four different auditory feedback conditions, with varying levels of automation to manipulate the virtual robot gripper. Experimental results showed that the proposed auditory feedback suppressed the decrease in the SoA at a medium level of automation. It is suggested that our proposed auditory feedback could blur the distinction between self and others, and that the operator attributes the subject of the motion of the manipulated object to himself.

The salience network is activated during self-recognition from both first-person and third-person perspectives

We usually observe ourselves from two perspectives. One is the first-person perspective, which we perceive directly with our own eyes, and the other is the third-person perspective, which we observe ourselves in a mirror or a picture. However, whether the self-recognition associated with these two perspectives has a common or separate neural basis remains unclear. To address this, we used functional magnetic resonance imaging to examine brain activity while participants viewed pretaped video clips of themselves and others engaged in meal preparation taken from first-person and third-person perspectives. We found that the first-person behavioral videos of the participants and others induced greater activation in the premotor-intraparietal region. In contrast, the third-person behavioral videos induced greater activation in the default mode network compared with the first-person videos. Regardless of the perspective, the videos of the participants induced greater activation in the salience network than the videos of others. On the other hand, the videos of others induced greater activation in the precuneus and lingual gyrus than the videos of the participants. These results suggest that the salience network is commonly involved in self-recognition from both perspectives, even though the brain regions involved in action observation for the two perspectives are distinct.

Clinical MRI morphological analysis of functional seizures compared to seizure-naïve and psychiatric controls

PURPOSE

Functional seizures (FS), also known as psychogenic nonepileptic seizures (PNES), are physical manifestations of acute or chronic psychological distress. Functional and structural neuroimaging have identified objective signs of this disorder. We evaluated whether magnetic resonance imaging (MRI) morphometry differed between patients with FS and clinically relevant comparison populations.

METHODS

Quality-screened clinical-grade MRIs were acquired from 666 patients from 2006 to 2020. Morphometric features were quantified with FreeSurfer v6. Mixed-effects linear regression compared the volume, thickness, and surface area within 201 regions-of-interest for 90 patients with FS, compared to seizure-naïve patients with depression (n = 243), anxiety (n = 68), and obsessive-compulsive disorder (OCD, n = 41), respectively, and to other seizure-naïve controls with similar quality MRIs, accounting for the influence of multiple confounds including depression and anxiety based on chart review. These comparison populations were obtained through review of clinical records plus research studies obtained on similar scanners.

RESULTS

After Bonferroni-Holm correction, patients with FS compared with seizure-naïve controls exhibited thinner bilateral superior temporal cortex (left 0.053 mm, p = 0.014; right 0.071 mm, p = 0.00006), thicker left lateral occipital cortex (0.052 mm, p = 0.0035), and greater left cerebellar white-matter volume (1085 mm³, p = 0.0065). These findings were not accounted for by lower MRI quality in patients with FS.

CONCLUSIONS

These results reinforce prior indications of structural neuroimaging correlates of FS and, in particular, distinguish brain morphology in FS from that in depression, anxiety, and OCD. Future work may entail comparisons with other psychiatric disorders including bipolar and schizophrenia, as well as exploration of brain structural heterogeneity within FS.

Multisensory and Sensorimotor Integration in the Embodied Self: Relationship between Self-Body Recognition and the Mirror Neuron System

The embodied self is rooted in the self-body in the "here and now". The senses of self-ownership and self-agency have been proposed as the basis of the sense of embodied self, and many experimental studies have been conducted on this subject. This review summarizes the experimental research on the embodied self that has been conducted over the past 20 years, mainly from the perspective of multisensory integration and sensorimotor integration regarding the self-body. Furthermore, the phenomenon of back projection, in which changes in an external object (e.g., a rubber hand) with which one has a sense of ownership have an inverse influence on the sensation and movement of one's own body, is discussed. This postulates that the self-body illusion is not merely an illusion caused by multisensory and/or sensorimotor integration, but is the incorporation of an external object into the self-body representation in the brain. As an extension of this fact, we will also review research on the mirror neuron system, which is considered to be the neural basis of recognition of others, and discuss how the neural basis of self-body recognition and the mirror neuron system can be regarded as essentially the same.

Impaired Relationship between Sense of Agency and Prediction Error Due to Post-Stroke Sensorimotor Deficits

Sense of agency refers to the experience of controlling one’s actions. Studies on healthy people indicated that their self-other attribution can be realized based on prediction error which is an inconsistency between the internal prediction and sensory feedback of the movements. However, studies on patients with post-stroke sensorimotor deficits hypothesized that their self-other attribution can be based on different attribution strategies. This preliminary study examined this hypothesis by investigating whether post-stroke sensorimotor deficits can diminish the correlation between prediction errors and self-other judgments. Participants performed sinusoidal movements with visual feedback and judged if it represented their or another’s movements (i.e., self-other judgment). The results indicated that the patient who had worse upper limb sensorimotor deficits and lesser paretic upper limb activity compared with the other patient made more misattributions and showed a lower correlation between prediction errors and self-other judgments. This finding suggests that post-stroke sensorimotor deficits can impair the relationship between prediction error and self-other attribution, supporting the hypothesis that patients with such deficits can have altered strategies for the registration of agency.

Sense of agency as synecdoche: Multiple neurobiological mechanisms may underlie the phenomenon summarized as sense of agency

Functional magnetic resonance imaging (fMRI) studies on the sense of agency (SoA) have yielded heterogeneous findings identifying regional brain activity during tasks that probed SoA. In this review, we argue that the reason behind this between-study heterogeneity is a "synecdochic" way the field conceptualizes and studies SoA. Typically, a single feature is experimentally manipulated and then this is interpreted as covering all aspects of SoA. The purpose of this paper is to give an overview of the fMRI studies of SoA and attempt to provide meaningful categories whereby the heterogeneous findings may be classified. This classification is based on a separation of the experimental paradigms (Feedback Manipulations of ongoing movements, Action-Effect, and Sensory Attenuation) and type of report employed (implicit, explicit reports of graded or dichotic nature, and whether these concern self-other distinctions or sense of control). We only find that Feedback Manipulation and Action-Effect share common activation in supplementary motor area, insula and cerebellum in positive SoA and inferior frontal gyrus in the negative SoA, but observe large networks related to SoA only in Feedback Manipulation studies. To illustrate the advantages of this approach, we discuss the findings from an fMRI study which we conducted, within this framework.

Cortical activation during cooperative joint actions and competition in children with and without an autism spectrum condition (ASC): an fNIRS study

Children with an Autism Spectrum Condition (ASC) have social communication and perceptuomotor difficulties that affect their ability to engage in dyadic play. In this study, we compared spatio-temporal errors and fNIRS-related cortical activation between children with and without an ASC during a Lincoln Log dyadic game requiring them to play leader or follower roles, move in synchrony or while taking turns, and move cooperatively or competitively with an adult partner. Children with an ASC had greater motor, planning, and spatial errors and took longer to complete the building tasks compared to typically developing (TD) children. Children with an ASC had lower superior temporal sulcus (STS) activation during Turn-take and Compete, and greater Inferior Parietal Lobe (IPL) activation during Lead and Turn-take compared to TD children. As dyadic play demands increased, TD children showed greater STS activation during Turn-take (vs. Synchrony) and Compete (vs. Cooperate) whereas children with an ASC showed greater IPL activation during Lead and Compete (vs. Cooperate). Our findings suggest that children with an ASC rely on self-generated action plans (i.e., increased IPL activation) more than relying on their partner’s action cues (i.e., reduced STS activation) when engaging in dyadic play including joint actions and competition.

The Role of the Interaction between the Inferior Parietal Lobule and Superior Temporal Gyrus in the Multisensory Go/No-go Task

Information from multiple sensory modalities interacts. Using functional magnetic resonance imaging (fMRI), we aimed to identify the neural structures correlated with how cooccurring sound modulates the visual motor response execution. The reaction time (RT) to audiovisual stimuli was significantly faster than the RT to visual stimuli. Signal detection analyses showed no significant difference in the perceptual sensitivity (d') between audiovisual and visual stimuli, while the response criteria (β or c) of the audiovisual stimuli was decreased compared to the visual stimuli. The functional connectivity between the left inferior parietal lobule (IPL) and bilateral superior temporal gyrus (STG) was enhanced in Go processing compared with No-go processing of audiovisual stimuli. Furthermore, the left precentral gyrus (PreCG) showed enhanced functional connectivity with the bilateral STG and other areas of the ventral stream in Go processing compared with No-go processing of audiovisual stimuli. These results revealed that the neuronal network correlated with modulations of the motor response execution after the presentation of both visual stimuli along with cooccurring sound in a multisensory Go/Nogo task, including the left IPL, left PreCG, bilateral STG and some areas of the ventral stream. The role of the interaction between the IPL and STG in transforming audiovisual information into motor behavior is discussed. The current study provides a new perspective for exploring potential brain mechanisms underlying how humans execute appropriate behaviors on the basis of multisensory information.

Neuroimaging Examination of Driving Mode Switching Corresponding to Changes in the Driving Environment

Car driving is supported by perceptual, cognitive, and motor skills trained through continuous daily practice. One of the skills that characterize experienced drivers is to detect changes in the driving environment and then flexibly switch their driving modes in response to the changes. Previous functional neuroimaging studies on motor control investigated the mechanisms underlying behaviors adaptive to changes in control properties or parameters of experimental devices such as a computer mouse or a joystick. The switching of multiple internal models mainly engages adaptive behaviors and underlies the interplay between the cerebellum and frontoparietal network (FPN) regions as the neural process. However, it remains unclear whether the neural mechanisms identified in previous motor control studies also underlie practical driving behaviors. In the current study, we measure functional magnetic resonance imaging (fMRI) activities while participants control a realistic driving simulator inside the MRI scanner. Here, the accelerator sensitivity of a virtual car is abruptly changed, requiring participants to respond to this change flexibly to maintain stable driving. We first compare brain activities before and after the sensitivity change. As a result, sensorimotor areas, including the left cerebellum, increase their activities after the sensitivity change. Moreover, after the change, activity significantly increases in the inferior parietal lobe (IPL) and dorsolateral prefrontal cortex (DLPFC), parts of the FPN regions. By contrast, the posterior cingulate cortex, a part of the default mode network, deactivates after the sensitivity change. Our results suggest that the neural bases found in previous experimental studies can serve as the foundation of adaptive driving behaviors. At the same time, this study also highlights the unique contribution of non-motor regions to addressing the high cognitive demands of driving.

Body-Specific Attention to the Hands and Feet in Healthy Adults

To execute the intended movement, the brain directs attention, called body-specific attention, to the body to obtain information useful for movement. Body-specific attention to the hands has been examined but not to the feet. We aimed to confirm the existence of body-specific attention to the hands and feet, and examine its relation to motor and sensory functions from a behavioral perspective. The study included two groups of 27 right-handed and right-footed healthy adults, respectively. Visual detection tasks were used to measure body-specific attention. We measured reaction times to visual stimuli on or off the self-body and calculated the index of body-specific attention score to subtract the reaction time on self-body from that off one. Participants were classified into low and high attention groups based on each left and right body-specific attention index. For motor functions, Experiment 1 comprised handgrip strength and ball-rotation tasks for the hands, and Experiment 2 comprised toe grip strength involved in postural control for the feet. For sensory functions, the tactile thresholds of the hands and feet were measured. The results showed that, in both hands, the reaction time to visual stimuli on the hand was significantly lesser than that offhand. In the foot, this facilitation effect was observed in the right foot but not the left, which showed the correlation between body-specific attention and the normalized toe gripping force, suggesting that body-specific attention affected postural control. In the hand, the number of rotations of the ball was higher in the high than in the low attention group, regardless of the elaboration exercise difficulty or the left or right hand. However, this relation was not observed in the handgripping task. Thus, body-specific attention to the hand is an important component of elaborate movements. The tactile threshold was higher in the high than in the low attention group, regardless of the side in hand and foot. The results suggested that more body-specific attention is directed to the limbs with lower tactile abilities, supporting the sensory information reaching the brain. Therefore, we suggested that body-specific attention regulates the sensory information to help motor control.

The role of motor memory dynamics in structuring bodily self-consciousness

Bodily self-consciousness has been considered a sensorimotor root of self-consciousness. If this is the case, how does sensorimotor memory, which is important for the prediction of sensory consequences of volitional actions, influence awareness of bodily self-consciousness? This question is essential for understanding the effective acquisition and recovery of self-consciousness following its impairment, but it has remained unexamined. Here, we investigated how body ownership and agency recovered following body schema distortion in a virtual reality environment along with two kinds of motor memories: memories that were rapidly updated and memories that were gradually updated. We found that, although agency and body ownership recovered in parallel, the recovery of body ownership was predicted by fast memories and that of agency was predicted by slow memories. Thus, the bodily self was represented in multiple motor memories with different dynamics. This finding demystifies the controversy about the causal relationship between body ownership and agency.

Transparency in Human-Machine Mutual Action

Recent advances in human-computer integration (HInt) have focused on the development of human-machine systems, where both human and machine autonomously act upon each other. However, a key challenge in designing such systems is augmenting the user’s physical abilities while maintaining their sense of self-attribution. This challenge is particularly prevalent when both human and machine are capable of acting upon each other, thereby creating a human-machine mutual action (HMMA) system. To address this challenge, we present a design framework that is based on the concept of transparency. We define transparency in HInt as the degree to which users can self-attribute an experience when machines intervene in the users’ action. Using this framework, we form a set of design guidelines and an approach for designing HMMA systems. By using transparency as our focus, we aim to provide a design approach for not only achieving human-machine fusion into a single agent, but also controlling the degrees of fusion at will. This study also highlights the effectiveness of our design approach through an analysis of existing studies that developed HMMA systems. Further development of our design approach is discussed, and future prospects for HInt and HMMA system designs are presented.

The Cognitive, Ecological, and Developmental Origins of Self-Disturbance in Borderline Personality Disorder

Self-disturbance is recognized as a key symptom of Borderline Personality Disorder (BPD). Although it is the source of significant distress and significant costs to society, it is still poorly specified. In addition, current research and models on the etiology of BPD do not provide sufficient evidence or predictions about who is at risk of developing BPD and self-disturbance, and why. The aim of this review is to lay the foundations of a new model inspired by recent developments at the intersection of social cognition, behavioral ecology, and developmental biology. We argue that the sense of agency is an important dimension to consider when characterizing self-disturbances in BPD. Second, we address the poorly characterized relation between self-disturbances and adverse life conditions encountered early in life. We highlight the potential relevance of Life-History Theory-a major framework in evolutionary developmental biology-to make sense of this association. We put forward the idea that the effect of early life adversity on BPD symptomatology depends on the way individuals trade their limited resources between competing biological functions during development.

Deceleration Assistance Mitigated the Trade-off Between Sense of Agency and Driving Performance

Driving assistance technology has gained traction in recent years and is becoming more widely used in vehicles. However, drivers usually experience a reduced sense of agency when driving assistance is active even though automated assistance improves driving performance by reducing human error and ensuring quick reactions. The present study examined whether driving assistance can maintain human sense of agency during early deceleration in the face of collision risk, compared with manual deceleration. In the experimental task, participants decelerate their vehicle in a driving simulator to avoid collision with a vehicle that suddenly cut in front of them and decelerated. In the assisted condition, the system performed deceleration 100 ms after the cut-in. Participants were instructed to decelerate their vehicle and follow the vehicle that cut-in. This design ensured that the deceleration assistance applied a similar control to the vehicle as the drivers intended to, only faster and smoother. Participants rated their sense of agency and their driving performance. The results showed that drivers maintained their sense of agency and improved driving performance under driving assistance. The findings provided insights into designing driving assistance that can maintain drivers' sense of agency while improving future driving performance. It is important to establish a mode of joint-control in which the system shares the intention of human drivers and provides improved execution of control.

Neural Correlates of Self-other Distinction in Patients with Schizophrenia Spectrum Disorders: The Roles of Agency and Hand Identity

Schizophrenia spectrum disorders (SSD) are characterized by disturbed self-other distinction. While previous studies associate abnormalities in the sense of agency (ie, the feeling that an action and the resulting sensory consequences are produced by oneself) with disturbed processing in the angular gyrus, passive movement conditions to isolate contributions of motor predictions are lacking. Furthermore, the role of body identity (ie, visual features determining whether a seen body part belongs to oneself) in self-other distinction is unclear. In the current study, fMRI was used to assess the roles of agency and hand identity in self-other distinction. Patients with SSD and healthy controls (HC) performed active and passive hand movements (agency manipulation) while seeing their own or someone else's hand moving in accordance with their action (hand identity manipulation). Variable delays (0-417 ms) between movement and feedback had to be detected. Our results showed overall lower delay detection performances during active than passive conditions; however, these differences were reduced in patients when the own hand was displayed. On a neural level, we found that in HC, activation in the right angular gyrus was modulated by agency and hand identity. In contrast, agency and hand identity revealed no overlapping activation in patients, due to reduced effects of agency. Importantly, HC and SSD patients shared similar effects of hand identity in the angular gyrus. Our results suggest that disturbances of self-other distinction in SSD are particularly driven by agency, while self-other distinction based on hand identity might be spared.

Categorical Perception of Control

The self is a distinct entity from the rest of the world, and actions and sensory feedback are our channels of interaction with the external world. This study examined how the sense of control influences people's perception of sensorimotor input under the framework of categorical perception. Twenty human participants (18 males, two females) took part in both experiments. Experiment 1 showed that the sensitivity (d') of detecting a 20% change in control from no change was higher when the changes occurred at the control-category boundary than within each category. Experiment 2 showed that the control categories greatly affected early attention allocation, even when the judgment of control was unnecessary to the task. Taken together, these results showed that our perceptual and cognitive systems are highly sensitive to small changes in control that build up to a determinant change in the control category within a relatively narrow boundary zone between categories, compared with a continuous, gradual physical change in control.