Robotic interfaces for cognitive psychology and embodiment research: A research roadmap

A Multimodal Bracelet to Acquire Muscular Activity and Gyroscopic Data to Study Sensor Fusion for Intent Detection

Researchers have attempted to control robotic hands and prostheses through biosignals but could not match the human hand. Surface electromyography records electrical muscle activity using non-invasive electrodes and has been the primary method in most studies. While surface electromyography-based hand motion decoding shows promise, it has not yet met the requirements for reliable use. Combining different sensing modalities has been shown to improve hand gesture classification accuracy. This work introduces a multimodal bracelet that integrates a 24-channel force myography system with six commercial surface electromyography sensors, each containing a six-axis inertial measurement unit. The device's functionality was tested by acquiring muscular activity with the proposed device from five participants performing five different gestures in a random order. A random forest model was then used to classify the performed gestures from the acquired signal. The results confirmed the device's functionality, making it suitable to study sensor fusion for intent detection in future studies. The results showed that combining all modalities yielded the highest classification accuracies across all participants, reaching 92.3±2.6% on average, effectively reducing misclassifications by 37% and 22% compared to using surface electromyography and force myography individually as input signals, respectively. This demonstrates the potential benefits of sensor fusion for more robust and accurate hand gesture classification and paves the way for advanced control of robotic and prosthetic hands.

Intuitive and versatile bionic legs: a perspective on volitional control

Active lower limb prostheses show large potential to offer energetic, balance, and versatility improvements to users when compared to passive and semi-active devices. Still, their control remains a major development challenge, with many different approaches existing. This perspective aims at illustrating a future leg prosthesis control approach to improve the everyday life of prosthesis users, while providing a research road map for getting there. Reviewing research on the needs and challenges faced by prosthesis users, we argue for the development of versatile control architectures for lower limb prosthetic devices that grant the wearer full volitional control at all times. To this end, existing control approaches for active lower limb prostheses are divided based on their consideration of volitional user input. The presented methods are discussed in regard to their suitability for universal everyday control involving user volition. Novel combinations of established methods are proposed. This involves the combination of feed-forward motor control signals with simulated feedback loops in prosthesis control, as well as online optimization techniques to individualize the system parameters. To provide more context, developments related to volitional control design are touched on.

Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses

INTRODUCTION

Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users' own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users' experience regarding embodiment and their interaction with different functional levels.

METHODS

To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey.

RESULTS

Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented.

CONCLUSIONS

Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Human-in-the-Loop Optimization of Wearable Robotic Devices to Improve Human-Robot Interaction: A Systematic Review

This article presents a systematic review on wearable robotic devices that use human-in-the-loop optimization (HILO) strategies to improve human-robot interaction. A total of 46 HILO studies were identified and divided into upper and lower limb robotic devices. The main aspects from HILO were identified, reviewed, and classified in four areas: 1) human-machine systems; 2) optimization methods; 3) control strategies; and 4) experimental protocols. A variety of objective functions (physiological, biomechanical, and subjective), optimization strategies, and optimized control parameters configurations used in different control strategies are presented and analyzed. An overview of experimental protocols is provided, including metrics, tasks, and conditions tested. Moreover, the relevance given to training or adaptation periods was explored. We outline an HILO framework that includes current wearable robots, optimization strategies, objective functions, control strategies, and experimental protocols. We conclude by highlighting current research gaps and defining future directions to improve the development of advanced HILO strategies in upper and lower limb wearable robots.

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

Simultaneous assessment and training of an upper-limb amputee using incremental machine-learning-based myocontrol: a single-case experimental design

BACKGROUND

Machine-learning-based myocontrol of prosthetic devices suffers from a high rate of abandonment due to dissatisfaction with the training procedure and with the reliability of day-to-day control. Incremental myocontrol is a promising approach as it allows on-demand updating of the system, thus enforcing continuous interaction with the user. Nevertheless, a long-term study assessing the efficacy of incremental myocontrol is still missing, partially due to the lack of an adequate tool to do so. In this work we close this gap and report about a person with upper-limb absence who learned to control a dexterous hand prosthesis using incremental myocontrol through a novel functional assessment protocol called SATMC (Simultaneous Assessment and Training of Myoelectric Control).

METHODS

The participant was fitted with a custom-made prosthetic setup with a controller based on Ridge Regression with Random Fourier Features (RR-RFF), a non-linear, incremental machine learning method, used to build and progressively update the myocontrol system. During a 13-month user study, the participant performed increasingly complex daily-living tasks, requiring fine bimanual coordination and manipulation with a multi-fingered hand prosthesis, in a realistic laboratory setup. The SATMC was used both to compose the tasks and continually assess the participant's progress. Patient satisfaction was measured using Visual Analog Scales.

RESULTS

Over the course of the study, the participant progressively improved his performance both objectively, e.g., the time required to complete each task became shorter, and subjectively, meaning that his satisfaction improved. The SATMC actively supported the improvement of the participant by progressively increasing the difficulty of the tasks in a structured way. In combination with the incremental RR-RFF allowing for small adjustments when required, the participant was capable of reliably using four actions of the prosthetic hand to perform all required tasks at the end of the study.

CONCLUSIONS

Incremental myocontrol enabled an upper-limb amputee to reliably control a dexterous hand prosthesis while providing a subjectively satisfactory experience. The SATMC can be an effective tool to this aim.

Embodiment for Robotic Lower-Limb Exoskeletons: A Narrative Review

Research on embodiment of objects external to the human body has revealed important information about how the human nervous system interacts with robotic lower limb exoskeletons. Typical robotic exoskeleton control approaches view the controllers as an external agent intending to move in coordination with the human. However, principles of embodiment suggest that the exoskeleton controller should ideally coordinate with the human such that the nervous system can adequately model the input-output dynamics of the exoskeleton controller. Measuring embodiment of exoskeletons should be a necessary step in the exoskeleton development and prototyping process. Researchers need to establish high fidelity quantitative measures of embodiment, rather than relying on current qualitative survey measures. Mobile brain imaging techniques, such as high-density electroencephalography, is likely to provide a deeper understanding of embodiment during human-machine interactions and advance exoskeleton research and development. In this review we show why future exoskeleton research should include quantitative measures of embodiment as a metric of success.

Neural signatures of visuo-motor integration during human-robot interactions

Visuo-motor integration shapes our daily experience and underpins the sense of feeling in control over our actions. The last decade has seen a surge in robotically and virtually mediated interactions, whereby bodily actions ultimately result in an artificial movement. But despite the growing number of applications, the neurophysiological correlates of visuo-motor processing during human-machine interactions under dynamic conditions remain scarce. Here we address this issue by employing a bimanual robotic interface able to track voluntary hands movement, rendered in real-time into the motion of two virtual hands. We experimentally manipulated the visual feedback in the virtual reality with spatial and temporal conflicts and investigated their impact on (1) visuo-motor integration and (2) the subjective experience of being the author of one's action (i.e., sense of agency). Using somatosensory evoked responses measured with electroencephalography, we investigated neural differences occurring when the integration between motor commands and visual feedback is disrupted. Our results show that the right posterior parietal cortex encodes for differences between congruent and spatially-incongruent interactions. The experimental manipulations also induced a decrease in the sense of agency over the robotically-mediated actions. These findings offer solid neurophysiological grounds that can be used in the future to monitor integration mechanisms during movements and ultimately enhance subjective experience during human-machine interactions.

Auditory Feedback for Enhanced Sense of Agency in Shared Control

There is a growing need for robots that can be remotely controlled to perform tasks of one's own choice. However, the SoA (Sense of Agency: the sense of recognizing that the motion of an observed object is caused by oneself) is reduced because the subject of the robot motion is identified as external due to shared control. To address this issue, we aimed to suppress the decline in SoA by presenting auditory feedback that aims to blur the distinction between self and others. We performed the tracking task in a virtual environment under four different auditory feedback conditions, with varying levels of automation to manipulate the virtual robot gripper. Experimental results showed that the proposed auditory feedback suppressed the decrease in the SoA at a medium level of automation. It is suggested that our proposed auditory feedback could blur the distinction between self and others, and that the operator attributes the subject of the motion of the manipulated object to himself.

The sense of agency in emerging technologies for human–computer integration: A review

Human–computer integration is an emerging area in which the boundary between humans and technology is blurred as users and computers work collaboratively and share agency to execute tasks. The sense of agency (SoA) is an experience that arises by a combination of a voluntary motor action and sensory evidence whether the corresponding body movements have somehow influenced the course of external events. The SoA is not only a key part of our experiences in daily life but also in our interaction with technology as it gives us the feeling of “I did that” as opposed to “the system did that,” thus supporting a feeling of being in control. This feeling becomes critical with human–computer integration, wherein emerging technology directly influences people’s body, their actions, and the resulting outcomes. In this review, we analyse and classify current integration technologies based on what we currently know about agency in the literature, and propose a distinction between body augmentation, action augmentation, and outcome augmentation. For each category, we describe agency considerations and markers of differentiation that illustrate a relationship between assistance level (low, high), agency delegation (human, technology), and integration type (fusion, symbiosis). We conclude with a reflection on the opportunities and challenges of integrating humans with computers, and finalise with an expanded definition of human–computer integration including agency aspects which we consider to be particularly relevant. The aim this review is to provide researchers and practitioners with guidelines to situate their work within the integration research agenda and consider the implications of any technologies on SoA, and thus overall user experience when designing future technology.

A Novel Deep Framework for English Communication Based on Educational Psychology Perspective

The impact of verbal reading practices on learning is examined from the perspective of educational psychology, using the motivation theory and the schema theory. This research intends to enhance learner's English communication abilities in response to the needs for national economic growth and scientific and technological development. To motivate students to improve their English, the research may address the issue of inadequate opportunities by adding an artificial intelligence (AI) conversation mechanism to the students speaking English exercise. First, cognitive psychology is analyzed in detail, and a model based on cognitive psychology is implemented to solve the problems existing in student's English communication. In addition, various measures are presented and used to increase student's oral English communication abilities. We used sixty students from North China University of Water Resources and Electric Power are separated into two classes: Class A and Class B. The experimental group is called Class A, while the control group is called Class B. Following a comparison of the outcomes obtained before and after training. The experimental group's reading comprehension, responding to questions, situational conversation, and subject description scores rose by 13.33, 15.19, 17.39, and 28.3 %, respectively. The overall average score of the class climbed by 17.75 %, whereas the scores of pupils in Class B improved just an undersized. The results reveal that following the vocalized reading exercise, the student's English grades, self-efficacy, and topic knowledge increased considerably in the experimental group. Moreover, the proposed model, employs computer simulation in the English communication teaching system and AI, which can aid in the creation of an interactive learning environment for students to improve their spoken English and English communication abilities.

Improving Mobile Device Security by Embodying and Co-adapting a Behavioral Biometric Interface

At present, interfaces between users and smart devices such as smart phones rely primarily on passwords. This has allowed for the intrusion and perturbation of the interface between the user and the device and has compromised security. Recently, Frank et al. have suggested that security could be improved by having an interface with biometric features of finger swiping. This approach has been termed touchalytics, in maintaining cybersecurity. The number of features of finger swiping have been large (32) and have been made available as a public database, which we utilize in our study. However, it has not been shown which of these features uniquely identify a particular user. In this paper, we study whether a subset of features that embody human cognitive motor features can be used to identify a particular user. We consider how the security might be made more efficient embodying Principal Component Analysis (PCA) into the interface, which has the potential of reducing the features utilized in the identification of intruders. We compare the accuracy and performance of the reduced feature space to that of having all the features. Embodying a robust continuous authentication system will give users an extra layer of security and an increased sense of peace of mind if their devices are lost or stolen. Consequently, such improvements may prevent access to sensitive information and thus will save businesses money. Consequently, such improvements may prevent access to sensitive information and thus will save businesses money. If continuous authentication models become successful and easily implementable, embodiment and co-adaptation of user authentication would inhibit the growing problem of mobile device theft.

Exploring the Embodiment of a Virtual Hand in a Spatially Augmented Respiratory Biofeedback Setting

Enhancing the embodiment of artificial limbs-the individuals' feeling that a virtual or robotic limb is integrated in their own body scheme-is an impactful strategy for improving prosthetic technology acceptance and human-machine interaction. Most studies so far focused on visuo-tactile strategies to empower the embodiment processes. However, novel approaches could emerge from self-regulation techniques able to change the psychophysiological conditions of an individual. Accordingly, this pilot study investigates the effects of a self-regulated breathing exercise on the processes of body ownership underlying the embodiment of a virtual right hand within a Spatially Augmented Respiratory Biofeedback (SARB) setting. This investigation also aims at evaluating the feasibility of the breathing exercise enabled by a low-cost SARB implementation designed for upcoming remote studies (a need emerged during the COVID-19 pandemic). Twenty-two subjects without impairments, and two transradial prosthesis users for a preparatory test, were asked (in each condition of a within-group design) to maintain a normal (about 14 breaths/min) or slow (about 6 breaths/min) respiratory rate to keep a static virtual right hand "visible" on a screen. Meanwhile, a computer-generated sphere moved from left to right toward the virtual hand during each trial (1 min) of 16. If the participant's breathing rate was within the target (slow or normal) range, a visuo-tactile event was triggered by the sphere passing under the virtual hand (the subjects observed it shaking while they perceived a vibratory feedback generated by a smartphone). Our results-mainly based on questionnaire scores and proprioceptive drift-highlight that the slow breathing condition induced higher embodiment than the normal one. This preliminary study reveals the feasibility and potential of a novel psychophysiological training strategy to enhance the embodiment of artificial limbs. Future studies are needed to further investigate mechanisms, efficacy and generalizability of the SARB techniques in training a bionic limb embodiment.

Feel-Good Requirements: Neurophysiological and Psychological Design Criteria of Affective Touch for (Assistive) Robots

Previous research has shown the value of the sense of embodiment, i.e., being able to integrate objects into one's bodily self-representation, and its connection to (assistive) robotics. Especially, tactile interfaces seem essential to integrate assistive robots into one's body model. Beyond functional feedback, such as tactile force sensing, the human sense of touch comprises specialized nerves for affective signals, which transmit positive sensations during slow and low-force tactile stimulations. Since these signals are extremely relevant for body experience as well as social and emotional contacts but scarcely considered in recent assistive devices, this review provides a requirement analysis to consider affective touch in engineering design. By analyzing quantitative and qualitative information from engineering, cognitive psychology, and neuroscienctific research, requirements are gathered and structured. The resulting requirements comprise technical data such as desired motion or force/torque patterns and an evaluation of potential stimulation modalities as well as their relations to overall user experience, e.g., pleasantness and realism of the sensations. This review systematically considers the very specific characteristics of affective touch and the corresponding parts of the neural system to define design goals and criteria. Based on the analysis, design recommendations for interfaces mediating affective touch are derived. This includes a consideration of biological principles and human perception thresholds which are complemented by an analysis of technical possibilities. Finally, we outline which psychological factors can be satisfied by the mediation of affective touch to increase acceptance of assistive devices and outline demands for further research and development.

Interaction in Assistive Robotics: A Radical Constructivist Design Framework

Despite decades of research, muscle-based control of assistive devices (myocontrol) is still unreliable; for instance upper-limb prostheses, each year more and more dexterous and human-like, still provide hardly enough functionality to justify their cost and the effort required to use them. In order to try and close this gap, we propose to shift the goal of myocontrol from guessing intended movements to creating new circular reactions in the constructivist sense defined by Piaget. To this aim, the myocontrol system must be able to acquire new knowledge and forget past one, and knowledge acquisition/forgetting must happen on demand, requested either by the user or by the system itself. We propose a unifying framework based upon Radical Constructivism for the design of such a myocontrol system, including its user interface and user-device interaction strategy.

Bringing Together Robotics, Neuroscience, and Psychology: Lessons Learned From an Interdisciplinary Project

The diversified methodology and expertise of interdisciplinary research teams provide the opportunity to overcome the limited perspectives of individual disciplines. This is particularly true at the interface of Robotics, Neuroscience, and Psychology as the three fields have quite different perspectives and approaches to offer. Nonetheless, aligning backgrounds and interdisciplinary expectations can present challenges due to varied research cultures and practices. Overcoming these challenges stands at the beginning of each productive collaboration and thus is a mandatory step in cognitive neurorobotics. In this article, we share eight lessons that we learned from our ongoing interdisciplinary project on human-robot and robot-robot interaction in social settings. These lessons provide practical advice for scientists initiating interdisciplinary research endeavors. Our advice can help to avoid early problems and deal with differences between research fields, prepare for and anticipate challenges, align project expectations, and speed up research progress, thus promoting effective interdisciplinary research across Robotics, Neuroscience, and Psychology.

Decision-Making in the Human-Machine Interface

If our choices make us who we are, then what does that mean when these choices are made in the human-machine interface? Developing a clear understanding of how human decision making is influenced by automated systems in the environment is critical because, as human-machine interfaces and assistive robotics become even more ubiquitous in everyday life, many daily decisions will be an emergent result of the interactions between the human and the machine – not stemming solely from the human. For example, choices can be influenced by the relative locations and motor costs of the response options, as well as by the timing of the response prompts. In drift diffusion model simulations of response-prompt timing manipulations, we find that it is only relatively equibiased choices that will be successfully influenced by this kind of perturbation. However, with drift diffusion model simulations of motor cost manipulations, we find that even relatively biased choices can still show some influence of the perturbation. We report the results of a two-alternative forced-choice experiment with a computer mouse modified to have a subtle velocity bias in a pre-determined direction for each trial, inducing an increased motor cost to move the cursor away from the pre-designated target direction. With queries that have each been normed in advance to be equibiased in people’s preferences, the participant will often begin their mouse movement before their cognitive choice has been finalized, and the directional bias in the mouse velocity exerts a small but significant influence on their final choice. With queries that are not equibiased, a similar influence is observed. By exploring the synergies that are developed between humans and machines and tracking their temporal dynamics, this work aims to provide insight into our evolving decisions.

Benchmarking Wearable Robots: Challenges and Recommendations From Functional, User Experience, and Methodological Perspectives

Wearable robots (WRs) are increasingly moving out of the labs toward real-world applications. In order for WRs to be effectively and widely adopted by end-users, a common benchmarking framework needs to be established. In this article, we outline the perspectives that in our opinion are the main determinants of this endeavor, and exemplify the complex landscape into three areas. The first perspective is related to quantifying the technical performance of the device and the physical impact of the device on the user. The second one refers to the understanding of the user's perceptual, emotional, and cognitive experience of (and with) the technology. The third one proposes a strategic path for a global benchmarking methodology, composed by reproducible experimental procedures representing real-life conditions. We hope that this paper can enable developers, researchers, clinicians and end-users to efficiently identify the most promising directions for validating their technology and drive future research efforts in the short and medium term.

Exploring the Effect of Cooperation in Reducing Implicit Racial Bias and Its Relationship With Dispositional Empathy and Political Attitudes

Previous research using immersive virtual reality (VR) has shown that after a short period of embodiment by White people in a Black virtual body, their implicit racial bias against Black people diminishes. Here we tested the effects of some socio-cognitive variables that could contribute to enhancing or reducing the implicit racial bias. The first aim of the study was to assess the beneficial effects of cooperation within a VR scenario, the second aim was to provide preliminary testing of the hypothesis that empathy and political attitudes could contribute to implicit bias about race, while the third aim was to explore the relationship between political attitudes and empathy. We had (Caucasian) participants embodied in a Black virtual body and engaged either in a cooperative (Coop group) or in a non-cooperative (Neutral group) activity with a confederate experimenter embodying another Black avatar. Before and after VR, we measured participants’ implicit racial bias by means of Implicit Association Test (IAT) and their perceived closeness toward the confederate experimenter. Before VR we also assessed participants’ political attitudes and empathy traits. Results revealed that, as compared to the Neutral group, the Coop group showed lower IAT scores after the social interaction. Interestingly, in the Neutral but not the Coop group the perceived closeness toward the confederate experimenter was associated with the initial racial bias: the more the participants reduced their distance, the more they reduced their IAT score. Moreover, reported traits of empathy and political attitudes significantly explained the variance observed in the initial implicit bias, with perspective-taking, empathic concern, and personal distress being significant predictors of the IAT scores. Finally, there was a relationship between political attitudes and empathy: the more participants considered themselves as left-wing voters, the higher their perspective-taking and empathic concern scores. We discuss these findings within the neuroscientific and social cognition field and encourage scholars from different domains to further explore whether and under which conditions a given manipulation for reducing racial bias could be efficiently transposed in VR.

The Interaction between Internet, Sustainable Development, and Emergence of Society 5.0

(1) Background: The importance of this article is to analyze the technological developments in the field of the Internet and Internet technologies and to determine their significance for sustainable development, which will result in the emergence of Society 5.0. (2) The authors used automated content analysis for the analysis of 552 articles published in 306 scientific journals indexed by SCII and/or SCI - EXPANDED (Web of Science (WOS) platform). The goal of the research was to present the relationship between the Internet and sustainable development. (3) Results: The results of the analysis show that the top four most important themes in the selected journals were “development”, “information”, “data”, and “business and services”. (4) Conclusions: Our research approach emphasizes the importance of the culmination of scientific innovation with the conceptual, technological and contextual frameworks of the Internet and Internet technology usage and its impact on sustainable development and the emergence of the Society 5.0.

The Merits of Dynamic Data Acquisition for Realistic Myocontrol

Natural myocontrol is the intuitive control of a prosthetic limb via the user's voluntary muscular activations. This type of control is usually implemented by means of pattern recognition, which uses a set of training data to create a model that can decipher these muscular activations. A consequence of this approach is that the reliability of a myocontrol system depends on how representative this training data is for all types of signal variability that may be encountered when the amputee puts the prosthesis into real use. Myoelectric signals are indeed known to vary according to the position and orientation of the limb, among other factors, which is why it has become common practice to take this variability into account by acquiring training data in multiple body postures. To shed further light on this problem, we compare two ways of collecting data: while the subjects hold their limb statically in several positions one at a time, which is the traditional way, or while they dynamically move their limb at a constant pace through those same positions. Since our interest is to investigate any differences when controlling an actual prosthetic device, we defined an evaluation protocol that consisted of a series of complex, bimanual daily-living tasks. Fourteen intact participants performed these tasks while wearing prosthetic hands mounted on splints, which were controlled via either a statically or dynamically built myocontrol model. In both cases all subjects managed to complete all tasks and participants without previous experience in myoelectric control manifested a significant learning effect; moreover, there was no significant difference in the task completion times achieved with either model. When evaluated in a simulated scenario with traditional offline performance evaluation, on the other hand, the dynamically-trained system showed significantly better accuracy. Regardless of the setting, the dynamic data acquisition was faster, less tiresome, and better accepted by the users. We conclude that dynamic data acquisition is advantageous and confirm the limited relevance of offline analyses for online myocontrol performance.

Action interference in simultaneous and proportional myocontrol: comparing force- and electromyography

Myocontrol, that is, control of a prosthesis via muscle signals, is still a surprisingly hard problem. Recent research indicates that surface electromyography (sEMG), the traditional technique used to detect a subject's intent, could proficiently be replaced, or conjoined with, other techniques (multi-modal myocontrol), with the aim to improve both on dexterity and reliability. Objective. In this paper we present an online assessment of multi-modal sEMG and force myography (FMG) targeted at hand and wrist myocontrol. Approach. Twenty sEMG and FMG sensors in total were used to enforce simultaneous and proportional control of hand opening/closing, wrist pronation/supination and wrist flexion/extension of 12 intact subjects. Main results and Significance. We found that FMG yields in general a better performance than sEMG, and that the main drawback of the sEMG array we used is not the inability to perform a desired action, but rather action interference, that is, the undesired concurrent activation of another action. FMG, on the other hand, causes less interference.

Toward Enhanced Teleoperation Through Embodiment

Telerobotics aims to transfer human manipulation skills and dexterity over an arbitrary distance and at an arbitrary scale to a remote workplace. A telerobotic system that is transparent enables a natural and intuitive interaction. We postulate that embodiment (with three sub-components: sense of ownership, agency, and self-location) of the robotic system leads to optimal perceptual transparency and increases task performance. However, this has not yet been investigated directly. We reason along four premises and present findings from the literature that substantiate each of them: (1) the brain can embody non-bodily objects (e.g., robotic hands), (2) embodiment can be elicited with mediated sensorimotor interaction, (3) embodiment is robust against inconsistencies between the robotic system and the operator's body, and (4) embodiment positively correlates to dexterous task performance. We use the predictive encoding theory as a framework to interpret and discuss the results reported in the literature. Numerous previous studies have shown that it is possible to induce embodiment over a wide range of virtual and real extracorporeal objects (including artificial limbs, avatars, and android robots) through mediated sensorimotor interaction. Also, embodiment can occur for non-human morphologies including for elongated arms and a tail. In accordance with the predictive encoding theory, none of the sensory modalities is critical in establishing ownership, and discrepancies in multisensory signals do not necessarily lead to loss of embodiment. However, large discrepancies in terms of multisensory synchrony or visual likeness can prohibit embodiment from occurring. The literature provides less extensive support for the link between embodiment and (dexterous) task performance. However, data gathered with prosthetic hands do indicate a positive correlation. We conclude that all four premises are supported by direct or indirect evidence in the literature, suggesting that embodiment of a remote manipulator may improve dexterous performance in telerobotics. This warrants further implementation testing of embodiment in telerobotics. We formulate a first set of guidelines to apply embodiment in telerobotics and identify some important research topics.

Sense of ownership and not the sense of agency is spatially bounded within the space reachable with the unaugmented hand

While reaching for a coffee cup, we are aware that the hand we see belongs to us and it moves at our will (reflecting our senses of ownership and agency, respectively), and that the cup is within our hand's reach rather than beyond it (i.e., in reachable space, RS, rather than in non-reachable space, NRS). Accepted psychological explanations of our sense of ownership, sense of agency, and our perception of space surrounding the body as RS or NRS propose a unitary dependence on Euclidean distance from the body. Here, we propose an alternate, affordance-based explanation of experienced ownership, agency, and perception of space surrounding the body as RS and NRS. Adult participants experienced the static rubber hand illusion (RHI) and its dynamic variant, while the rubber hand was either within their arm's reach (i.e., in self-identified RS) or beyond it (i.e., in self-identified NRS). We found that when the participants experienced synchronous visual and tactile signals in the static RHI, and synchronous visual and kinesthetic signals in the dynamic RHI, they felt illusory ownership when the rubber hand was in RS but not when it was in NRS. Conversely, when the participants experienced synchronous visual and kinesthetic signals in the dynamic RHI, they felt agency, regardless of the rubber hand's location. In addition, illusory ownership was accompanied by proprioceptive drift, a feeling that their hand was closer to the rubber hand than it actually was, but agency was not accompanied by proprioceptive drift. Together, these results indicate that our sense of ownership, while malleable enough to incorporate visible non-corporeal objects resembling a body part, is spatially constrained by proprioceptive signals specifying that body part's actual location. In contrast, our sense of agency can incorporate a visible non-corporeal object, independent of its location with respect to the body. We propose that the psychological processes mediating our sense of ownership are closely linked with our perception of space surrounding the body, and that the spatial independence of our sense of agency reflects the coupling between our actions and perception of the environment, such as while using handheld tools as extensions of our body.

How Cognitive Models of Human Body Experience Might Push Robotics

In the last decades, cognitive models of multisensory integration in human beings have been developed and applied to model human body experience. Recent research indicates that Bayesian and connectionist models might push developments in various branches of robotics: assistive robotic devices might adapt to their human users aiming at increased device embodiment, e.g., in prosthetics, and humanoid robots could be endowed with human-like capabilities regarding their surrounding space, e.g., by keeping safe or socially appropriate distances to other agents. In this perspective paper, we review cognitive models that aim to approximate the process of human sensorimotor behavior generation, discuss their challenges and potentials in robotics, and give an overview of existing approaches. While model accuracy is still subject to improvement, human-inspired cognitive models support the understanding of how the modulating factors of human body experience are blended. Implementing the resulting insights in adaptive and learning control algorithms could help to taylor assistive devices to their user's individual body experience. Humanoid robots who develop their own body schema could consider this body knowledge in control and learn to optimize their physical interaction with humans and their environment. Cognitive body experience models should be improved in accuracy and online capabilities to achieve these ambitious goals, which would foster human-centered directions in various fields of robotics.

Robotic hand illusion with tactile feedback: Unravelling the relative contribution of visuotactile and visuomotor input to the representation of body parts in space

The rubber hand illusion describes a phenomenon in which participants experience a rubber hand as being part of their body by the synchronous application of visuotactile stimulation to the real and the artificial limb. In the recently introduced robotic hand illusion (RobHI), a robotic hand is incorporated into one’s body representation due to the integration of synchronous visuomotor information. However, there are no setups so far that combine visuotactile and visuomotor feedback, which is expected to unravel mechanisms that cannot be detected in experimental designs applying this information in isolation. We developed a robotic hand, controlled by a sensor glove and equipped with pressure sensors, and varied systematically and separately the synchrony for motor feedback (MF) and tactile feedback (TF). In Experiment 1, we implemented a ball-grasping task and assessed the perceived proprioceptive drift of one’s own hand as a behavioral measure of the spatial calibration of body coordinates as well as explicit embodiment experiences by a questionnaire. Results revealed significant main effects of both MF and TF for proprioceptive drift data, but we only observed main effects for MF on perceived embodiment. Furthermore, for the proprioceptive drift we found that synchronous feedback in one factor compensates for asynchronous feedback in the other. In Experiment 2, including a new sample of naïve participants, we further explored this finding by adding unimodal conditions, in which we manipulated the presence or absence of MF and/or TF. These findings replicated the results from Experiment 1 and we further found evidence for a supper-additive multisensory effect on spatial body representation caused by the presence of both factors. Results on conscious body perception were less consistent across both experiments. The findings indicate that sensory and motor input equally contribute to the representation of spatial body coordinates which for their part are subject to multisensory enhancing effects. The results outline the potential of human-in-the-loop approaches and might have important implications for clinical applications such as for the future design of robotic prostheses.