Steer by ear: Myoelectric auricular control of powered wheelchairs for individuals with spinal cord injury

Exploring crossmodal correspondences for future research in human movement augmentation

"Crossmodal correspondences" are the consistent mappings between perceptual dimensions or stimuli from different sensory domains, which have been widely observed in the general population and investigated by experimental psychologists in recent years. At the same time, the emerging field of human movement augmentation (i.e., the enhancement of an individual's motor abilities by means of artificial devices) has been struggling with the question of how to relay supplementary information concerning the state of the artificial device and its interaction with the environment to the user, which may help the latter to control the device more effectively. To date, this challenge has not been explicitly addressed by capitalizing on our emerging knowledge concerning crossmodal correspondences, despite these being tightly related to multisensory integration. In this perspective paper, we introduce some of the latest research findings on the crossmodal correspondences and their potential role in human augmentation. We then consider three ways in which the former might impact the latter, and the feasibility of this process. First, crossmodal correspondences, given the documented effect on attentional processing, might facilitate the integration of device status information (e.g., concerning position) coming from different sensory modalities (e.g., haptic and visual), thus increasing their usefulness for motor control and embodiment. Second, by capitalizing on their widespread and seemingly spontaneous nature, crossmodal correspondences might be exploited to reduce the cognitive burden caused by additional sensory inputs and the time required for the human brain to adapt the representation of the body to the presence of the artificial device. Third, to accomplish the first two points, the benefits of crossmodal correspondences should be maintained even after sensory substitution, a strategy commonly used when implementing supplementary feedback.

Human-machine interface for two-dimensional steering control with the auricular muscles

Human-machine interfaces (HMIs) can be used to decode a user's motor intention to control an external device. People that suffer from motor disabilities, such as spinal cord injury, can benefit from the uses of these interfaces. While many solutions can be found in this direction, there is still room for improvement both from a decoding, hardware, and subject-motor learning perspective. Here we show, in a series of experiments with non-disabled participants, a novel decoding and training paradigm allowing naïve participants to use their auricular muscles (AM) to control two degrees of freedom with a virtual cursor. AMs are particularly interesting because they are vestigial muscles and are often preserved after neurological diseases. Our method relies on the use of surface electromyographic records and the use of contraction levels of both AMs to modulate the velocity and direction of a cursor in a two-dimensional paradigm. We used a locking mechanism to fix the current position of each axis separately to enable the user to stop the cursor at a certain location. A five-session training procedure (20-30 min per session) with a 2D center-out task was performed by five volunteers. All participants increased their success rate (Initial: 52.78 ± 5.56%; Final: 72.22 ± 6.67%; median ± median absolute deviation) and their trajectory performances throughout the training. We implemented a dual task with visual distractors to assess the mental challenge of controlling while executing another task; our results suggest that the participants could perform the task in cognitively demanding conditions (success rate of 66.67 ± 5.56%). Finally, using the Nasa Task Load Index questionnaire, we found that participants reported lower mental demand and effort in the last two sessions. To summarize, all subjects could learn to control the movement of a cursor with two degrees of freedom using their AM, with a low impact on the cognitive load. Our study is a first step in developing AM-based decoders for HMIs for people with motor disabilities, such as spinal cord injury.

Atavistic and vestigial anatomical structures in the head, neck, and spine: an overview

Organisms may retain nonfunctional anatomical features as a consequence of evolutionary natural selection. Resultant atavistic and vestigial anatomical structures have long been a source of perplexity. Atavism is when an ancestral trait reappears after loss through an evolutionary change in previous generations, whereas vestigial structures are remnants that are largely or entirely functionless relative to their original roles. While physicians are cognizant of their existence, atavistic and vestigial structures are rarely emphasized in anatomical curricula and can, therefore, be puzzling when discovered incidentally. In addition, the literature is replete with examples of the terms atavistic and vestigial being used interchangeably without careful distinction between them. We provide an overview of important atavistic and vestigial structures in the head, neck, and spine that can serve as a reference for anatomists and clinical neuroscientists. We review the literature on atavistic and vestigial anatomical structures of the head, neck, and spine that may be encountered in clinical practice. We define atavistic and vestigial structures and employ these definitions consistently when classifying anatomical structures. Pertinent anatomical structures are numerous and include human tails, plica semilunaris, the vomeronasal organ, levator claviculae, and external ear muscles, to name a few. Atavistic and vestigial structures are found throughout the head, neck, and spine. Some, such as human tails and branchial cysts may be clinically symptomatic. Literature reports indicate that their prevalence varies across populations. Knowledge of atavistic and vestigial anatomical structures can inform diagnoses, prevent misrecognition of variation for pathology, and guide clinical interventions.

Electromyography of Extrinsic and Intrinsic Ear Muscles in Healthy Probands and Patients with Unilateral Postparalytic Facial Synkinesis

There are currently no data on the electromyography (EMG) of all intrinsic and extrinsic ear muscles. The aim of this work was to develop a standardized protocol for a reliable surface EMG examination of all nine ear muscles in twelve healthy participants. The protocol was then applied in seven patients with unilateral postparalytic facial synkinesis. Based on anatomic preparations of all ear muscles on two cadavers, hot spots for the needle EMG of each individual muscle were defined. Needle and surface EMG were performed in one healthy participant; facial movements could be defined for the reliable activation of individual ear muscles’ surface EMG. In healthy participants, most tasks led to the activation of several ear muscles without any side difference. The greatest EMG activity was seen when smiling. Ipsilateral and contralateral gaze were the only movements resulting in very distinct activation of the transversus auriculae and obliquus auriculae muscles. In patients with facial synkinesis, ear muscles’ EMG activation was stronger on the postparalytic compared to the contralateral side for most tasks. Additionally, synkinetic activation was verifiable in the ear muscles. The surface EMG of all ear muscles is reliably feasible during distinct facial tasks, and ear muscle EMG enriches facial electrodiagnostics.

Performance Analysis of a Head and Eye Motion-Based Control Interface for Assistive Robots

Assistive robots support people with limited mobility in their everyday life activities and work. However, most of the assistive systems and technologies for supporting eating and drinking require a residual mobility in arms or hands. For people without residual mobility, different hands-free controls have been developed. For hands-free control, the combination of different modalities can lead to great advantages and improved control. The novelty of this work is a new concept to control a robot using a combination of head and eye motions. The control unit is a mobile, compact and low-cost multimodal sensor system. A Magnetic Angular Rate Gravity (MARG)-sensor is used to detect head motion and an eye tracker enables the system to capture the user’s gaze. To analyze the performance of the two modalities, an experimental evaluation with ten able-bodied subjects and one subject with tetraplegia was performed. To assess discrete control (event-based control), a button activation task was performed. To assess two-dimensional continuous cursor control, a Fitts’s Law task was performed. The usability study was related to a use-case scenario with a collaborative robot assisting a drinking action. The results of the able-bodied subjects show no significant difference between eye motions and head motions for the activation time of the buttons and the throughput, while, using the eye tracker in the Fitts’s Law task, the error rate was significantly higher. The subject with tetraplegia showed slightly better performance for button activation when using the eye tracker. In the use-case, all subjects were able to use the control unit successfully to support the drinking action. Due to the limited head motion of the subject with tetraplegia, button activation with the eye tracker was slightly faster than with the MARG-sensor. A further study with more subjects with tetraplegia is planned, in order to verify these results.

Development of a training paradigm for voluntary control of the peri-auricular muscles: a feasibility study

BACKGROUND

Spinal cord injury (SCI) can lead to severe and permanent functional deficits. In humans, peri-auricular muscles (PAMs) do not serve any physiological function, though their innervation is preserved in even high level SCI. Auricular control systems provide a good example of leveraging contemporary technologies (e.g., sEMG controlled computer games) to enable those with disabilities. Our primary objective is to develop and test the effectiveness of an auricular muscle training protocol to facilitate isolated and coordinated, bilateral voluntary control that could be used in individuals without volitional control of the vestigial PAMs.

METHODS

Seventeen non-disabled persons were screened; 13 were eligible and 10 completed the entire protocol. The facilitation phase, included one session of sub-motor threshold, sensory electrical stimulation followed by neuromuscular electrical stimulation paired with ear movement feedback for up to 8 additional sessions. Participants progressed to the skill acquisition phase where they dawned an auricular control device that used sEMG signals to control movements of a cursor through three levels of computer games, each requiring increasingly more complex PAM coordination.

RESULTS

The 10 who completed the protocol, finished the facilitation phase in 3 to 9 sessions and achieved some level of voluntary auricle movement that ranged between 1 and 5 mm. Qualitative analysis of longitudinal post-session auricular movement, revealed two subgroups of learners. Six successfully completed all 3 games-the "Learners". Two were partially successful in game completion and two were unable to complete a single game--"Poor/Non-Learners". Quantitative analysis revealed a significant group difference in auricular amplitude for both facilitation and skill phases (p < .05), and a significant relationship between performance in the two phases (R² = 0.84, p = 0.004).

CONCLUSION

Sixty percent of those who completed the facilitation phase were able to learn and demonstrate functional voluntary control of the vestigial PAMs. Those who progressed the fastest through facilitation were also those who were most proficient in skill acquisition with the device. There was considerable variability in progression through the two-phase protocol, with 20% deemed Poor/Non-Learners and unable to complete even the most basic game following training. There were no serious adverse events.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02358915 , first posted February 9, 2015.

A hybrid auricular control system: direct, simultaneous, and proportional myoelectric control of two degrees of freedom in prosthetic hands

OBJECTIVE

The conventional myoelectric control scheme of hand prostheses provides a high level of robustness during continuous use. Typically, the electrical activity of an agonist/antagonist muscle pair in the forearm is detected and used to control either opening/closing or rotation of the prosthetic hand. The translation of more sophisticated control approaches (e.g. regression-based classifiers) to clinical practice is limited mainly because of their lack of robustness in real-world conditions (e.g. due to different arm positions). We therefore explore a new hybrid approach, in which a second degree of freedom (DOF) controlled by the myoelectric activity of the posterior auricular muscles is added to the conventional forearm control. With this, an independent, simultaneous and proportional control of rotation and opening/closing of the hand is possible.

APPROACH

In this study, we compared the hybrid auricular control system (hACS) to the two most commonly used control techniques for two DOF. Ten able-bodied subjects and one person with transradial amputation performed two standardizes tests in three different arm positions.

MAIN RESULTS

Subjects controlled a hand prosthesis significantly more rapidly and more accurately using the hACS. Moreover, the robustness of the system was not influenced by different arm positions.

SIGNIFICANCE

The hACS therefore offers an alternative solution for simultaneous and proportional myoelectric control of two degrees of freedom that avoids several robustness issues related to machine learning based approaches.

Cortical representation of auricular muscles in humans: A robot-controlled TMS mapping and fMRI study

BACKGROUND

Most humans have the ability to activate the auricular muscles. Although (intentional) control suggests an involvement of higher cortical centers underlying posterior auricular muscle (PAM) activation, the cortical representation of the auricular muscles is still unknown.

METHODS

With the purpose of identifying a possible cortical representation area we performed automated robotic and image-guided transcranial magnetic stimulation (TMS) mapping (n = 8) and functional magnetic resonance imaging (fMRI) (n = 13). For topographical comparison, a similar experimental protocol was applied for the first dorsal interosseus muscle (FDI) of the hand.

RESULTS

The calculated centers of gravity (COGs) of both muscles were located on the precentral gyrus with the PAM COGs located more laterally compared to the FDI. The distance between the mean PAM and mean FDI COG was 26.3 mm. The TMS mapping results were confirmed by fMRI, which showed a dominance of cortical activation within the precentral gyrus during the corresponding motor tasks. The correspondence of TMS and fMRI results was high.

CONCLUSION

The involvement of the primary motor cortex in PAM activation might point to an evolved function of the auricular muscles in humans and/or the ability of intentional (and selective) muscle activation.

Neuroprosthetics for Auricular Muscles: Neural Networks and Clinical Aspects

The mammalian external ear houses extrinsic and intrinsic auricular muscles. There are three extrinsic auricular muscles-the posterior, superior, and anterior auricular muscles-and six intrinsic muscles-the helicis major and minor, tragicus, anti-tragicus, transverse and oblique muscles. These muscles have been considered vestigial in humans. However, numerous therapeutic and diagnostic wearable devices are designed to monitor and alleviate the symptoms of neurological disorders, brainstem injuries, emotional states, and auditory functions, by making use of the neural networks of the auricular muscles and their locations, which are easily accessible for ergonomic wearable biomedical devices. They can also serve as a bio-controller of human neuroprosthetics. The functionality of these auricular muscles remains elusive and requires further experimentation for a more in-depth understanding of their anatomy. The aims of this review are (1) to provide a detailed account of the neural networks of the extrinsic and intrinsic auricular muscles, (2) to describe diagnostic and therapeutic functions of these muscles as demonstrated in the current literature, and (3) to outline existing and potential neuroprosthetic applications making use of the auricular muscles and their neural networks.

Using Self-Reliance Factors to Decide How to Share Control Between Human Powered Wheelchair Drivers and Ultrasonic Sensors

A shared-control scheme for a powered wheelchair is presented. The wheelchair can be operated by a wheelchair driver using a joystick, or directed by a sensor system, or control can be combined between them. The wheelchair system can modify direction depending on the local environment. Sharing the control allows a disabled wheelchair driver to drive safely and efficiently. The controller automatically establishes the control gains for the sensor system and the human driver by calculating a self-reliance factor for the wheelchair driver. The sensor system can influence the motion of the wheelchair to compensate for some deficiency in a disabled driver. Practical tests validate the proposed techniques and designs.

Control scheme selection in human-machine- interfaces by analysis of activity signals

Human-Machine Interfaces in rehabilitation engineering often use activity signals. Examples are electrical wheelchairs or prostheses controlled by means of muscle contractions. Activity signals are user-dependent and often reflect neurological weaknesses. Thus, not all users are able to operate the same control scheme in a robust manner. To avoid under- and overstraining, the interface ideally uses a control scheme which reflects the user’s control ability best. Therefore, we explored typical phenomena of activation signals. We derive criteria to quantify the user’s performance and abilities and present a routine which automatically selects and adapts one of three control schemes being best suited.