Overcoming the ceiling effects of experts' motor expertise through active haptic training

Validity of a virtual reality-based straight coloanal anastomosis simulator

PURPOSE

Current training methods for surgical trainees are inadequate because they are costly, low-fidelity, or have a low skill ceiling. This work aims to expand available virtual reality training options by developing a VR trainer for straight coloanal anastomosis (SCA), one of the Colorectal Objective Structured Assessment of Technical Skills (COSATS) tasks.

METHODS

We developed a VR-based SCA simulator to evaluate trainees based on their performance. To increase the immersiveness, alongside the VR headset, we used haptics as the primary method of interaction with the simulation. We also implemented objective performance metrics to evaluate trainee performance throughout the simulation.

RESULTS

We presented our performance metrics to 27 participants for an Expert Consensus Survey (5-point Likert scale) and created weights for our metrics. The weighted average scores for the 24 task-specific metrics ranged from 3.5 to 5. Additionally, for the general metrics, the scores spanned from 3.3 to 4.6. In the second phase of our study, we conducted a study with 16 participants (novice n = 9, expert n = 7). Based on the performance, experts outperformed novices by 8.56% when referring to the total score (p = 0.0041). Three of the measurable metrics, purse suture (p = 0.0797), retracting the anvil (p = 0.0738), and inserting the colonoscope (p = 0.0738) showed a significant difference between experts and novices. Experts were smoother with their hand motions by 3.67% per second and took 70.77% longer paths to complete the same tasks.

CONCLUSION

We created a high-fidelity coloanal anastomosis VR simulator. The simulator runs in real-time while allowing high immersion with a VR headset, deformable bodies, and a haptic device while providing objective feedback through performance metrics. Experts obtained higher scores throughout the simulation, including the quiz to demonstrate procedural knowledge, the metrics to demonstrate experience in steps/procedure, and control of their basic surgical skills and hand movements.

Enhanced Somatosensory Inhibition Sharpens Hand Representation and Sensorimotor Skills in Pianists

Dexterous motor skills, like those needed for playing musical instruments and sports, require the somatosensory system to accurately and rapidly process somatosensory information from multiple body parts. This is challenging due to the convergence of afferent inputs from different body parts into a single neuron and the overlapping representation of neighboring body parts in the somatosensory cortices. How do trained individuals, such as pianists and athletes, manage this? Here, a series of five experiments with pianists and nonmusicians (female and male) shows that pianists have enhanced inhibitory function in the somatosensory system, which isolates the processing of somatosensory afferent inputs from each finger. This inhibitory function was assessed using a paired-pulse paradigm of somatosensory evoked potentials in electroencephalography, which measures the suppressive effect of a first stimulus [i.e., conditioning stimulus (CS)] on the response to a subsequent second stimulus. We found that pianists and nonmusicians showed an inhibitory response to the sequential stimuli to the peripheral somatosensory nerve at the wrist when the CS was intense. However, only pianists exhibited an inhibitory response to a weak CS, indicating enhanced inhibitory function in pianists. Additionally, the CS increased the information content segregating individual fingers represented in the cortical activity evoked by passive finger movements and improved the perception of fast multifinger sequential movements, specifically for pianists. Our findings provide the first evidence for experience-dependent plasticity in somatosensory inhibitory function and highlight its role in the expert motor performance of pianists.

Decomposition of a complex motor skill with precise error feedback and intensive training breaks expertise ceiling

Complex motor skills involve intricate sequences of movements that require precise temporal coordination across multiple body parts, posing challenges to mastery based on perceived error or reward. One approach that has been widely used is to decompose such skills into simpler, constituent movement elements during the learning process, thereby aligning the task complexity with the learners' capacity for accurate execution. Despite common belief and prevalent adoption, the effectiveness of this method remains elusive. Here we addressed this issue by decomposing a sequence of precisely timed coordination of movements across multiple fingers into individual constituent elements separately during piano practice. The results demonstrated that the decomposition training enhanced the accuracy of the original motor skill, a benefit not achieved through mere repetition of movements alone, specifically when skilled pianists received explicit visual feedback on timing error in the order of milliseconds during training. During the training, the patterns of multi-finger movements changed significantly, suggesting exploration of movements to refine the skill. By contrast, neither unskilled pianists who underwent the same training nor skilled pianists who performed the decomposition training without receiving visual feedback on the error showed improved skill through training. These findings offer novel evidences suggesting that decomposing a complex motor skill, coupled with receiving feedback on subtle movement error during training, further enhances motor expertise of skilled individuals by facilitating exploratory refinement of movements.

Surmounting the ceiling effect of motor expertise by novel sensory experience with a hand exoskeleton

For trained individuals such as athletes and musicians, learning often plateaus after extensive training, known as the "ceiling effect." One bottleneck to overcome it is having no prior physical experience with the skill to be learned. Here, we challenge this issue by exposing expert pianists to fast and complex finger movements that cannot be performed voluntarily, using a hand exoskeleton robot that can move individual fingers quickly and independently. Although the skill of moving the fingers quickly plateaued through weeks of piano practice, passive exposure to otherwise impossible complex finger movements generated by the exoskeleton robot at a speed faster than the pianists' fastest one enabled them to play faster. Neither a training for fast but simple finger movements nor one for slow but complex movements with the exoskeleton enhanced the overtrained motor skill. The exoskeleton training with one hand also improved the motor skill of the untrained contralateral hand, demonstrating the intermanual transfer effect. The training altered patterns of coordinated activities across multiple finger muscles during piano playing but not in general motor and somatosensory functions or in anatomical characteristics of the hand (range of motion). Patterns of the multifinger movements evoked by transcranial magnetic stimulation over the left motor cortex were also changed through passive exposure to fast and complex finger movements, which accompanied increased involvement of constituent movement elements characterizing the individuated finger movements. The results demonstrate evidence that somatosensory exposure to an unexperienced motor skill allows surmounting of the ceiling effect in a task-specific but effector-independent manner.

Haptic Technology: Exploring Its Underexplored Clinical Applications-A Systematic Review

BACKGROUND/OBJECTIVES

Haptic technology has transformed interactions between humans and both tangible and virtual environments. Despite its widespread adoption across various industries, the potential therapeutic applications of this technology have yet to be fully explored.

METHODS

A systematic review of randomized controlled trials (RCTs) and randomized crossover trials was conducted, utilizing databases such as PubMed, Embase, Cochrane Library, and Web of Science. This review included studies reporting clinical applications of haptic technology in rehabilitation, cognition, wellness, and mental health among adult subjects.

RESULTS

This systematic review included 34 studies, of which 20 focused on clinical outcomes and 14 on learning clinical skills. The results showed that haptic devices, both robotic and non-robotic, enhance sensorimotor performance and motor function in rehabilitation settings, especially in post-stroke recovery, with reported effect sizes ranging from 0.2 to 0.7. The majority of the haptic technologies reported were integrated into robotic systems (40%). Haptic devices were also reported to improve clinical skills training by providing tactile feedback that enhances procedural performance and trainee self-efficacy. In fact, surgical simulations accounted for 79% of all the modalities used for medical training.

CONCLUSIONS

This review underscores the potential yet underexplored applications of haptic technology in healthcare, including medical education, rehabilitation, cognition, and mental health. The key limitations of this review include heterogeneity across studies, small sample sizes, and a scarcity of comprehensive, long-term investigations. Therefore, future research should aim to validate these findings further and expand the applications of haptic technology to maximize its utility in the healthcare industry and clinical practice.

Enhance Kinesthetic Experience in Perceptual Learning for Welding Motor Skill Acquisition With Virtual Reality and Robot-Based Haptic Guidance

Welding is an important operation in many industries, including construction and manufacturing, which requires extensive training and practices. Although welding simulators have been used to accommodate welding training, it is still challenging to enable novice trainees to effectively understand the kinesthetic experience of the expert in an egocentric manner, such as the proper way of force exertion in complex welding operations. This study implements a robot-assisted perceptual learning system to transfer the expert welders' experience to trainees, including both the positional and force control actions. A human-subject experiment (N = 30) was performed to understand the motor skill acquisition process. Three conditions (control, robotic positional guidance with force visualization, and force perceptual learning with position visualization) were tested to evaluate the role of robotic guidance in welding motion control and force exertion. The results indicated various benefits related to task completion time and force control accuracy under the robotic guidance. The findings can inspire the design of future welding training systems enabled by external robotic systems.

Adaptive tactile interaction transfer via digitally embroidered smart gloves

Human-machine interfaces for capturing, conveying, and sharing tactile information across time and space hold immense potential for healthcare, augmented and virtual reality, human-robot collaboration, and skill development. To realize this potential, such interfaces should be wearable, unobtrusive, and scalable regarding both resolution and body coverage. Taking a step towards this vision, we present a textile-based wearable human-machine interface with integrated tactile sensors and vibrotactile haptic actuators that are digitally designed and rapidly fabricated. We leverage a digital embroidery machine to seamlessly embed piezoresistive force sensors and arrays of vibrotactile actuators into textiles in a customizable, scalable, and modular manner. We use this process to create gloves that can record, reproduce, and transfer tactile interactions. User studies investigate how people perceive the sensations reproduced by our gloves with integrated vibrotactile haptic actuators. To improve the effectiveness of tactile interaction transfer, we develop a machine-learning pipeline that adaptively models how each individual user reacts to haptic sensations and then optimizes haptic feedback parameters. Our interface showcases adaptive tactile interaction transfer through the implementation of three end-to-end systems: alleviating tactile occlusion, guiding people to perform physical skills, and enabling responsive robot teleoperation.

Active perceptual learning involves motor exploration and adaptation of predictive sensory integration

Our ability to perceive both externally generated and self-generated sensory stimuli can be enhanced through training, known as passive and active perceptual learning (APL). Here, we sought to explore the mechanisms underlying APL by using active haptic training (AHT), which has been demonstrated to enhance the somatosensory perception of a finger in a trained motor skill. In total 120 pianists participated in this study. First, AHT reorganized the muscular coordination during the piano keystroke. Second, AHT increased the relative reliance on afferent sensory information relative to predicted one, in contrast to no increment of overall perceptual sensitivity. Finally, AHT improved feedback movement control of keystrokes. These results suggest that APL involves active exploration and adaptation of predictive sensory integration, which underlies the co-enhancement of active perception and feedback control of movements of well-trained individuals.

The Effect of Different Combinations of Practice Schedules on Motor Response Stability during Practice

Many results in motor learning have indicated that relative and absolute timing dimensions are modulated by factors that modify response stability among trials. One of these factors is the combination of constant and variable practices. Although many researchers have investigated the combination of practice schedules, these researchers have used measurements that do not assess performance and motor response separately. This study aimed to investigate the effect of different combinations of practice schedules on motor response stability during practice. Participants performed a sequential key-pressing task with two goals: (1) to learn the relative timing dimension and (2) the absolute timing dimension. Participants were assigned to one of two groups: constant-variable or variable-constant. Our findings indicate an influence of the increase in variability over the practice in the constant-variable group. Precisely, the increase in variability of total time in the second half (constant-variable group) of practice was followed by the maintenance of the same level of cross-correlate between absolute timing error and variability of total time. Finally, our findings support the hypothesis that practicing in a constant schedule favors the relative timing dimension of learning regardless of the order in which the constant practice is provided.

Passive somatosensory training enhances piano skill in adolescent and adult pianists: A preliminary study

Sensory afferent information, such as auditory and somatosensory feedback while moving, plays a crucial role in both control and learning of motor performance across the lifespan. Music performance requires skillful integration of multimodal sensory information for the production of dexterous movements. However, it has not been understood what roles somatosensory afferent information plays in the acquisition and sophistication of specialized motor skills of musicians across different stages of development. In the present preliminary study, we addressed this issue by using a novel technique with a hand exoskeleton robot that can externally move the fingers of pianists. Short-term exposure to fast and complex finger movements generated by the exoskeleton (i.e., passive movements) increased the maximum rate of repetitive piano keystrokes by the pianists. This indicates that somatosensory inputs derived from the externally generated motions enhanced the quickness of the sequential finger movements in piano performance, even though the pianists did not voluntarily move the fingers. The enhancement of motor skill through passive somatosensory training using the exoskeleton was more pronounced in adolescent pianists than adult pianists. These preliminary results implicate a sensitive period of neuroplasticity of the somatosensory-motor system of trained pianists, which emphasizes the importance of somatosensory-motor training in professional music education during adolescence.

The plyometric activity as a conditioning to enhance strength and precision of the finger movements in pianists

Stability of timing and force production in repetitive movements characterizes skillful motor behaviors such as surgery and playing musical instruments. However, even trained individuals such as musicians undergo further extensive training for the improvement of these skills. Previous studies that investigated the lower extremity movements such as jumping and sprinting demonstrated enhancement of the maximum force and rate of force development immediately after the plyometric exercises. However, it remains unknown whether the plyometric exercises enhance the stability of timing and force production of the dexterous finger movements in trained individuals. Here we address this issue by examining the effects of plyometric exercise specialized for finger movements on piano performance. We compared the training-related changes in the piano-key motion and several physiological features of the finger muscles (e.g., electromyography, rate of force development, and muscle temperature) by well-trained pianists. The conditioning demonstrated a decrease of the variation in timing and velocity of successive keystrokes, along with a concomitant increase in the rate of force development of the four fingers, but not the thumb, although there was no change in the finger muscular activities through the activity. By contrast, such a conditioning effect was not evident following a conventional repetitive piano practice. In addition, a significant increase in the forearm muscle temperature was observed specifically through performing the plyometric exercise with the fingers, implying its association with improved performance. These results indicate effectiveness of the plyometric exercises for improvement of strength, precision, and physiological efficiency of the finger movements even in expert pianists, which implicates that ways of practicing play a key role in enhancing experts' expertise.

Noncontact and High-Precision Sensing System for Piano Keys Identified Fingerprints of Virtuosity

Dexterous tool use is typically characterized by fast and precise motions performed by multiple fingers. One representative task is piano playing, which involves fast performance of a sequence of complex motions with high spatiotemporal precision. However, for several decades, a lack of contactless sensing technologies that are capable of precision measurement of piano key motions has been a bottleneck for unveiling how such an outstanding skill is cultivated. Here, we developed a novel sensing system that can record the vertical position of all piano keys with a time resolution of 1 ms and a spatial resolution of 0.01 mm in a noncontact manner. Using this system, we recorded the piano key motions while 49 pianists played a complex sequence of tones that required both individuated and coordinated finger movements to be performed as fast and accurately as possible. Penalized regression using various feature variables of the key motions identified distinct characteristics of the key-depressing and key-releasing motions in relation to the speed and accuracy of the performance. For the maximum rate of the keystrokes, individual differences across the pianists were associated with the peak key descending velocity, the key depression duration, and key-lift timing. For the timing error of the keystrokes, the interindividual differences were associated with the peak ascending velocity of the key and the inter-strike variability of both the peak key descending velocity and the key depression duration. These results highlight the importance of dexterous control of the vertical motions of the keys for fast and accurate piano performance.

Back to feedback: aberrant sensorimotor control in music performance under pressure

Precisely timed production of dexterous actions is often destabilized in anxiogenic situations. Previous studies demonstrated that cognitive functions such as attention and working memory as well as autonomic nervous functions are susceptible to psychological stress in skillful performance while playing sports or musical instruments. However, it is not known whether the degradation of sensorimotor functions underlies such a compromise of skillful performance due to psychophysiological distress. Here, we addressed this issue through a set of behavioral experiments. After artificially delaying the timing of tone production while playing the piano, the local tempo was abnormally disrupted only under pressure. The results suggest that psychological stress degraded the temporal stability of movement control due to an abnormal increase in feedback gain. A learning experiment further demonstrated that the temporal instability of auditory-motor control under pressure was alleviated after practicing piano while ignoring delayed auditory feedback but not after practicing while compensating for the delayed feedback. Together, these findings suggest an abnormal transition from feedforward to feedback control in expert piano performance with psychological stress, which can be mitigated through specialized sensorimotor training that involves piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

In order to establish if the degradation of sensorimotor functions underlies the stress-associated disruption of skilful performance, Furuya et al examined participants undergoing a piano playing task under stress. Their data suggests the occurrence of a stress-induced transition from feedforward to feedback control, which can be mitigated through sensorimotor training involving piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

Enhancement of loudness discrimination acuity for self-generated sound is independent of musical experience

Musicians tend to have better auditory and motor performance than non-musicians because of their extensive musical experience. In a previous study, we established that loudness discrimination acuity is enhanced when sound is produced by a precise force generation task. In this study, we compared the enhancement effect between experienced pianists and non-musicians. Without the force generation task, loudness discrimination acuity was better in pianists than non-musicians in the condition. However, the force generation task enhanced loudness discrimination acuity similarly in both pianists and non-musicians. The reaction time was also reduced with the force control task, but only in the non-musician group. The results suggest that the enhancement of loudness discrimination acuity with the precise force generation task is independent of musical experience and is, therefore, a fundamental function in auditory-motor interaction.