Effect of Dual Tasking on Vibrotactile Feedback Guided Reaching - a Pilot Study

Evaluation of perceived urgency from single-trial EEG data elicited by upper-body vibration feedback using deep learning

Notification systems that convey urgency without adding cognitive burden are crucial in human-computer interaction. Haptic feedback systems, particularly those utilizing vibration feedback, have emerged as a compelling solution, capable of providing desirable levels of urgency depending on the application. High-risk applications require an evaluation of the urgency level elicited during critical notifications. Traditional evaluations of perceived urgency rely on subjective self-reporting and performance metrics, which, while useful, are not real-time and can be distracting from the task at hand. In contrast, EEG technology offers a direct, non-intrusive method of assessing the user's cognitive state. Leveraging deep learning, this study introduces a novel approach to evaluate perceived urgency from single-trial EEG data, induced by vibration stimuli on the upper body, utilizing our newly collected urgency-via-vibration dataset. The proposed model combines a 2D convolutional neural network with a temporal convolutional network to capture spatial and temporal EEG features, outperforming several established EEG models. The proposed model achieves an average classification accuracy of 83% through leave-one-subject-out cross-validation across three urgency classes (not urgent, urgent, and very urgent) from a single trial of EEG data. Furthermore, explainability analysis showed that the prefrontal brain region, followed by the central brain region, are the most influential in predicting the urgency level. A follow-up neural statistical analysis revealed an increase in event-related synchronization (ERS) in the theta frequency band (4-7 Hz) with the increased level of urgency, which is associated with high arousal and attention in the neuroscience literature. A limitation of this study is that the proposed model's performance was tested only the urgency-via-vibration dataset, which may affect the generalizability of the findings.

EEG correlates to perceived urgency elicited by vibration stimulation of the upper body

Conveying information effectively while minimizing user distraction is critical to human-computer interaction. As the proliferation of audio-visual communication pushes human information processing capabilities to the limit, researchers are turning their attention to haptic interfaces. Haptic feedback has the potential to create a desirable sense of urgency that allows users to selectively focus on events/tasks or process presented information with minimal distraction or annoyance. There is a growing interest in understanding the neural mechanisms associated with haptic stimulation. In this study, we aim to investigate the EEG correlates associated with the perceived urgency elicited by vibration stimuli on the upper body using a haptic vest. A total of 31 participants enrolled in this experiment and were exposed to three conditions: no vibration pattern (NVP), urgent vibration pattern (UVP), and very urgent vibration pattern (VUVP). Through self-reporting, participants confirmed that the vibration patterns elicited significantly different levels of perceived urgency (Friedman test, Holm-Bonferroni correction, p < 0.01). Furthermore, neural analysis revealed that the power spectral density of the delta, theta, and alpha frequency bands in the middle central area (C1, Cz, and C2) significantly increased for the UVP and VUVP conditions as compared to the NVP condition (One-way ANOVA test, Holm-Bonferroni correction, p < 0.01). While the perceptual experience of haptic-induced urgency is well studied with self-reporting and behavioral evidence, this is the first effort to evaluate the neural correlates to haptic-induced urgency using EEG. Further research is warranted to identify unique correlates to the cognitive processes associated with urgency from sensory feedback correlates.

Cognitive and Physical Activities Impair Perception of Smartphone Vibrations

Vibration feedback is common in everyday devices, from virtual reality systems to smartphones. However, cognitive and physical activities may impede our ability to sense vibrations from devices. In this study, we develop and characterize a smartphone platform to investigate how a shape-memory task (cognitive activity) and walking (physical activity) impair human perception of smartphone vibrations. We measured how Apple's Core Haptics Framework parameters can be used for haptics research, namely how hapticIntensity modulates amplitudes of 230 Hz vibrations. A 23-person user study found that physical ( ) and cognitive ( p=0.012) activity increase vibration perception thresholds. Cognitive activity also increases vibration response time ( ). This work also introduces a smartphone platform that can be used for out-of-lab vibration perception testing. Researchers can use our smartphone platform and results to design better haptic devices for diverse, unique populations.

Extended training improves the accuracy and efficiency of goal-directed reaching guided by supplemental kinesthetic vibrotactile feedback

Prior studies have shown that the accuracy and efficiency of reaching can be improved using novel sensory interfaces to apply task-specific vibrotactile feedback (VTF) during movement. However, those studies have typically evaluated performance after less than 1 h of training using VTF. Here, we tested the effects of extended training using a specific form of vibrotactile cues-supplemental kinesthetic VTF-on the accuracy and temporal efficiency of goal-directed reaching. Healthy young adults performed planar reaching with VTF encoding of the moving hand's instantaneous position, applied to the non-moving arm. We compared target capture errors and movement times before, during, and after approximately 10 h (20 sessions) of training on the VTF-guided reaching task. Initial performance of VTF-guided reaching showed that people were able to use supplemental VTF to improve reaching accuracy. Performance improvements were retained from one training session to the next. After 20 sessions of training, the accuracy and temporal efficiency of VTF-guided reaching were equivalent to or better than reaches performed with only proprioception. However, hand paths during VTF-guided reaching exhibited a persistent strategy where movements were decomposed into discrete sub-movements along the cardinal axes of the VTF display. We also used a dual-task condition to assess the extent to which performance gains in VTF-guided reaching resist dual-task interference. Dual-tasking capability improved over the 20 sessions, such that the primary VTF-guided reaching and a secondary choice reaction time task were performed with increasing concurrency. Thus, VTF-guided reaching is a learnable skill in young adults, who can achieve levels of accuracy and temporal efficiency equaling or exceeding those observed during movements guided only by proprioception. Future studies are warranted to explore learnability in older adults and patients with proprioceptive deficits, who might benefit from using wearable sensory augmentation technologies to enhance control of arm movements.

Effect of Short-Term Exposure to Supplemental Vibrotactile Kinesthetic Feedback on Goal-Directed Movements after Stroke: A Proof of Concept Case Series

Many survivors of stroke have persistent somatosensory deficits on the contralesional side of their body. Non-invasive supplemental feedback of limb movement could enhance the accuracy and efficiency of actions involving the upper extremity, potentially improving quality of life after stroke. In this proof-of-concept study, we evaluated the feasibility and the immediate effects of providing supplemental kinesthetic feedback to stroke survivors, performing goal-directed actions with the contralesional arm. Three survivors of stroke in the chronic stage of recovery participated in experimental sessions wherein they performed reaching and stabilization tasks with the contralesional arm under different combinations of visual and vibrotactile feedback, which was induced on the ipsilesional arm. Movement kinematics were encoded by a vibrotactile feedback interface in two ways: state feedback—an optimal combination of hand position and velocity; and error feedback—the difference between the actual hand position and its instantaneous target. In each session we evaluated the feedback encoding scheme’s immediate objective utility for improving motor performance as well as its perceived usefulness. All three participants improved their stabilization performance using at least one of the feedback encoding schemes within just one experimental session. Two of the participants also improved reaching performance with one or the other of the encoding schemes. Although the observed beneficial effects were modest in each participant, these preliminary findings show that supplemental vibrotactile kinesthetic feedback can be readily interpreted and exploited to improve reaching and object stabilizing actions performed with the contralesional arm after stroke. These short-term training results motivate a longer multisession training study using personalized vibrotactile feedback as a means to improve the accuracy and efficacy of contralesional arm actions after stroke.

Integrating Tactile Feedback Technologies Into Home-Based Telerehabilitation: Opportunities and Challenges in Light of COVID-19 Pandemic

The COVID-19 pandemic has highlighted the need for advancing the development and implementation of novel means for home-based telerehabilitation in order to enable remote assessment and training for individuals with disabling conditions in need of therapy. While somatosensory input is essential for motor function, to date, most telerehabilitation therapies and technologies focus on assessing and training motor impairments, while the somatosensorial aspect is largely neglected. The integration of tactile devices into home-based rehabilitation practice has the potential to enhance the recovery of sensorimotor impairments and to promote functional gains through practice in an enriched environment with augmented tactile feedback and haptic interactions. In the current review, we outline the clinical approaches for stimulating somatosensation in home-based telerehabilitation and review the existing technologies for conveying mechanical tactile feedback (i.e., vibration, stretch, pressure, and mid-air stimulations). We focus on tactile feedback technologies that can be integrated into home-based practice due to their relatively low cost, compact size, and lightweight. The advantages and opportunities, as well as the long-term challenges and gaps with regards to implementing these technologies into home-based telerehabilitation, are discussed.

Vibration Propagation on the Skin of the Arm

Vibrotactile interfaces are an inexpensive and non-invasive way to provide performance feedback to body-machine interface users. Interfaces for the upper extremity have utilized a multi-channel approach using an array of vibration motors placed on the upper extremity. However, for successful perception of multi-channel vibrotactile feedback on the arm, we need to account for vibration propagation across the skin. If two stimuli are delivered within a small distance, mechanical propagation of vibration can lead to inaccurate perception of the distinct vibrotactile stimuli. This study sought to characterize vibration propagation across the hairy skin of the forearm. We characterized vibration propagation by measuring accelerations at various distances from a source vibration of variable intensities (100–240 Hz). Our results showed that acceleration from the source vibration was present at a distance of 4 cm at intensities >150 Hz. At distances greater than 8 cm from the source, accelerations were reduced to values substantially below vibrotactile discrimination thresholds for all vibration intensities. We conclude that in future applications of vibrotactile interfaces, stimulation sites should be separated by a distance of at least 8 cm to avoid potential interference in vibration perception caused by propagating vibrations.