The use of proprioception and tactile information in haptic search

Effects of Sensing Tactile Arrays, Shear Force, and Proprioception of Robot on Texture Recognition

In robotics, tactile perception is important for fine control using robot grippers and hands. To effectively incorporate tactile perception in robots, it is essential to understand how humans use mechanoreceptors and proprioceptors to perceive texture. Thus, our study aimed to investigate the impact of tactile sensor arrays, shear force, and the positional information of the robot's end effector on its ability to recognize texture. A deep learning network was employed to classify tactile data from 24 different textures that were explored by a robot. The input values of the deep learning network were modified based on variations in the number of channels of the tactile signal, the arrangement of the tactile sensor, the presence or absence of shear force, and the positional information of the robot. By comparing the accuracy of texture recognition, our analysis revealed that tactile sensor arrays more accurately recognized the texture compared to a single tactile sensor. The utilization of shear force and positional information of the robot resulted in an improved accuracy of texture recognition when using a single tactile sensor. Furthermore, an equal number of sensors placed in a vertical arrangement led to a more accurate distinction of textures during exploration when compared to sensors placed in a horizontal arrangement. The results of this study indicate that the implementation of a tactile sensor array should be prioritized over a single sensor for enhanced accuracy in tactile sensing, and the use of integrated data should be considered for single tactile sensing.

Different flooring surfaces affect infants' crawling performance

This study assessed the influence of different types of flooring on infants' crawling motion patterns and performance. Each participating infant (range: 8.7-12.4 months) was encouraged to crawl on a tatami mat made of woven straw as well as other flooring types such as hardwood, carpet, and joint mat. Material tests were conducted to quantify the friction and shock absorption of the flooring. A three-dimensional motion capture system was used to measure spatiotemporal and kinematic variables during hands-and-knees crawling. An increased crawling rate was associated with a faster cadence of cyclic arm movements, but not with crawling stride length. Hardwood flooring had a significantly lower crawling rate and longer duration of hand-floor contact than tatami, while the crawling stride length and range of motion of joint movements were hardly affected by flooring type. The results of this study suggest a drawback of hardwood flooring in terms of infants' effective quadrupedal locomotion.

On the Efficiency of Haptic Based Object Identification: Determining Where to Grasp to Get the Most Distinguishing Information

Haptic perception is one of the key modalities in obtaining physical information of objects and in object identification. Most existing literature focused on improving the accuracy of identification algorithms with less attention paid to the efficiency. This work aims to investigate the efficiency of haptic object identification to reduce the number of grasps required to correctly identify an object out of a given object set. Thus, in a case where multiple grasps are required to characterise an object, the proposed algorithm seeks to determine where the next grasp should be on the object to obtain the most amount of distinguishing information. As such, the paper proposes the construction of the object description that preserves the association of the spatial information and the haptic information on the object. A clustering technique is employed both to construct the description of the object in a data set and for the identification process. An information gain (IG) based method is then employed to determine which pose would yield the most distinguishing information among the remaining possible candidates in the object set to improve the efficiency of the identification process. This proposed algorithm is validated experimentally. A Reflex TakkTile robotic hand with integrated joint displacement and tactile sensors is used to perform both the data collection for the dataset and the object identification procedure. The proposed IG approach was found to require a significantly lower number of grasps to identify the objects compared to a baseline approach where the decision was made by random choice of grasps.

Static and dynamic proprioceptive recognition through vibrotactile stimulation

Objective.Proprioceptive information provides individuals with a sense of our limb's static position and dynamic movement. Impaired or a lack of such feedback can diminish our ability to perform dexterous motions with our biological limbs or assistive devices. Here we seek to determine whether both static and dynamic components of proprioception can be recognized using variation of the spatial and temporal components of vibrotactile feedback.Approach.An array of five vibrotactors was placed on the forearm of each subject. Each tactor was encoded to represent one of the five forearm postures. Vibratory stimulus was elicited to convey the static position and movement of the forearm. Four experimental blocks were performed to test each subject's recognition of a forearm's simulated static position, rotational amplitude, rotational amplitude and direction, and rotational speed.Main results.Our results showed that the subjects were able to perform proprioceptive recognition based on the delivered vibrotactile information. Specifically, rotational amplitude recognition resulted in the highest level of accuracy (99.0%), while the recognition accuracy of the static position and the rotational amplitude-direction was the lowest (91.7% and 90.8%, respectively). Nevertheless, all proprioceptive properties were perceived with >90% accuracy, indicating that the implemented vibrotactile encoding scheme could effectively provide proprioceptive information to the users.Significance.The outcomes suggest that information pertaining to static and dynamic aspects of proprioception can be accurately delivered using an array of vibrotactors. This feedback approach could be used to potentially evaluate the sensorimotor integration processes during human-machine interactions, and to improve sensory feedback in clinical populations with somatosensory impairments.

Artificial tactile and proprioceptive feedback improves performance and confidence on object identification tasks

Somatosensory feedback of the hand is essential for object identification. Without somatosensory feedback, individuals cannot reliably determine the size or compliance of an object. Electrical nerve stimulation can restore localized tactile and proprioceptive feedback with intensity discrimination capability similar to natural sensation. We hypothesized that adding artificial somatosensation improves object recognition accuracy when using a prosthesis. To test this hypothesis, we provided different forms of sensory feedback–tactile, proprioceptive, or both–to two subjects with upper limb loss. The subjects were asked to identify the size or mechanical compliance of different foam blocks placed in the prosthetic hand while visually and audibly blinded. During trials, we did not inform the subjects of their performance, but did ask them about their confidence in correctly identifying objects. Finally, we recorded applied pressures during object interaction. Subjects were free to use any strategy they chose to examine the objects. Object identification was most accurate with both tactile and proprioceptive feedback. The relative importance of each type of feedback, however, depended on object characteristics and task. Sensory feedback increased subject confidence and was directly correlated with accuracy. Subjects applied less pressure to the objects when they had tactile pressure feedback. Artificial somatosensory feedback improves object recognition and the relative importance of tactile versus proprioceptive feedback depends on the test set. We believe this test battery provides an effective means to assess the impact of sensory restoration and the relative contribution of different forms of feedback (tactile vs. kinesthetic) within the neurorehabilitation field.

Scanning movements during haptic search: similarity with fixations during visual search

Finding relevant objects through vision, or visual search, is a crucial function that has received considerable attention in the literature. After decades of research, data suggest that visual fixations are more crucial to understanding how visual search works than are the attributes of stimuli. This idea receives further support from the field of haptic search.

Age-based model for metacarpophalangeal joint proprioception in elderly

Neurological injuries such as stroke can lead to proprioceptive impairment. For an informed diagnosis, prognosis, and treatment planning, it is essential to be able to distinguish between healthy performance and deficits following the neurological injury. Since there is some evidence that proprioception declines with age and stroke occurs predominantly in the elderly population, it is important to create a healthy reference model in this specific age group. However, most studies investigate age effects by comparing young and elderly subjects and do not provide a model within a target age range. Moreover, despite the functional relevance of the hand in activities of daily living, age-based models of distal proprioception are scarce. Here, we present a proprioception model based on the assessment of the metacarpophalangeal joint angle difference threshold in 30 healthy elderly subjects, aged 55-80 years (median: 63, interquartile range: 58-66), using a robotic tool to apply passive flexion-extension movements to the index finger. A two-alternative forced-choice paradigm combined with an adaptive algorithm to define stimulus magnitude was used. The mixed-effects model analysis revealed that aging has a significant, increasing effect on the difference threshold at the metacarpophalangeal joint, whereas other predictors (eg, tested hand or sex) did not show a significant effect. The adaptive algorithm allowed reaching an average assessment duration <15 minutes, making its clinical applicability realistic. This study provides further evidence for an age-related decline in proprioception at the level of the hand. The established age-based model of proprioception in elderly may serve as a reference model for the proprioceptive performance of stroke patients, or of any other patient group with central or peripheral proprioceptive impairments. Furthermore, it demonstrates the potential of such automated robotic tools as a rapid and quantitative assessment to be used in research and clinical settings.

Can a Soft Robotic Probe Use Stiffness Control Like a Human Finger to Improve Efficacy of Haptic Perception?

When humans are asked to palpate a soft tissue to locate a hard nodule, they regulate the stiffness, speed, and force of the finger during examination. If we understand the relationship between these behavioral variables and haptic information gain (transfer entropy) during manual probing, we can improve the efficacy of soft robotic probes for soft tissue palpation, such as in tumor localization in minimally invasive surgery. Here, we recorded the muscle co-contraction activity of the finger using EMG sensors to address the question as to whether joint stiffness control during manual palpation plays an important role in the haptic information gain. To address this question, we used a soft robotic probe with a controllable stiffness joint and a force sensor mounted at the base to represent the function of the tendon in a biological finger. Then, we trained a Markov chain using muscle co-contraction patterns of human subjects, and used it to control the stiffness of the soft robotic probe in the same soft tissue palpation task. The soft robotic experiments showed that haptic information gain about the depth of the hard nodule can be maximized by varying the internal stiffness of the soft probe.

Systematic Movements in Haptic Search: Spirals, Zigzags, and Parallel Sweeps

Movement strategies were investigated in a one-handed haptic search task in which blindfolded-sighted participants used either one or five fingers to find a landmark on an unstructured tactile map. Search theory predicts that systematic strategies, such as spirals, zigzags, and parallel sweeps, should be more prevalent when the searcher's detection radius is small (one finger) and less common when the detection radius is large (five fingers). As predicted, systematic strategies were more common in one-finger than five-finger search. Participants were able to exploit the larger detection radius during five-finger searches to detect targets with any of their fingers, and in one-finger search used more systematic strategies. For the most part, participants' fingers moved together during five-finger search, expanding and moving quickly when looking for search targets/distractors, and contracting and moving slowly when examining search targets/distractors. There was no evidence of fingers being used as spatial anchors or other independent finger movements in five-finger search. While targets could be found with any fingers, examination was primarily accomplished using the index and middle fingers. Overall, these results indicate that untrained sighted participants will use optimal systematic strategies during haptic search, and this behavior is appropriately modulated by detection radius.

Assessment from Functional Perspectives: Using Sensorimotor Control in the Hand as an Outcome Indicator in the Surgical Treatment of Carpal Tunnel Syndrome

To investigate whether sensorimotor control of the hand could be an outcome indicator after carpal tunnel release (CTR), this work examined changes in the results of patients’ manual tactile test (MTT), pinch-holding-up activity (PHUA), two-point discrimination (2PD) and Semmes-Weinstein monofilament (SWM) tests. Participants included 30 predominantly sensory neuropathy CTS patients, as confirmed by a nerve conduction study. The MTT, precision pinch performance in PHUA and traditional sensibility (2PD and SWM) tests were used to examine different aspects of sensory status at the time-points of two weeks before operation and one month post-operation, with a single-blind design. The results showed significant improvements in the sensory function as detected by the 2PD and SWM tests (p<0.001) and sensorimotor function as detected by the MTT (p<0.001) and PHUA test (p<0.05) for patients receiving CTR. The responsiveness of the SWM, MTT and PHUA tests (effect size>0.5, p<0.01) are better than that of two-point discrimination test (effect size<0.5, p<0.001). However, pinch strength saw a decline compared to baseline with a moderate effect sizes (effect size = 0.7, p<0.001). This cohort study found that the MTT and PHUA test can both meet all the statistical criteria with regard to assessing treatment outcomes for patients with CTS. In addition, the results of this work provide clinicians with the information that the sensorimotor functions of the hands, as assessed by MTT and PHUA, are responsive to clinical changes due to CTR.

Effects of using multiple hands and fingers on haptic performance in individuals who are blind

In a previous paper we documented that sighted participants complete haptic tasks faster with two hands and multiple fingers, but that these benefits are task specific. The present study investigates whether these effects are the same for participants who are blind. We compared the performance of fourteen blind participants on seven tactile-map tasks using seven finger conditions. As with sighted participants, blind participants performed all tasks faster with multiple fingers. Line-tracing tasks were faster with fingers added to an already in-use hand, and sometimes when added to the second hand. Local and global search tasks were faster with multiple fingers and two hands. Distance comparison tasks were performed faster with multiple fingers, but not two hands. Lastly, moving in a straight line was faster with multiple fingers. These results reinforce our previous finding that the haptic system performs best when it can exploit the independence of multiple fingers. Furthermore, in every instance that an effect was different between sighted and blind participants, the blind participants benefitted more from two hands or multiple fingers than the sighted participants. This indicates that the blind participants have learned, through experience or training, how to best take advantage of multiple fingers during haptic tasks.

Intra-areal and corticocortical circuits arising in the dysgranular zone of rat primary somatosensory cortex that processes deep somatic input

Somesthesis-guided exploration of the external world requires cortical processing of both cutaneous and proprioceptive information and their integration into motor commands to guide further haptic movement. In the past, attention has been given mostly to the cortical circuits processing cutaneous information for somatic motor integration. By comparison, little has been examined about how cortical circuits are organized for higher order proprioceptive processing. Using the rat cortex as a model, we characterized the intrinsic and corticocortical circuits arising in the major proprioceptive region of the primary somatosensory cortex (SI) that is conventionally referred to as the dysgranular zone (DSZ). We made small injections of biotinylated dextran amine (BDA) as an anterograde tracer in various parts of the DSZ, revealing three distinct principles of its cortical circuit organization. First, its intrinsic circuits extend mainly along the major axis of DSZ to organize multiple patches of interconnections. Second, the central and peripheral regions of DSZ produce differential patterns of intra-areal and corticocortical circuits. Third, the projection fields of DSZ encompass only selective regions of the second somatic (SII), posterior parietal (PPC), and primary motor (MI) cortices. These projection fields are at least partially separated from those of SI cutaneous areas. We hypothesize, based on these observations, that the cortical circuits of DSZ facilitate a modular integration of proprioceptive information along its major axis and disseminate this information to only selective parts of higher order somatic and MI cortices in parallel with cutaneous information.

Grouping by proximity in haptic contour detection

We investigated the applicability of the Gestalt principle of perceptual grouping by proximity in the haptic modality. To do so, we investigated the influence of element proximity on haptic contour detection. In the course of four sessions ten participants performed a haptic contour detection task in which they freely explored a haptic random dot display that contained a contour in 50% of the trials. A contour was defined by a higher density of elements (raised dots), relative to the background surface. Proximity of the contour elements as well as the average proximity of background elements was systematically varied. We hypothesized that if proximity of contour elements influences haptic contour detection, detection will be more likely when contour elements are in closer proximity. This should be irrespective of the ratio with the proximity of the background elements. Results showed indeed that the closer the contour elements were, the higher the detection rates. Moreover, this was the case independent of the contour/background ratio. We conclude that the Gestalt law of proximity applies to haptic contour detection.

Virtual Sectioning and Haptic Exploration of Volumetric Shapes in the Absence of Visual Feedback

The reduced behavior for exploration of volumetric data based on the virtual sectioning concept was compared with the free scanning at the use of the StickGrip linkage-free haptic device. Profiles of the virtual surface were simulated through the penholder displacements in relation to the pen tip of the stylus. One or two geometric shapes (cylinder, trapezoidal prism, ball, and torus) or their halves and the ripple surface were explored in the absence of visual feedback. In the free scanning, the person physically moved the stylus. In the parallel scanning, cross-sectional profiles were generated automatically starting from the location indicated by the stylus. Analysis of the performance of 18 subjects demonstrated that the new haptic visualization and exploration technique allowed to create accurate mental images, to recognize and identify virtual shapes. The mean number of errors was about 2.5% in the free scanning mode and 1.9% and 1.5% in the parallel scanning mode at the playback velocity of 28 mm/s and 42 mm/s, respectively. All participants agreed that the haptic visualization of the 3D virtual surface presented as the cross-sectional slices of the workspace was robust and easy to use. The method was developed for visualization of spatially distributed data collected by sensors.

The effect of feature saliency on haptic subitizing

'Subitizing' refers to fast and error-free numerosity judgment for small (<4) sets of items. For larger sets, the slower process of 'counting' is used. Counting has a serial character, whereas subitizing is believed to have a parallel character. While subitizing was initially found in vision, it has been shown to exist in touch as well. In vision, it has been demonstrated that adding distractor items to a set of target items influences numerosity judgment of the target items. Subitizing was in this case only possible if the distractor item is highly salient among the targets. In the present study, we investigated the effect of adding a distractor item on haptic judgement of a set of target items. To this end, we asked subjects to judge the number of spheres grasped in their hand. Either a cube or an ellipsoid could be added to the set. A cube among spheres has been shown to be highly salient, while an ellipsoid among spheres is not. Our results show that adding a distractor item led to an increase in the response time slopes regardless of the distractor shape. Subitizing was, however, only possible in the case of a salient distractor. This is in agreement with results from vision.

Visually Guided Haptic Search

In this study, we investigate the influence of visual feedback on haptic exploration. A haptic search task was designed in which subjects had to haptically explore a virtual display using a force-feedback device and to determine whether a target was present among distractor items. Although the target was recognizable only haptically, visual feedback of finger position or possible target positions could be given. Our results show that subjects could use visual feedback on possible target positions even in the absence of feedback on finger position. When there was no feedback on possible target locations, subjects scanned the whole display systematically. When feedback on finger position was present, subjects could make well-directed movements back to areas of interest. This was not the case without feedback on finger position, indicating that showing finger position helps to form a spatial representation of the display. In addition, we show that response time models of visual serial search do not generally apply for haptic serial search. Consequently, in teleoperation systems, for instance, it is helpful to show the position of the probe even if visual information on the scene is poor.

Serial search for fingers of the same hand but not for fingers of different hands

In most haptic search tasks, tactile stimuli are presented to the fingers of both hands. In such tasks, the search pattern for some object features, such as the shape of raised line symbols, has been found to be serial. The question is whether this search is serial over all fingers irrespective of the hand, or whether it is serial over the fingers of each hand and parallel over the two hands. To investigate this issue, we determined the speed of static haptic search when two items are presented to two fingers of the same hand and when two items are presented to two fingers of different hands. We compared the results with predictions for parallel and serial search based on the results of a previous study using the same items and a similar task. The results indicate that two fingers of the same hand process information in a serial manner, while two fingers of two different hands process information in parallel. Thus, considering the individual fingers as independent units in haptic search may not be justified, because the hand that they belong to matters.