Ultra-fast selection of grasping points

How visuomotor predictability and task demands affect tactile sensitivity on a moving limb during object interaction in a virtual environment

Tactile sensitivity is decreased on a moving limb compared to the same static limb. This tactile suppression likely reflects an interplay between sensorimotor predictions and sensory feedback. Here, we examined how visuomotor predictability influences tactile suppression. Participants were instructed to hit an approaching virtual object, with the object either never rotating, or always rotating, or rotating unpredictably, prompting related movement adjustments. We probed tactile suppression by delivering a vibrotactile stimulus of varying intensities to the moving hand briefly after the object's rotation and asked participants to indicate if they had felt a vibration. We hypothesized that Unpredictable Rotations would require upweighting of somatosensory feedback from the hand and therefore decrease suppression. Instead, we found stronger suppression with unpredictable than Predictable Rotations. This finding persisted even when visual input from the moving hand was removed and participants had to rely solely on somatosensory feedback of their hand. Importantly, we found a correlation between task demand and tactile suppression in both experiments, indicating that task load can amplify tactile suppression, possibly by downweighting task-irrelevant somatosensory feedback signals to allow for successful task performance when visuomotor task demands are high.

Sensorimotor memories influence movement kinematics but not associated tactile processing

When interacting with objects, we often rely on visual information. However, vision is not always the most reliable sense for determining relevant object properties. For example, when the mass distribution of an object cannot be inferred visually, humans may rely on predictions about the object's dynamics. Such predictions may not only influence motor behavior but also associated processing of movement-related afferent information, leading to reduced tactile sensitivity during movement. We examined whether predictions based on sensorimotor memories influence grasping kinematics and associated tactile processing. Participants lifted an object of unknown mass distribution and reported whether they detected a tactile stimulus on their grasping hand during the lift. In Experiment 1, the mass distribution could change from trial to trial, whereas in Experiment 2, we intermingled longer with shorter parts of constant and variable mass distributions, while also providing implicit or explicit information about the trial structure. In both experiments, participants grasped the object by predictively choosing contact points that would compensate the mass distribution experienced in the previous trial. Tactile suppression during movement, however, was invariant across conditions. These results suggest that predictions based on sensorimotor memories can influence movement kinematics but not associated tactile perception.

Reach Corrections Toward Moving Objects are Faster Than Reach Corrections Toward Instantaneously Switching Targets

Visually guided reaching is a common motor behavior that engages subcortical circuits to mediate rapid corrections. Although these neural mechanisms have evolved for interacting with the physical world, they are often studied in the context of reaching toward virtual targets on a screen. These targets often change position by disappearing from one place reappearing in another instantaneously. In this study, we instructed participants to perform rapid reaches to physical objects that changed position in different ways. In one condition, the objects moved very rapidly from one place to another. In the other condition, illuminated targets instantaneously switched position by being extinguished in one position and illuminating in another. Participants were consistently faster in correcting their reach trajectories when the object moved continuously.

How similar are responses to background motion and target displacements?

When making a goal-directed movement towards a target, our hand follows abrupt background motion. This response resembles that of a shift in the target’s position. Does background motion simply change the position towards which the movement is guided? If so, the response to background motion should resemble the response to a target displacement. To find out whether this is the case, we ran two exploratory studies where we asked participants to hit a moving target at a specified moment. At various times during the hand’s movement, the background could move briefly at one of several speeds, and for various durations. The response to abrupt background motion was larger when the background moved later in the movement and when the background moved faster, in line with known responses to target displacements. The response to a second epoch of background motion was smaller than it would have been if there had been no first epoch, in contrast to responses to multiple target displacements. If the background was already moving before the target appeared, the hand even moved in the opposite direction. Thus, the response to background motion and that to a target displacement are clearly not identical, but they do share several features.

Having several options does not increase the time it takes to make a movement to an adequate end point

Throughout the day, people constantly make choices such as where to direct their gaze or place their foot. When making such movement choices, there are usually multiple acceptable options, although some are more advantageous than others. How much time does it take to make such choices and to what extent is the most advantageous option chosen from the available alternatives? To find out, we asked participants to collect points by tapping on any of several targets with their index finger. It did not take participants more time to direct their movements to an advantageous target when there were more options. Participants chose targets that were advantageous because they were easier to reach. Targets could be easier to reach because the finger was already moving in their direction when they appeared, or because they were larger or oriented along the movement direction so that the finger could move faster towards them without missing them. When the target’s colour indicated that it was worth more points they chose it slightly less fast, presumably because it generally takes longer to respond to colour than to respond to attributes such as size. They also chose it less often than they probably should have, presumably because the advantage of choosing it was established arbitrarily. We conclude that having many options does not increase the time it takes to move to an adequate target.

The influences of target size and recent experience on the vigour of adjustments to ongoing movements

People adjust their on-going movements to changes in the environment. It takes about 100 ms to respond to an abrupt change in a target’s position. Does the vigour of such responses depend on the extent to which responding is beneficial? We asked participants to tap on targets that jumped laterally once their finger started to move. In separate blocks of trials the target either remained at the new position so that it was beneficial to respond to the jump, or jumped back almost immediately so that it was disadvantageous to do so. We also varied the target’s size, because a smaller, less vigorous adjustment is enough to place the finger within a larger target. There was a systematic relationship between the vigour of the response and the remaining time until the tap: the shorter the remaining time the more vigorous the response. This relationship did not depend on the target’s size or whether or not the target jumped back. It was already known that the vigour of responses to target jumps depends on the magnitude of the jump and on the time available for adjusting the movement to that jump. We show that the vigour of the response is precisely tuned to the time available for making the required adjustment irrespective of whether responding in this manner is beneficial.

Grasping Assisting Algorithm in Tele-Operated Robotic Gripper

The involvement of Robots and automated machines in different industries has increased drastically in recent years. Part of this revolution is accomplishing tasks previously performed by humans with advanced robots, which would replace the entire human workforce in the future. In some industries the workers are required to complete different operations in hazardous or difficult environments. Operations like these could be replaced with the use of tele-operated systems that have the capability of grasping objects in their surroundings, thus abandoning the need for the physical presence of the human operator at the area while still allowing control. In this research our goal is to create an assisting system that would improve the grasping of a human operator using a tele-operated robotic gripper and arm, while advising the operator but not forcing a solution. For a given set of objects we computed the optimal grasp to be achieved by the gripper, based on two grasp quality measures of our choosing (namely power grasp and precision grasp). We then tested the performance of different human subjects who tried to grasp the different objects with the tele-operated system, while comparing their success to unassisted and assisted grasping. Our goal is to create an assisting algorithm that would compute optimal grasps and might be integrated into a complete, state-of-the-art tele-operated system.

Friction is preferred over grasp configuration in precision grip grasping

How humans visually select where to grasp an object depends on many factors, including grasp stability and preferred grasp configuration. We examined how endpoints are selected when these two factors are brought into conflict: Do people favor stable grasps or do they prefer their natural grasp configurations? Participants reached to grasp one of three cuboids oriented so that its two corners were either aligned with, or rotated away from, each individual's natural grasp axis (NGA). All objects were made of brass (mass: 420 g), but the surfaces of their sides were manipulated to alter friction: 1) all-brass, 2) two opposing sides covered with wood, and the other two remained of brass, or 3) two opposing sides covered with sandpaper, and the two remaining brass sides smeared with Vaseline. Grasps were evaluated as either clockwise (thumb to the left of finger in frontal plane) or counterclockwise of the NGA. Grasp endpoints depended on both object orientation and surface material. For the all-brass object, grasps were bimodally distributed in the NGA-aligned condition but predominantly clockwise in the NGA-unaligned condition. These data reflected participants' natural grasp configuration independently of surface material. When grasping objects with different surface materials, endpoint selection changed: Participants sacrificed their usual grasp configuration to choose the more stable object sides. A model in which surface material shifts participants' preferred grip angle proportionally to the perceived friction of the surfaces accounts for our results. Our findings demonstrate that a stable grasp is more important than a biomechanically comfortable grasp configuration.NEW & NOTEWORTHY When grasping an object, humans can place their fingers at several positions on its surface. The selection of these endpoints depends on many factors, with two of the most important being grasp stability and grasp configuration. We put these two factors in conflict and examine which is considered more important. Our results highlight that humans are not reluctant to adopt unusual grasp configurations to satisfy grasp stability.

Anticipatory grasping control modulates somatosensory perception

Somatosensory perception is hampered on the moving limb during a goal-directed movement. This somatosensory suppression is mostly attributed to a forward model that predicts future states of the system based on the established motor command. Here, we examined whether and how this suppression is modulated by the predictability of object features important for controlling a grasping movement. Participants reached to grasp an object between thumb and index finger and then lifted it as straight as possible. Objects with symmetric or asymmetric mass distributions were presented either in a blocked or random manner, so that the object's mass distribution could be anticipated or not. At the moment of object contact, a brief vibrotactile stimulus of varying intensities was presented on the dorsal part of the moving index finger. Participants had to report whether they detected the stimulus. When the object's mass distribution was predictable, contact points with the object were modulated to the object's centre of mass. This modulation contributed to a minimized resultant object roll during lifting. When the object's mass distribution was unpredictable, participants chose a default grasping configuration, resulting in greater object roll for asymmetric mass distributions. Somatosensory perception was hampered when grasping both types of objects compared to baseline (no-movement). Importantly, somatosensory suppression was stronger when participants could predict the object's mass distribution. We suggest that the strength of somatosensory suppression depends on the predictability of movement-relevant object features.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

Highs and Lows in Motor Control Development

Motor control is classically described as relying on two components: anticipatory control (feedforward processing) and online control (feedback processing). Here we aimed to unveil the developmental steps of both feedback and feedforward control in 5-10 years old children, using a simple and ecological task. We manipulated object's weight in a reach-to-displace paradigm. When the weight was known before lifting it, anticipatory processes were quantifiable during the reaching phase. Conversely, an unknown weight triggered online corrections during the displacing phase. Movement kinematics revealed that children anticipate this objet property as young as 5 y-o. This anticipation becomes adequate around 7 y-o and is paralleled by poor online corrections. This simple yet relevant paradigm should allow quantifying deviations from neurotypical patterns in disorders of motor control.

Does visuomotor adaptation contribute to illusion-resistant grasping?

Do illusory distortions of perceived object size influence how wide the hand is opened during a grasping movement? Many studies on this question have reported illusion-resistant grasping, but this finding has been contradicted by other studies showing that grasping movements and perceptual judgments are equally susceptible. One largely unexplored explanation for these contradictions is that illusion effects on grasping can be reduced with repeated movements. Using a visuomotor adaptation paradigm, we investigated whether an adaptation model could predict the time course of Ponzo illusion effects on grasping. Participants performed a series of trials in which they viewed a thin wooden target, manually reported an estimate of the target's length, then reached to grasp the target. Manual size estimates (MSEs) were clearly biased by the illusion, but maximum grip apertures (MGAs) of grasping movements were consistently accurate. Illusion-resistant MGAs were observed immediately upon presentation of the illusion, so there was no decrement in susceptibility for the adaptation model to explain. To determine whether online corrections based on visual feedback could have produced illusion-resistant MGAs, we performed an exploratory post hoc analysis of movement trajectories. Early portions of the illusion effect profile evolved as if they were biased by the illusion to the same magnitude as the perceptual responses (MSEs), but this bias was attenuated prior to the MGA. Overall, this preregistered study demonstrated that visuomotor adaptation of grasping is not the primary source of illusion resistance in closed-loop grasping.

Continuously updating one’s predictions underlies successful interception

This paper reviews our understanding of the interception of moving objects. Interception is a demanding task that requires both spatial and temporal precision. The required precision must be achieved on the basis of imprecise and sometimes biased sensory information. We argue that people make precise interceptive movements by continuously adjusting their movements. Initial estimates of how the movement should progress can be quite inaccurate. As the movement evolves, the estimate of how the rest of the movement should progress gradually becomes more reliable as prediction is replaced by sensory information about the progress of the movement. The improvement is particularly important when things do not progress as anticipated. Constantly adjusting one’s estimate of how the movement should progress combines the opportunity to move in a way that one anticipates will best meet the task demands with correcting for any errors in such anticipation. The fact that the ongoing movement might have to be adjusted can be considered when determining how to move, and any systematic anticipation errors can be corrected on the basis of the outcome of earlier actions.

How removing visual information affects grasping movements

Our interaction with objects is facilitated by the availability of visual feedback. Here, we investigate how and when visual feedback affects the way we grasp an object. Based on the main views on grasping (reach-and-grasp and double-pointing views), we designed four experiments to test: (1) whether the availability of visual feedback influences the digits independently, and (2) whether the absence of visual feedback affects the initial part of the movement. Our results show that occluding (part of) the hand's movement path influences the movement trajectory from the beginning. Thus, people consider the available feedback when planning their movements. The influence of the visual feedback depends on which digit is occluded, but its effect is not restricted to the occluded digit. Our findings indicate that the control mechanisms are more complex than those suggested by current views on grasping.

State Estimation for Early Feedback Responses in Reaching: Intramodal or Multimodal?

Humans are highly skilled in controlling their reaching movements, making fast and task-dependent movement corrections to unforeseen perturbations. To guide these corrections, the neural control system requires a continuous, instantaneous estimate of the current state of the arm and body in the world. According to Optimal Feedback Control theory, this estimate is multimodal and constructed based on the integration of forward motor predictions and sensory feedback, such as proprioceptive, visual and vestibular information, modulated by context, and shaped by past experience. But how can a multimodal estimate drive fast movement corrections, given that the involved sensory modalities have different processing delays, different coordinate representations, and different noise levels? We develop the hypothesis that the earliest online movement corrections are based on multiple single modality state estimates rather than one combined multimodal estimate. We review studies that have investigated online multimodal integration for reach control and offer suggestions for experiments to test for the existence of intramodal state estimates. If proven true, the framework of Optimal Feedback Control needs to be extended with a stage of intramodal state estimation, serving to drive short-latency movement corrections.

Unusual prism adaptation reveals how grasping is controlled

There are three main theories on how human grasping movements are controlled. Two of them state that grip aperture and the movement of the hand are controlled. They differ in whether the wrist or the thumb of the hand is controlled. We have proposed a third theory, which states that grasping is a combination of two goal-directed single-digit movements, each directed at a specific position on the object. In this study, we test predictions based on each of the theories by examining the transfer of prism adaptation during single-digit pointing movements to grasping movements. We show that adaptation acquired during single-digit movements transfers to the hand opening when subsequently grasping objects, leaving the movement of the hand unaffected. Our results provide strong evidence for our theory that grasping with the thumb and index finger is based on a combination of two goal-directed single-digit movements.

Effects of material properties and object orientation on precision grip kinematics

Successfully picking up and handling objects requires taking into account their physical properties (e.g., material) and position relative to the body. Such features are often inferred by sight, but it remains unclear to what extent observers vary their actions depending on the perceived properties. To investigate this, we asked participants to grasp, lift and carry cylinders to a goal location with a precision grip. The cylinders were made of four different materials (Styrofoam, wood, brass and an additional brass cylinder covered with Vaseline) and were presented at six different orientations with respect to the participant (0°, 30°, 60°, 90°, 120°, 150°). Analysis of their grasping kinematics revealed differences in timing and spatial modulation at all stages of the movement that depended on both material and orientation. Object orientation affected the spatial configuration of index finger and thumb during the grasp, but also the timing of handling and transport duration. Material affected the choice of local grasp points and the duration of the movement from the first visual input until release of the object. We find that conditions that make grasping more difficult (orientation with the base pointing toward the participant, high weight and low surface friction) lead to longer durations of individual movement segments and a more careful placement of the fingers on the object.

Automatic adjustments toward unseen visual targets during grasping movements

We investigated whether control of hand movements can be driven by visual information that is not consciously perceived. Subjects performed reach-to-grasp movements toward 2D virtual objects that were projected onto a rigid surface. On perturbed trials, the target object was briefly presented at a different orientation (±20° rotation) or different size (±20 % scaling) during movement. The perturbed objects were presented for 33 ms, followed by a 200-ms mask and reappearance of the original target object. Subjects perceived only the mask and were not aware of the preceding perturbed stimuli. Unperturbed trials were identical except that there was no change in the target object before the mask. Despite being unaware of the brief perturbed stimuli, subjects showed corrective adjustments to their movements: rotation of the grip axis in response to orientation perturbations, and scaling of grip aperture in response to size perturbations. Responses were detectable 250-300 ms after the perturbation onset and began to reduce 250-300 ms after the reappearance of the original target. Our results demonstrate that the visuomotor system can utilize visual information for control of grasping even when this information is not available for conscious perception. We suggest that this dissociation is due to different temporal resolution of visual processing mechanisms underlying conscious perception and control of actions.

Fixation Biases towards the Index Finger in Almost-Natural Grasping

We use visual information to guide our grasping movements. When grasping an object with a precision grip, the two digits need to reach two different positions more or less simultaneously, but the eyes can only be directed to one position at a time. Several studies that have examined eye movements in grasping have found that people tend to direct their gaze near where their index finger will contact the object. Here we aimed at better understanding why people do so by asking participants to lift an object off a horizontal surface. They were to grasp the object with a precision grip while movements of their hand, eye and head were recorded. We confirmed that people tend to look closer to positions that a digit needs to reach more accurately. Moreover, we show that where they look as they reach for the object depends on where they were looking before, presumably because they try to minimize the time during which the eyes are moving so fast that no new visual information is acquired. Most importantly, we confirmed that people have a bias to direct gaze towards the index finger’s contact point rather than towards that of the thumb. In our study, this cannot be explained by the index finger contacting the object before the thumb. Instead, it appears to be because the index finger moves to a position that is hidden behind the object that is grasped, probably making this the place at which one is most likely to encounter unexpected problems that would benefit from visual guidance. However, this cannot explain the bias that was found in previous studies, where neither contact point was hidden, so it cannot be the only explanation for the bias.

Quickly making the correct choice

In daily life, unconscious choices guide many of our on-going actions. Such choices need to be made quickly, because the options change as the action progresses. We confirmed that people make reasonable choices when they have to quickly decide between two alternatives, and studied the basis of such decisions. The task was to tap with their finger on as many targets as possible within 2 min. A new target appeared after every tap, sometimes accompanied by a second target that was easier to hit. When there was only one target, subjects had to find the right balance between speed and accuracy. When there were two targets, they also had to choose between them. We examined to what extent subjects switched to the target that was easier to hit when it appeared some time after the original one. Subjects generally switched to the easier target whenever doing so would help them hit more targets within the 2-min session. This was so, irrespective of whether the different delays were presented in separate sessions or were interleaved within one session. Whether or not they switched did not depend on how successful they were at hitting the targets on earlier attempts, but it did depend on the position of the finger at the moment that the easy target appeared. We conclude that people have continuous access to reasonable estimates of how long various movement options would take and of how precise the endpoints are likely to be, given the instantaneous circumstances.

How moving backgrounds influence interception

Reaching movements towards an object are continuously guided by visual information about the target and the arm. Such guidance increases precision and allows one to adjust the movement if the target unexpectedly moves. On-going arm movements are also influenced by motion in the surrounding. Fast responses to motion in the surrounding could help cope with moving obstacles and with the consequences of changes in one’s eye orientation and vantage point. To further evaluate how motion in the surrounding influences interceptive movements we asked subjects to tap a moving target when it reached a second, static target. We varied the direction and location of motion in the surrounding, as well as details of the stimuli that are known to influence eye movements. Subjects were most sensitive to motion in the background when such motion was near the targets. Whether or not the eyes were moving, and the direction of the background motion in relation to the direction in which the eyes were moving, had very little influence on the response to the background motion. We conclude that the responses to background motion are driven by motion near the target rather than by a global analysis of the optic flow and its relation with other information about self-motion.